a/b test

A/B test a landing page: one change, one metric

AA
Aurum Avis Labs Author
6 min read

A/B test a landing page

You A/B test a landing page by showing two versions to comparable visitors at the same time, changing only one element between them, and measuring one primary action you defined before traffic starts. The winner is the variant that moves more strangers toward that action under the same channel and budget, not the headline your team prefers in a meeting.

This is for founders and innovation leads in Switzerland who already have a live page, some targeted traffic, and a specific disagreement to settle. If you have never measured a single action on a page, start with how to validate a startup idea and a smoke test landing page before you split variants.

Control
The version that matches what you run today, or your best current guess before the test.
Variant
The single deliberate change you want evidence for, everything else held constant.
Primary metric
The one rate or count you will use to pick a winner, usually completion of your main form or button.

How to A/B test a landing page

Treat the test like a small study, not a design review.

  1. Write the disagreement in one sentence

    Example shape: «Headline A names the job; headline B names the pain. We expect B to raise form starts from cold LinkedIn traffic.» If both could be true, you are not ready to split.

  2. Pick one primary metric

    Form submit, calendar booking, or first click on the main button. Secondary metrics can inform replays, but only one decides the winner.

  3. Build two URLs or one tool split

    Two static pages with the same layout and different copy are enough. Route half of each ad click to each URL, or use your analytics tool’s experiment feature if setup is clean.

  4. Run both variants together

    Same dates, same budget, same targeting. Sequential weeks confuse seasonality and ad learning.

  5. Log traffic and spend per variant

    UTM parameters or ad-set labels should make it obvious which visits belong where when you reconcile numbers.

  6. Apply the stop rule you wrote first

    End on a fixed calendar date or when each variant has the minimum visits in your rule. Do not keep the test open until a favourite wins.

On an internal service page we once compared three hook headlines with a simple GA4 funnel: one entry message, one downstream action. We do not publish those results as benchmarks.

What an A/B test can show, and what it cannot

  • ShowsRelative preference

    Which of two honest promises moves more people toward the action you chose, under the traffic you bought.

  • ShowsWhere copy fails

    Session recordings on both variants often reveal hesitation on one field even when clicks look similar.

  • Does not showAbsolute demand

    Both variants can lose. A winner among weak results is still weak.

  • Does not showProduct fit

    You tested words on a page, not usage, retention, or payment.

A fake door test asks whether people click as if a product existed. An A/B test assumes the page structure is settled and argues about one lever. Neither replaces a build when the risk is behaviour inside software.

A made-up example: two headlines for a compliance checklist

A made-up example: a founder in Basel sells a downloadable checklist for GDPR-style documentation to Swiss SMEs. The page already earns a steady trickle of form fills from Google search ads. The team disagrees on the hero line. Version A leads with «Document your processing activities in one afternoon». Version B leads with «Stop guessing what regulators expect in your privacy folder». Layout, form, and price mention stay identical. Each ad click lands on one URL or the other with even rotation for three weeks.

They set a rule: if one variant beats the other on form submits by a margin they wrote down, they retire the loser and only then test the button label. After three weeks, B wins on submit rate but total submits are only slightly above the prior month. Session replays show many visitors on both pages stop at the industry dropdown. They keep B, fix the form, and schedule a new test on the dropdown copy instead of doubling ad spend on a headline that merely beat a weaker headline.

Who should run a page A/B test, and who should skip

Who this is for

  • Teams with two concrete copy options and enough parallel traffic to apply a written stop rule
  • Founders who already ran a smoke test and need to choose between messages that both sound plausible
  • Innovation leads who must show the board a controlled comparison, not another opinion poll

Who should skip or wait

  • Pages that still lack a baseline action count from real strangers
  • Tests that would change layout, offer, and audience at the same time
  • Marketplaces or two-sided products where one page cannot stand in for both sides

Is an A/B test the same as multivariate testing?

No. Multivariate designs change several elements at once. They need more traffic and more discipline. For most early-stage landing pages, one change between two variants is enough.

How long should the test run?

Long enough to meet the rule you wrote before launch, often two to four weeks for small ad budgets. There is no universal conversion figure worth citing here; low traffic with clear abandonment on one field is still useful.

When the landing page package fits

The landing page package fits when you want a measured demand page with GA4 and Microsoft Clarity, code you own, and a stable base for honest A/B splits, but you are not ready for a twelve-week product validation build. It is the right scale while the open question is still which message moves strangers, and you are willing to stop or pivot when both variants underperform. It does not replace usage data from a deployed app. If strangers already complete your action and the bottleneck is behaviour inside the product, move to a build conversation on the contact page instead of another headline round.

a/b test landing page startup validation conversion
AA

Written by

Aurum Avis Labs

Builds and ships at Aurum Avis Labs. Writes here about what we learn working with founders and SMEs in the DACH region.

Customize your cookie preferences. Essential cookies cannot be disabled as they are required for the website to function properly.

Essential Cookies

Required for basic website functionality, security, user authentication, and error tracking.

Always active

Analytics Cookies

Help us understand how visitors interact with our website to improve user experience. Includes Google Analytics and Microsoft Clarity session recordings.

Marketing Cookies

Used to track visitors across websites to display relevant and engaging advertisements.