A/B Testing
A controlled experiment that shows two versions of the same screen to comparable groups of users to find out which one performs better.
Growth & MetricsA/B testing splits live traffic between a control and one variant, then measures a single pre-agreed metric. The point is not to collect opinions but to remove them: whichever version moves the metric wins, regardless of who argued for it. Anything more than one variable at a time and you can no longer say what caused the change.
Most A/B tests fail for arithmetic reasons rather than design ones. Low-traffic products cannot reach significance on a button colour in any reasonable timeframe, so the honest choice is to test structural changes — pricing layout, onboarding order, form length — or to skip testing and use usability testing instead. Testing is a tool for settling expensive disagreements, not for validating every decision.
In practice
A fintech dashboard team disagreed about whether to ask for KYC documents before or after the first deposit. Before: 41% completed signup. After moving KYC behind the deposit step: 58%. The catch — support tickets rose, because some users deposited and then discovered they could not withdraw yet.
Learn more
You may ask
Frequently Asked Questions
How much traffic do I need for an A/B test?
Enough to detect the effect you care about. Detecting a 1% lift needs tens of thousands of sessions per variant; detecting a 20% lift on a signup flow may need only a few hundred. If you cannot estimate the effect size, you are not ready to test.
Can I A/B test a redesign?
You can, but a full redesign changes dozens of variables at once, so a win tells you the new version is better without telling you why. Test a redesign as a whole to decide whether to ship it, then test individual elements afterwards to learn.
Related terms
All terms- Conversion Rate Optimization (CRO)The practice of systematically increasing the share of visitors who complete a desired action, using research and experiments rather than opinion.
- Usability TestingWatching real users attempt real tasks in a product or prototype to find where they struggle, without steering them.
- Conversion FunnelThe ordered set of steps a user passes through toward a goal, measured so you can see where people drop out.
- ActivationThe moment a new user first gets real value from a product — the step that turns a signup into someone who has a reason to come back.
Defined by Mara Last reviewed .
Let's talk about your product.
Happy to look at what you're building and say where design would move the needle.
Contact Us