← Back to Glossary
Research and DiscoveryA

A/B Testing

Definition

A controlled experiment in which two or more variants of a page, feature, or flow are shown to different user segments at the same time to determine which variant performs better against a defined metric. Statistical significance is required before drawing conclusions. PMs use A/B tests to make data-informed decisions about design changes, pricing, copy, and feature rollouts rather than relying on opinion.

Why It Matters for Product Managers

Understanding a/b testing is critical for product managers because it directly influences how teams prioritize work, measure progress, and deliver value to users. PMs use A/B tests to make data-informed decisions about design changes, pricing, copy, and feature rollouts rather than relying on opinion. Without a clear grasp of this concept, PMs risk making decisions based on assumptions rather than evidence, which can lead to wasted engineering effort and missed market opportunities.

How It Works in Practice

In practice, product teams apply this technique during the discovery phase of product development:

  • Plan — Define the research question and decide on the appropriate method, sample size, and timeline.
  • Recruit — Identify and schedule participants who represent the target user segment.
  • Execute — Conduct the research following the methodology, capturing both qualitative observations and quantitative data.
  • Synthesize — Analyze findings, identify patterns, and translate insights into actionable recommendations for the product team.

    • Effective use of a/b testing prevents teams from building features based on assumptions and ensures that investment flows toward validated user needs.

    Common Pitfalls

  • Running the technique without a clear hypothesis or research question, which leads to unfocused results.
  • Relying on a single research method instead of triangulating with complementary approaches.
  • Letting stakeholder opinions override what the data and user feedback actually reveal.

    • Related Concepts

    To deepen your understanding, explore the related concept: Feature Flag.

    Related Terms

    Feature Flag

    Frequently Asked Questions

    What is a/b testing in product management?+
    A controlled experiment in which two or more variants of a page, feature, or flow are shown to different user segments at the same time to determine which variant performs better against a defined metric. Product managers use this concept to make more informed decisions and deliver better outcomes for users and the business.
    When should a product team use a/b testing?+
    Product teams should use a/b testing during the discovery phase when they need to validate assumptions, understand user behavior, or test demand for a new feature. It is most effective when combined with complementary research methods to triangulate findings.
    Next →AARRR (Pirate Metrics)

    Explore More PM Terms

    Browse our complete glossary of 100+ product management terms.

    Full GlossaryExplore Frameworks