Variant

Origin & evolution

Comparing arms by random assignment predates software by a century. The randomised controlled trial grew from agricultural and medical statistics, with R. A. Fisher's The Design of Experiments🧫ExperimentValidationA test designed to validate a hypothesisView reference → (1935) establishing randomisation as the basis for causal inference. The web inherited the method wholesale. Treatment and control became "variant A" and "variant B", and the controlled experiment became the default way large product teams decide what to ship.

The hard-won discipline is the one-variable rule. A variant should differ from the control in a single dimension, because a treatment that changes the button colour and the headline and the price leaves you unable to attribute the lift to any of them. Multivariate testing relaxes this deliberately and pays for it in the sample size needed to read each interaction. The clean A/B, two variants apart by one change, stays the workhorse precisely because its result is interpretable.

The subtle failure that matured the field is sample ratio mismatch. A 50/50 split that arrives as 5,000 against 4,982 may look like noise, but a statistically significant deviation from the intended ratio signals broken randomisation, a redirect bug🐛BugProduct SpecificationA defect or unexpected behaviourView reference →, or lost telemetry, any of which invalidates the comparison (VWO). Teams now run a chi-squared SRM check as a gate before trusting any variant's numbers, treating an unexpected allocation as a reason to discard the result rather than interpret it (Microsoft Research).

How it works in practice

A pricing team suspects the annual plan is under-chosen because it is buried below the monthly option. The control variant keeps the existing order. The treatment variant moves annual to the top and adds a "two months free" label. Nothing else changes: same copy, same prices, same checkout.

Traffic splits 50/50 across 12,000 visitors. Before reading the conversion numbers, the team checks the split: 6,040 versus 5,960, a chi-squared p-value of 0.41, no SRM, so the randomisation is sound. The treatment lifts annual-plan selection from 18 to 24 percent with the monthly rate roughly flat, significant at their threshold. Because the two variants differed by exactly one change, the team can attribute the lift to ordering and labelling, and ship the treatment as the new control for the next test.

Variant vs. its neighbours

• Experiment run▶️Experiment RunValidationAn execution instance of an experiment that records actual conditions, observations, and raw results.View reference → is the execution of a test over a window: the variants, the traffic, the dates, the result. A variant is one arm inside that run. The run holds the comparison; the variant is a thing being compared.
• Hypothesis🧪HypothesisValidationA testable belief about a solutionView reference → is the prediction a test exists to check ("moving annual to the top will raise annual selection"). The treatment variant is the embodiment of that prediction; the control is the baseline it is measured against. The variant tests the hypothesis through variant_tests_hypothesisVarianttestsHypothesiscross-domain.
• Growth campaign📢Growth CampaignGrowthA growth-focused campaignView reference → is the larger push that may spin up many variants across several tests toward one objective🏁ObjectiveStrategyA strategic goal (OKR)View reference →. The campaign sets the goal; the variant is the smallest unit that actually gets shown to a user.

In the graph

In the Unified Product Graph, a variant is a leaf in the Business, GTM & Growth region. Three edges fix its meaning: growth_campaign_tests_variantGrowth CampaigntestsVarianthierarchy ties it to the push that created it, experiment_run_tests_variantExperiment RuntestsVariantsemantic ties it to the specific run measuring it, and variant_tests_hypothesisVarianttestsHypothesiscross-domain ties it to the claim it exists to confirm or refute. Holding the variant as its own node, beneath both a run and a hypothesis, is what lets the graph answer "which change did we actually test, and what did we expect from it?": a variant with no hypothesis is a tweak no one can learn from, and the structure makes that gap explicit.