Test Plan

Origin & evolution

The discipline traces to the lean startup movement's insistence that beliefs be tested cheaply before they are built expensively. The clearest codification arrived with David J. Bland and Alexander Osterwalder's Testing Business Ideas (2019), which cataloged 43 experiment🧫ExperimentValidationA test designed to validate a hypothesisView reference → types organised by cost, time, and strength of evidence🔎EvidenceValidationData supporting or refuting a hypothesisView reference →, each framed as a way to attack a stated hypothesis under desirability, feasibility, or viability.

Bland's earlier work, developed alongside Jeff Gothelf and Josh Seiden, sharpened the targeting logic. Assumptions🤔AssumptionStrategyA belief taken as true that underpins a strategyView reference → mapping ranks a team's beliefs by how risky each one is and how little evidence supports it, so that testing effort lands on the assumption whose failure would sink the whole idea. That assumption became known as the riskiest assumption, and testing it first is the economic case for writing a test plan at all: you spend on evidence where the payoff of being wrong early is highest.

The thinking has settled on a separation that older "test everything" habits blurred. A test plan is scoped to one hypothesis and one decision⚖️DecisionStrategyA recorded decision with context, rationale, and consequencesView reference →. It names success before the test runs, which is the discipline that stops teams from reading whatever result they get as confirmation.

How it works in practice

A subscription🔂SubscriptionSales & RevenueA recurring subscriptionView reference → team believes that solo founders will pay £19 a month for an automated bookkeeping feature⭐FeatureProduct SpecificationA product capability or featureView reference →. That belief is the riskiest assumption on their map: high impact, thin evidence. The test plan reads as follows. Method: a pricing page with a live checkout, driven by a £400 ad spend. Sample: 600 visitors from the solo-founder segment. Success criterion: at least 4% click through to checkout and 1.5% complete payment intent. Decision rule: below 1.5%, the feature is reframed or dropped; at or above, it proceeds to a built prototype📱PrototypeExperience DesignAn interactive mockup for testingView reference →.

The test runs for nine days. Click-through lands at 5.1%, but payment-intent completion stalls at 0.7%. The pre-committed rule does its job🔨JobUserJob To Be Done: what the user is trying to accomplishView reference →: interest is real, willingness to pay at £19 is not. The team learns this for £400 rather than for a quarter of engineering.

Test plan vs. its neighbours

• Experiment plan📐Experiment PlanValidationAn experiment plan describing the hypothesis, setup, success criteria, and methodology before a test runs.View reference → designs the experiment itself: variables, control, how the run is instrumented. A test plan starts from a hypothesis and asks whether to keep believing it. One specifies the apparatus; the other specifies the verdict.
• Research plan🗺️Research PlanValidationA plan for gathering evidence to validate a draft entityView reference → runs a discovery study to understand a problem or a user, with open questions and qualitative analysis. A test plan runs validation against a claim you already hold, with a success threshold set in advance. Research asks "what is going on here"; a test plan asks "is this specific belief true enough to build on".
• Experiment run▶️Experiment RunValidationAn execution instance of an experiment that records actual conditions, observations, and raw results.View reference → is the execution: the actual nine days, the actual 600 visitors, the recorded numbers. The test plan is the document that the run instantiates, written before any data exists.

In the graph

In the Unified Product Graph, test_plan📝Test PlanValidationA structured plan for testing a hypothesis sits in the validation region as the bridge between a belief and its evidence. A hypothesis🧪HypothesisValidationA testable belief about a solutionView reference → connects down to it via hypothesis_planned_via_test_planHypothesisplanned viaTest Planhierarchy, and the plan connects forward to its execution via test_plan_ran_as_experiment_runTest Planran asExperiment Runhierarchy. Both edges are hierarchical, which encodes the real dependency⛓️DependencyTeam & OrganisationA cross-team or system dependencyView reference →: a plan that validates no hypothesis is busywork, and a plan that never ran as an experiment is an intention with no outcome🏆OutcomeStrategyA desired business or user outcomeView reference →. The structure makes the riskiest-assumption discipline queryable, because you can ask which hypotheses still lack a plan and which plans still lack a run.