Test Suite

Origin & evolution

The grouping of tests into managed suites is as old as automated testing itself, but the modern thinking about how to shape a suite traces to Mike Cohn. He popularised the test pyramid in Succeeding with Agile (2009), arguing for a broad base of fast unit tests, a thinner middle of service tests, and a small cap of slow end-to-end UI tests. The economic logic is simple: tests near the base are cheap to write, quick to run, and pinpoint failures precisely; tests near the top are expensive, slow, and brittle.

Martin Fowler later refined the model in his Test Pyramid bliki entry, and a long debate followed about whether the pyramid still holds. Critics point out that the layer names have aged badly and that fast integration tests now blur the middle. The durable part survived the argument: suite shape is a budget, and inverting the pyramid (lots of slow UI tests, few unit tests) produces a suite that runs for an hour and flakes constantly. Run time is a first-class design constraint🔒ConstraintStrategyA constraint entityView reference →, not an afterthought.

How it works in practice

A team ships a B2B web app with 2,400 tests. The suite breaks down as 2,000 unit tests (90 seconds), 350 service tests (4 minutes), and 50 Selenium UI tests (11 minutes), so a full run takes about 16 minutes on CI. That is already at the edge of tolerable; once a pull-request check passes 15 minutes, developers start context-switching away and stop trusting the signal.

The team splits the suite. Unit and service tests run on every push and gate the merge. The 50 UI tests move to a nightly run plus a pre-release🚢ReleaseProduct SpecificationA shipped version of the productView reference → gate, because their job🔨JobUserJob To Be Done: what the user is trying to accomplishView reference → is guarding the release, not blocking each commit. Six weeks later the per-push wait is under 6 minutes, merge frequency rises, and the UI tests still catch the cross-browser breakages they were written for. The suite did not get smaller; it got organised by how fast feedback needs🫀NeedUserA user need, pain, desire, or constraintView reference → to be.

Test suite vs. its neighbours

• Test case is the atom; a suite is the molecule. A single test case asserts one behaviour. The suite is the curated set, plus the decisions about ordering, parallelism, and what runs when. Reasoning about coverage and run time happens at the suite level, never at the single case.
• Regression test♻️Regression TestQuality AssuranceA regression testView reference → is a role a test plays inside the suite, not a separate collection. Any case that exists to catch a reintroduced defect is a regression test; the suite is where it lives and runs alongside feature⭐FeatureProduct SpecificationA product capability or featureView reference → tests.
• Test result🎰Test ResultQuality AssuranceA test execution resultView reference → is what a suite produces, not what it is. The suite is the durable artefact you maintain across releases; the result is the per-run verdict it emits each time it executes.

In the graph

In the Unified Product Graph, test_suite📂Test SuiteQuality AssuranceA suite of related tests sits in the quality and testing region as the organising entity for verification. A product owns its suites through product_maintains_test_suiteProductmaintainsTest Suitehierarchy, and each suite gathers its members through test_suite_contains_test_caseTest SuitecontainsTest Casehierarchy and test_suite_includes_regression_testTest SuiteincludesRegression Testhierarchy. Manual exploration connects through test_suite_tested_via_qa_sessionTest Suitetested viaQA Sessionhierarchy. Modelling the suite as a first-class node, rather than a flat bag of cases, lets the graph answer the questions that actually decide releases: which features are covered, how the regression set has grown, and where the slow tests cluster.