Test Coverage Report

Origin & evolution

Coverage measurement is old. Statement, branch, and path coverage were formalised in software-testing literature through the 1960s and 70s, and the distinctions still hold: line coverage reports which lines executed, branch coverage reports which decision⚖️DecisionStrategyA recorded decision with context, rationale, and consequencesView reference → outcomes🏆OutcomeStrategyA desired business or user outcomeView reference → (the true and false sides of each if) executed, and branch coverage is the stricter bar because a single line can hide an untested path.

The instructive evolution is the critique. Martin Fowler argues that test coverage is a useful tool for finding untested parts of a codebase and of little use as a numeric statement of how good the tests are. He warns that he would be suspicious of anything near 100%, because it smells of tests written to satisfy the number rather than to think about behaviour. This is Goodhart's Law applied to engineering: make coverage a target and people reach it cheaply, writing tests that execute code without asserting anything about it. The field has largely landed on Fowler's reading. Coverage is a gap-finder, read in the gaps rather than the headline figure.

How it works in practice

A service reports 92% line coverage and the team treats it as healthy. A closer read of the report tells a different story. The uncovered 8% is concentrated in the payment-retry and error-handling paths, the exact code most expensive to get wrong, while the well-covered 92% is mostly getters and serialisation. Branch coverage, lower at 71%, reveals that several if statements have only their happy path tested. The team stops chasing the top-line number and uses the report as a map: it writes tests for the retry logic and the untested error branches, accepting that the line percentage barely moves. Three months later a production incident🚨IncidentDevOps & PlatformA production incidentView reference → traces to exactly the kind of error branch they had filled in, and the test catches the regression. The report did its job🔨JobUserJob To Be Done: what the user is trying to accomplishView reference → by showing where the tests were not, not by being high.

Test coverage report vs. its neighbours

• Test suite is the set of tests that run; the coverage report is the measurement of what that suite reached. A suite can be large and still cover little of the risky code, which is precisely what the report exposes.
• Service is the unit being measured; the report scopes coverage to it. Splitting reports per service stops a monolith's high average from hiding one component that is barely tested.
• Metric🔢MetricStrategyA unified metric that measures progress, health, or behaviour across the productView reference → is the general category, and coverage is a metric that fails the gaming test badly. Treating it as a target invites the assertion-free test; treating it as a diagnostic keeps it honest. The line between the two is the whole lesson.

In the graph

In the Unified Product Graph, a test coverage report sits in the testing region as a measurement node. It attaches to what it measures through product_measured_by_test_coverage_reportProductmeasured byTest Coverage Reporthierarchy and test_suite_measured_by_test_coverage_reportTest Suitemeasured byTest Coverage Reporthierarchy, and it points at the code it covers through test_coverage_report_covers_serviceTest Coverage ReportcoversServicecross-domain. Keeping the report distinct from the test_suite📂Test SuiteQuality AssuranceA suite of related testsView reference → records coverage as an observation👁️ObservationUser ResearchA specific behaviour or statement observedView reference → with a target, so a team can track whether a service's coverage is rising or quietly eroding as the codebase grows. Tying it to service⚙️ServiceEngineeringA deployable service or microserviceView reference → rather than to the product as a whole keeps the measurement honest at the level where untested risk🎲RiskComplianceA risk to the product or businessView reference → actually lives.