Data Lineage

How it works in practice

A revenue figure on an executive dashboard🎛️DashboardData & AnalyticsAn analytics dashboardView reference → reads 12 percent low one morning. With table-level lineage, an analyst learns that five tables feed the dashboard and has to inspect each. With column-level lineage, the trace is sharper: the net_revenue field depends on fx_rate from one upstream table, and that table's last load failed silently overnight. The root cause🪝Root CauseEngineeringAn identified root cause of an issueView reference → is found in minutes instead of an afternoon. The same map run forwards answers the other question: if someone drops fx_rate, which downstream models and dashboards go dark?

Zagni's treatment of dbt shows this forward-trace capability💪CapabilityStrategyAn ability that enables value deliveryView reference → made operational: dbt represents each SQL model as a node in a directed acyclic graph, built automatically from `ref()` and `source()` macro calls, so the tool knows at execution time which downstream models to rebuild or retest when an upstream model changes. The column-level example above is the everyday version of what that DAG encodes.

Data lineage vs. its neighbours

• Data pipeline🪣Data PipelineData & AnalyticsAn automated pipeline for data transformationView reference → is the machinery that moves and transforms data. Lineage is the record of what that machinery did, the graph you read after the fact to understand or debug it.

The lineage obligation in analytical systems has a long pedigree. Kimball and Ross's dimensional modeling methodology — fact tables joined to conformed dimensions shared across subject areas — creates precisely the cross-source dependencies⛓️DependencyTeam & OrganisationA cross-team or system dependencyView reference → that lineage must track: when a conformed dimension (say, a customer record) is updated, every fact table that references it is affected. Their ETL design chapters treat knowing which source system fed which dimension attribute as a prerequisite to any trustworthy load, which is an early formal statement of the lineage principle the body opens with.

• Data quality rule🧹Data Quality RuleData & AnalyticsA data quality validation ruleView reference → tells you a value is wrong. Lineage tells you where it went wrong, by tracing the failing field back to its source. The two pair naturally for root-cause work.
• Table-level vs column-level is the granularity choice within lineage itself. Table-level shows which datasets connect; column-level shows which specific fields depend on the one you are about to change, which is what real impact analysis needs🫀NeedUserA user need, pain, desire, or constraintView reference →.

In the graph

In the Unified Product Graph, data lineage sits in the data and analytics region and ties tightly to data_source🛢️Data SourceData & AnalyticsA data source or integrationView reference →. A source is traced via lineage (data_source_traced_via_data_lineageData Sourcetraced viaData Lineagehierarchy), and lineage both draws from a source (data_lineage_sourced_from_data_sourceData Lineagesourced fromData Sourcecross-domain) and feeds into one (data_lineage_feeds_data_sourceData LineagefeedsData Sourcecross-domain). Those two directional edges are the point: lineage is inherently a path with an upstream and a downstream, so the graph models it as a directed connection between sources rather than a flat label, which is what lets you walk it for impact analysis.