Data Lineage & Dependency Mapping
A practical way to trace where data came from, how it changed, and what depends on it.
Data lineage is sometimes treated as metadata.
For critical data, it is better understood as a dependency and control map.
If we cannot explain where important data came from, what happened to it, and where it goes next, it becomes very difficult to govern that data properly.
A simple lineage model
Source → Ingest → Transform → Store → Serve → Consume → Retain / Delete
This model is deliberately simple.
The technology behind each stage may be very different, but the governance questions remain broadly the same.
At each stage, capture:
- System / Dataset
- Authoritative Source
- Owner
- Classification
- Interface
- Transformation
- Downstream Dependency
- Access Model
- Quality Control
- Retention
- Evidence
- Criticality
- Review Date / Trigger
The aim is not to document every field in every system.
The aim is to understand the important movement, transformation and dependency points well enough to manage risk, change and accountability.
Start with the authoritative source
One of the first questions should be:
Where did this data originally come from?
That may sound obvious, but once information has moved through several applications, integrations, data platforms and reporting layers, the answer can become surprisingly unclear.
A useful lineage record should identify the authoritative source or system of record.
Without that, teams may end up making decisions using copied, stale or transformed versions of data without understanding how they differ from the original.
Document how the data moves
Data can move through many different integration patterns:
- APIs
- Events and messaging
- EDI
- Managed file transfer
- SFTP
- ETL / ELT pipelines
- Database replication
- SaaS integrations
- Manual spreadsheets
The interface itself matters because different patterns introduce different operational and security considerations.
For example, an API may introduce authentication, authorisation and availability dependencies, while a scheduled file transfer may introduce reconciliation and timing risks.
The technology is important, but lineage should capture the business dependency created by that technology.
Transformations matter
The data arriving at a downstream system may not be the same data that left the source.
Transformations can include:
- field mappings
- calculations
- enrichment
- aggregation
- masking
- tokenisation
- de-identification
- currency conversion
- timezone conversion
- identity matching
- data consolidation
- quality corrections
- business-rule application
These changes can affect meaning as well as format.
If a downstream report shows a value that differs from the source, lineage should help explain why.
This is particularly important where data drives financial reporting, regulatory obligations, customer decisions or automated processes.
Classification should follow the data
Classification and lineage are closely connected.
Classification tells us the required control baseline.
Lineage tells us where that control requirement needs to follow.
If a dataset is classified as Confidential or Restricted, copying it into another system should not automatically reduce that requirement.
A practical rule is:
Downstream data should inherit the source classification unless there is a documented and approved reason to change it.
There may be legitimate reasons for reclassification, such as verified aggregation, masking, tokenisation, de-identification or removal of sensitive fields.
But the change should be deliberate rather than accidental.
Downstream dependency is often the missing piece
Lineage should not stop once data reaches a database or data warehouse.
It should continue into the systems and processes that rely on it.
That may include:
- operational applications
- reports and dashboards
- customer-facing services
- APIs
- exported files
- third parties
- automated business processes
- machine-learning models
- AI agents
- vector stores
This becomes especially important during system changes.
If a source system is retired, an interface changes, or a data field is modified, good lineage helps answer:
What else will this break?
That makes lineage useful for architecture, change management and operational resilience — not just data governance.
Quality and reconciliation belong in lineage
Knowing where data moves is only part of the picture.
For critical data, I also want to know how we confirm that the movement worked correctly.
Typical controls might include:
- record-count reconciliation
- financial-total reconciliation
- duplicate detection
- validation rules
- exception reporting
- completeness checks
- timeliness checks
- failed-interface monitoring
This links lineage directly to data quality.
Instead of saying only:
System A sends data to System B.
A stronger lineage record can explain:
System A sends the daily transaction dataset to System B through an API, and the receiving process reconciles record counts and financial totals before processing completes.
That is much more useful operationally.
Access and ownership should travel with the record
A lineage map should also help answer:
- Who owns this data?
- Who supports the platform?
- Who is allowed to access it?
- Which service accounts or integrations can read or modify it?
- Who approves changes?
- Who accepts an exception?
Ownership without decision rights becomes little more than a title.
Lineage becomes substantially more useful when ownership and access controls are visible alongside the technical flow.
Third-party lineage matters too
Data does not stop being our responsibility just because it leaves our network.
If data is provided to a third party, lineage should record:
- what data was provided
- why it was provided
- the classification
- the transfer method
- the receiving organisation
- applicable contractual controls
- retention expectations
- return or deletion requirements
This becomes particularly important during supplier transitions, contract termination, M&A and divestment activity.
AI extends the lineage boundary
AI introduces another set of dependencies.
A modern lineage path may look like:
Curated Dataset → Embedding / Indexing → Vector Store → RAG / Agent → Model → User Output
Or for model development:
Source → Preparation → Training Dataset → Model Version → Evaluation → Deployment
Questions then include:
- What data was used?
- Where did it originate?
- Was that use permitted?
- How was it transformed?
- Which model or agent consumed it?
- Can the source be updated or deleted?
- How do we know which model version used which dataset?
This is one reason I see lineage becoming increasingly important as organisations expand their AI use.
Different levels of lineage are useful
Not every audience needs the same level of detail.
Business lineage
Focuses on:
- business domain
- ownership
- purpose
- major sources
- major consumers
- critical dependencies
Useful for executives, risk teams and governance forums.
System lineage
Focuses on:
- applications
- interfaces
- integrations
- transformations
- downstream systems
- operational ownership
Useful for architecture, IT operations and change management.
Technical lineage
May extend into:
- tables
- fields
- pipelines
- transformations
- schemas
- model features
- detailed dependencies
Useful where engineering, regulatory requirements or data complexity justify the additional depth.
The important point is to use the level of detail that supports the decision being made.
Lineage also needs a lifecycle
A lineage diagram created during a project and never updated quickly loses value.
Useful review triggers include:
- new integrations
- system migrations
- major application changes
- third-party onboarding
- AI or analytics use
- significant transformation changes
- acquisitions or divestments
- incidents
- regulatory or contractual changes
- system retirement
The goal is not perfect documentation.
The goal is current enough information to support real decisions.
The practical test
For an important dataset, I should be able to ask:
- Where did it come from?
- Who owns it?
- How is it classified?
- How did it get here?
- What happened to it along the way?
- Who can access it?
- How do we know it is correct?
- Where does it go next?
- Who or what depends on it?
- When was this last reviewed?
If those questions cannot be answered, there is probably a governance gap worth addressing.
Final thought
Data lineage is sometimes presented as a specialist data-management activity.
I see it more broadly.
It supports:
- data governance
- architecture
- operational resilience
- incident response
- change management
- security
- privacy
- M&A and divestment
- AI governance
The core principle is simple:
You cannot govern data well if you cannot explain where it came from, what happened to it, and where it goes next.