Tech Brief: Data Lineage

Organizations challenged to turn their data policies into actual processes and procedures are starting to identify tools and technologies to help with implementation and practice.

Data lineage is gaining prominence as a core operational business component of data governance technology architecture because it includes both processes and technology for full-spectrum visibility into the ways data flows across an enterprise.

Its advantages include:

• Operational intelligence
• Business terminology consistency
• Data incident root cause analysis
• Data quality remediation assessment
• Impact analysis
• Performance assessment
• Policy compliance
• Auditability of data pipelines

This technology brief by data management expert David Loshin explores these advantages, as well as some of the common challenges.

It also discusses using a data catalog to capture data asset metadata with a data mapping framework that documents connections between the data assets – including how to automate the data mapping.