The Data Maturity, Quality and Lineage Assessment: Establishing What Can Be Trusted for Which Decision

Before relying on data, ask whether it is fit for the decision you need to make. Trace it from source to use, record its limits and name who will resolve gaps. An overall maturity score cannot answer those questions for you.

1. The Market Connection Context

The recognisable condition is an executive meeting where teams dispute a number, trace it to different extracts, or discover that a dashboard field changed meaning upstream. The customer or operating decision still proceeds, but nobody can state the source population, transformation chain, quality rule, access authority, limitation or owner.

Customer or operating event
→ captured source record
→ controlled transformations and joins
→ decision dataset
→ interpreted signal
→ named human decision
→ observed consequence and review

Poor fitness may expose revenue, margin, cost, capacity, retention, risk, investment efficiency or service value, but any amount is REQUIRES VALIDATION until population, period, unit, baseline and causal boundary are accepted. Quality is use-specific: a dataset may support directional service triage but not customer eligibility, statutory reporting or a financial decision. Completeness cannot compensate for stale data; a valid format does not prove accuracy; lineage does not prove fitness.

By [decision date], the CDO or named decision owner will classify each decision-critical dataset and derived field as TRUST WITHIN LIMIT, VALIDATE, REMEDIATE, RESTRICT, REPLACE, DEFER or STOP USE, with an owner, evidence boundary, control and next review. No composite maturity score makes this decision.

Use the assessment for a named decision or use case where source identity, transformation, quality, access or ownership is disputed. Do not use it as the enterprise-wide audit under The Measurement and Decision Intelligence Audit: An Executive Standard for Trustworthy Action, strategy design under The Data Strategy and Investment Portfolio: A Leadership Playbook from Decisions to Architecture, detailed metric and semantic contract under The Metric Contract and Semantic Model Standard: One Definition, Known Limits and Accountable Ownership, a privacy/security/legal assurance opinion, or a platform implementation. Scope includes critical data elements, collection, transformations, joins, exclusions, quality rules, access, retention, owners, downstream interpretation and remediation options. Expected outputs are a charter, lineage register, quality test ledger, trust disposition and review, not perfect data, a certification or a guaranteed outcome.

The UK Government Data Quality Framework defines quality as fitness for purpose and advises quality assessment across the lifecycle rather than one-size-fits-all assurance (Government Data Quality Framework, VERIFIED official guidance). Its companion guidance says rules should align with user needs and business objectives (Data Quality Framework guidance, VERIFIED). W3C PROV-O provides a standard way to represent provenance entities, activities and agents (W3C PROV-O, VERIFIED recommendation). NIST’s Privacy Framework is a voluntary tool for managing privacy risk across data processing (NIST Privacy Framework, VERIFIED official source). These support the controls; the assessment sequence is an Abrasim method, PROPOSED.

2. The Leadership vs. Delegation Boundary

The reader recognises that each use case needs source-to-decision lineage, quality tests, ownership, access and remediation economics, not a universal maturity badge.

An internal team can lead when it has source and transformation access, can challenge producers and consumers, protects sensitive data, and can stop unsafe use. Independent leadership is proportionate when owners dispute definitions, a platform or programme sponsor controls the evidence, regulated uses are implicated, or remediation investment crosses budgets. A data engineer may map technical lineage; a statistician may test sampling; qualified privacy, security, legal and records authorities decide within their mandates. None replaces the business decision owner.

Pause for unauthorised access, uncontrolled personal data, missing source owners, evidence tampering, an unchallengeable preferred conclusion or a decision whose harm cannot be contained. Ask providers how they bound the use case, trace manual and technical transformations, select fit-for-use rules, preserve uncertainty, test bias and representativeness, price remediation without inventing benefits, and transfer ownership.

3. The Step-by-Step Execution Standard

Step 01: Charter one data-supported decision

Purpose and decision served Bound fitness to one important decision and use.

Accountable owner CDO with the business decision owner.

Delegated lead and contributors Assessment lead, data producer, analyst, process owner and control authorities.

Preconditions, inputs and access Named decision, population, period, decision date and known data products.

Actions

1. State the decision, consequence and evidence threshold.
2. Define in-scope datasets, fields, uses, exclusions and dispositions.
3. Record assumptions, safeguards, conflicts and stop routes.

Evidence to retain Charter, use-case boundary and assumption log.

Decision or scoring rule Proceed only when fitness can be judged for a named use.

Output and template Data-use charter and authority register.

Acceptance gate PROCEED, CHANGE, VALIDATE or STOP.

Leadership checkpoint Decision owner accepts consequences of incorrect or unavailable data.

Human control point CDO approves scope; qualified owners approve sensitive use.

Failure modes and safeguards Enterprise scoring is replaced by use-specific boundaries.

Change, stop or escalation condition Stop if no accountable decision or lawful purpose exists.

Handoff to the next step The charter sets authority and evidence access.

Step 02: Establish ownership, access and use controls

Purpose and decision served Make source, transformation, decision and control accountability explicit.

Accountable owner CDO.

Delegated lead and contributors Data steward, platform owner, privacy, security, records and legal representatives.

Preconditions, inputs and access Charter, data catalogue, policies and known access paths.

Actions

1. Name business, source, technical, quality and use owners.
2. Approve minimised access, retention, evidence handling and challenge routes.
3. Disclose conflicts and unauthorised copies.

Evidence to retain Authority register, approvals, use restrictions and conflict log.

Decision or scoring rule No data enters assessment without owner, purpose and permitted use.

Output and template Authority rows in the charter register.

Acceptance gate AUTHORISED or PAUSE.

Leadership checkpoint Sponsor resolves missing cross-functional authority.

Human control point Qualified authorities decide privacy, security and legal boundaries.

Failure modes and safeguards Convenient access is not treated as lawful authority.

Change, stop or escalation condition Escalate uncontrolled personal or confidential data immediately.

Handoff to the next step Approved access controls source inventory.

Step 03: Inventory critical data elements

Purpose and decision served Identify the smallest data set that materially influences the decision.

Accountable owner Head of Data.

Delegated lead and contributors Analyst, data steward, source owners and decision users.

Preconditions, inputs and access Decision logic, reports, queries, models and source catalogues.

Actions

1. Trace decision inputs to fields, records and reference data.
2. Mark criticality, population, grain, unit and use.
3. Separate required elements from convenient context.

Evidence to retain Critical-element inventory, definitions and source references.

Decision or scoring rule An element is critical only if its absence or error can change the decision.

Output and template Source-to-decision lineage register.

Acceptance gate IN SCOPE, CONTEXT, VALIDATE or EXCLUDE.

Leadership checkpoint Decision owner confirms material inputs.

Human control point Source owners confirm identity without asserting fitness.

Failure modes and safeguards Dashboard visibility does not define criticality.

Change, stop or escalation condition Re-charter if the decision logic materially expands.

Handoff to the next step Critical elements define lineage depth.

Step 04: Trace lineage from capture to decision

Purpose and decision served Expose how records are created, changed, joined and interpreted.

Accountable owner CIO.

Delegated lead and contributors Data engineers, analysts, process owners and stewards.

Preconditions, inputs and access Element inventory, code, jobs, manual procedures and report logic.

Actions

1. Record source event, entity, actor, system and timestamp semantics.
2. Trace transformations, joins, filters, overrides and manual steps.
3. Connect outputs to interpretation and human decision.

Evidence to retain Versioned lineage, code references, manual controls and gaps.

Decision or scoring rule Lineage is complete when every critical element reaches an authoritative origin or visible gap.

Output and template Completed lineage register.

Acceptance gate TRACEABLE, PARTIAL, UNKNOWN or STOP USE.

Leadership checkpoint CIO and decision owner review opaque transformations.

Human control point Technical owners confirm implementation; business owners confirm meaning.

Failure modes and safeguards Automated lineage does not erase spreadsheets, overrides or interpretation.

Change, stop or escalation condition Restrict use where a critical transformation cannot be reproduced.

Handoff to the next step Lineage informs lifecycle and control review.

Step 05: Examine collection, lifecycle and population fitness

Purpose and decision served Test whether recorded data represents the intended event and population.

Accountable owner Head of Data.

Delegated lead and contributors Research, operations, records, data and domain specialists.

Preconditions, inputs and access Lineage, collection instruments, policies and sampling facts.

Actions

1. Compare intended with observed collection and coverage.
2. Examine exclusions, missing populations, retention and destruction.
3. Record systematic missingness, proxy use and time-window limits.

Evidence to retain Collection method, coverage evidence, bias risks and lifecycle controls.

Decision or scoring rule Do not infer representativeness from record volume.

Output and template Collection and population rows in the lineage register.

Acceptance gate FIT, FIT WITH LIMIT, VALIDATE, RESTRICT or STOP USE.

Leadership checkpoint Decision owner accepts population limitations.

Human control point Domain and privacy authorities review sensitive inference.

Failure modes and safeguards Missing data is not interpreted as absence of the event.

Change, stop or escalation condition Escalate material exclusion of protected or high-risk populations.

Handoff to the next step Accepted use and population set quality rules.

Step 06: Define use-specific quality rules

Purpose and decision served Turn fitness into testable rules rather than a generic grade.

Accountable owner CDO.

Delegated lead and contributors Data steward, analyst, decision user and domain owner.

Preconditions, inputs and access Critical elements, lineage, use consequences and population limits.

Actions

1. Select applicable completeness, uniqueness, consistency, timeliness, validity and accuracy rules.
2. Define numerator, denominator, exclusions and evidence method.
3. Set human-approved use, validation and stop thresholds.

Evidence to retain Rule definitions, rationale, thresholds and exceptions.

Decision or scoring rule Only dimensions material to the use are applied; no average maturity score is created.

Output and template Quality rule and remediation ledger.

Acceptance gate Every critical element has a rule or explicit justified exclusion.

Leadership checkpoint Business decision owner accepts fitness thresholds.

Human control point Data owners cannot lower thresholds unilaterally.

Failure modes and safeguards Easy-to-measure dimensions do not displace material ones.

Change, stop or escalation condition Pause when a rule cannot be tested with permitted evidence.

Handoff to the next step Approved rules enter controlled execution.

Step 07: Execute tests and preserve exceptions

Purpose and decision served Measure actual fitness and variation within the accepted boundary.

Accountable owner Head of Data.

Delegated lead and contributors Data quality analyst, engineers, stewards and source owners.

Preconditions, inputs and access Approved rules, test environment, protected extracts and reproducible code.

Actions

1. Run tests by population, source, period and material variant.
2. Retain counts, samples, code version and failed records safely.
3. Reconcile automated results with domain and source evidence.

Evidence to retain Test results, code, exceptions, samples and reproducibility record.

Decision or scoring rule Aggregate pass rates cannot hide a severe material subgroup failure.

Output and template Test-result rows in the quality ledger.

Acceptance gate PASS FOR USE, CONDITIONAL, REMEDIATE, RESTRICT or STOP USE.

Leadership checkpoint CDO reviews decision-critical failures and uncertainty.

Human control point Domain owners can challenge test interpretation, not alter retained results.

Failure modes and safeguards Cleaning outputs do not overwrite source evidence.

Change, stop or escalation condition Stop decision use when an agreed non-waivable rule fails.

Handoff to the next step Results are reconciled with meaning and access controls.

Step 08: Test semantic, access and interpretation fitness

Purpose and decision served Ensure technically valid data is understood and used within authority.

Accountable owner CDO.

Delegated lead and contributors Business steward, analyst, privacy, security and decision users.

Preconditions, inputs and access Definitions, lineage, test results, access roles and report usage.

Actions

1. Compare source meaning, transformed meaning and user interpretation.
2. Test access, purpose limitation, aggregation and disclosure controls.
3. Record unresolved semantic conflicts for The Metric Contract and Semantic Model Standard: One Definition, Known Limits and Accountable Ownership where needed.

Evidence to retain Meaning reconciliation, access tests, use restrictions and dissent.

Decision or scoring rule Correct values with disputed meaning are not decision-fit.

Output and template Semantic and control rows in the quality ledger.

Acceptance gate UNDERSTOOD AND AUTHORISED, CONDITIONAL, RESTRICT or STOP USE.

Leadership checkpoint Decision owner accepts interpretation limits.

Human control point Qualified authorities decide access and privacy conditions.

Failure modes and safeguards Access permission is not treated as purpose authority.

Change, stop or escalation condition Escalate unauthorised disclosure or material semantic conflict.

Handoff to the next step Quality and control findings enter root-cause analysis.

Step 09: Diagnose causes and remediation options

Purpose and decision served Address the source of material defects rather than repeatedly cleaning outputs.

Accountable owner CDO.

Delegated lead and contributors Assessment lead, source owners, engineers, process owners and finance.

Preconditions, inputs and access Lineage, failed rules, exceptions, controls and ownership facts.

Actions

1. Trace defects to collection, process, definition, transformation, control or ownership conditions.
2. Compare remediate, restrict, replace, tolerate within limit and stop-use options.
3. Record burden ranges, dependencies, risks and evidence gaps without invented benefit.

Evidence to retain Causal claims, options, dissent, burden sources and validation actions.

Decision or scoring rule Prefer the smallest coherent action that changes the governing defect within guardrails.

Output and template Remediation rows in the quality ledger.

Acceptance gate REMEDIATE, VALIDATE, RESTRICT, REPLACE, TOLERATE WITHIN LIMIT or STOP USE.

Leadership checkpoint CDO and business owner review trade-offs and residual exposure.

Human control point Source owners accept operational action; finance validates economic claims.

Failure modes and safeguards Downstream cleansing does not mask an upstream incentive or process cause.

Change, stop or escalation condition Stop remediation design when the use itself is unjustified.

Handoff to the next step Decision-ready findings move to trust reconciliation.

Step 10: Reconcile trust by decision use

Purpose and decision served Produce a bounded trust recommendation without collapsing dimensions into a universal score.

Accountable owner Business decision owner.

Delegated lead and contributors Assessment lead, independent challenger and evidence owners.

Preconditions, inputs and access Lineage, tests, semantic controls, options and dissent.

Actions

1. State each use, critical elements, passed and failed rules and limitations.
2. Test alternative interpretations and consequence of error.
3. Recommend one disposition with conditions and expiry.

Evidence to retain Trust dossier, rejected conclusions, dissent and evidence gaps.

Decision or scoring rule One failed non-waivable rule cannot be offset by unrelated quality strength.

Output and template Decision-ready trust dossier.

Acceptance gate TRUST WITHIN LIMIT, VALIDATE, REMEDIATE, RESTRICT, REPLACE, DEFER or STOP USE.

Leadership checkpoint Decision owner sees uncertainty and harm boundaries.

Human control point Independent challenger can preserve minority conclusions.

Failure modes and safeguards RAG ratings and maturity averages cannot replace rule-level evidence.

Change, stop or escalation condition Return when a gap could change the disposition.

Handoff to the next step The dossier goes to executive decision.

Step 11: Make the executive trust disposition

Purpose and decision served Authorise bounded use, remediation or stop action.

Accountable owner CDO with named business decision owner.

Delegated lead and contributors Assessment lead presents; CIO, data owners and qualified authorities advise.

Preconditions, inputs and access Trust dossier, evidence index, dissent, conditions and ownership.

Actions

1. Review fitness, lineage, controls, alternatives and error consequences.
2. Assign the disposition for each named use.
3. Name owner, resource boundary, gate, review trigger and expiry.

Evidence to retain Decision record, rationale, limits, dissent and approvals.

Decision or scoring rule No use proceeds without owner, permissible purpose and explicit limitations.

Output and template Executive data-trust decision and review register.

Acceptance gate ACCEPT, RETURN, RESTRICT or STOP.

Leadership checkpoint Executive records remediation funding separately.

Human control point Only authorised owners accept business and specialist risk.

Failure modes and safeguards Conditional trust is not rewritten as universal certification.

Change, stop or escalation condition Stop where mandatory control or ownership is missing.

Handoff to the next step Accepted definitions may move to The Metric Contract and Semantic Model Standard: One Definition, Known Limits and Accountable Ownership; remediation moves to operating owners.

Step 12: Transfer ownership and monitor fitness

Purpose and decision served Keep trust conditions current as sources, transformations and uses change.

Accountable owner Head of Data.

Delegated lead and contributors Stewards, engineers, process owners, analysts and decision users.

Preconditions, inputs and access Accepted disposition, artefacts, controls and resourced owners.

Actions

1. Transfer lineage, rules, tests, restrictions and remediation actions.
2. Establish monitoring, evidence expiry and change triggers.
3. Schedule continue, change, re-assess, restrict or stop-use decisions.

Evidence to retain Handover, test schedule, change log and review record.

Decision or scoring rule Handover completes when owners can reproduce tests and decision limits.

Output and template Completed executive decision and review register.

Acceptance gate OPERATING OWNERSHIP ACCEPTED or RETURN.

Leadership checkpoint CDO confirms next decision and escalation route.

Human control point Operating owner may reject an unfunded or unsafe transfer.

Failure modes and safeguards Static documentation does not substitute for monitored lineage and rules.

Change, stop or escalation condition Re-charter after material change to use, population, source, transformation or authority.

Handoff to the next step Operating owners maintain fitness; The Metric Contract and Semantic Model Standard: One Definition, Known Limits and Accountable Ownership governs detailed metric semantics where applicable.

4. The Evidence & Measurement Checklist

Preflight and authority

• One important decision and use are named. Evidence: charter. Owner: CDO.
• Population, period and consequences of error are explicit. Evidence: use boundary. Owner: Decision Owner.
• Source, technical, quality and use owners are named. Evidence: authority register. Owner: Head of Data.
• Sensitive access and permitted purpose are approved. Evidence: access record. Owner: Privacy Authority.
• Conflicts and protected conclusions are disclosed. Evidence: conflict log. Owner: CDO.

Evidence access and quality

• Critical elements trace to authoritative origins or visible gaps. Evidence: lineage register. Owner: Data Steward.
• Manual transformations and overrides are included. Evidence: lineage evidence. Owner: Process Owner.
• Population, grain, period and units accompany test evidence. Evidence: metadata. Owner: Analytics Lead.
• Contradictions and failed records remain retained safely. Evidence: test repository. Owner: Head of Data.
• Missing evidence has an owned validation action. Evidence: gap register. Owner: Assessment Lead.

Method execution

• Collection and population fitness are assessed. Evidence: collection review. Owner: Domain Owner.
• Applicable quality rules have definitions and denominators. Evidence: quality ledger. Owner: Data Steward.
• Tests are reproducible by version and period. Evidence: test code and results. Owner: Data Engineer.
• Meaning is reconciled from source to decision. Evidence: semantic review. Owner: Business Steward.
• Remediation addresses evidenced causes rather than symptoms. Evidence: causal option record. Owner: CDO.

Governance, privacy, security and risk

• Data minimisation and retention controls are active. Evidence: handling plan. Owner: Privacy Authority.
• Access permission and purpose authority are separately evidenced. Evidence: access matrix. Owner: Security Authority.
• High-risk inferences route to qualified review. Evidence: escalation record. Owner: Risk Executive.
• Source evidence is protected from destructive cleaning. Evidence: immutable reference. Owner: CIO.
• Stop-use controls can be applied to downstream consumers. Evidence: restriction procedure. Owner: Head of Data.

Deliverables and acceptance

• Charter, lineage, rules and trust dossier reconcile. Evidence: artefact cross-reference. Owner: Assessment Lead.
• Every trust claim names a use and limitation. Evidence: trust dossier. Owner: CDO.
• No universal maturity score replaces rule evidence. Evidence: acceptance review. Owner: Independent Challenger.
• Rejected and minority conclusions remain visible. Evidence: dissent log. Owner: Decision Owner.
• Remediation options have owners and next gates. Evidence: option register. Owner: Head of Data.

Measurement

• Each measure has numerator, denominator and exclusions. Evidence: measure contract. Owner: Analytics Lead.
• Baselines are evidenced or marked baseline required. Evidence: baseline plan. Owner: Measure Owner.
• Quality dimensions remain use-specific. Evidence: rule-to-use map. Owner: Data Steward.
• Harm and privacy guardrails accompany fitness measures. Evidence: guardrail row. Owner: Risk Executive.
• Attribution and lag limits are stated. Evidence: attribution note. Owner: Decision Owner.

Decision and handover

• The executive disposition names the permitted use. Evidence: decision record. Owner: CDO.
• Resource commitment is separate from trust classification. Evidence: funding record. Owner: CFO.
• Conditions and dissent reach the decision forum. Evidence: decision pack. Owner: Assessment Lead.
• Operating owners accept lineage, rules and monitoring. Evidence: handover. Owner: Head of Data.
• The Metric Contract and Semantic Model Standard: One Definition, Known Limits and Accountable Ownership receives only accepted semantic questions. Evidence: handoff record. Owner: Business Steward.

Review, change and stop controls

• Evidence expiry and review triggers are dated. Evidence: review register. Owner: Data Steward.
• Source or transformation change reopens fitness. Evidence: change rule. Owner: CIO.
• New use requires a new use-specific decision. Evidence: use approval. Owner: CDO.
• Failed non-waivable rules stop the named use. Evidence: stop record. Owner: Decision Owner.
• Retired data products have consumer and retention actions. Evidence: retirement record. Owner: Head of Data.

Topic-specific measurement contracts

Measure A: Critical lineage coverage

Measure B: Use-specific rule conformance

Measure C: Defect recurrence at source

Measure D: Decision use within accepted limit

Measure E: Unauthorised or harmful data use

Evidence classifications used in this guide

• VERIFIED: evidence supports a fact within its recorded use and boundary.
• INFERRED: interpretation remains distinguishable from source facts.
• PROPOSED: a rule, remediation or Abrasim method awaiting acceptance.
• REQUIRES VALIDATION: a material fitness claim or lineage link is unresolved.

Source notes

1. Government Data Quality Framework: UK official guidance used for fitness-for-purpose and lifecycle quality; it is not local assurance; recheck at publication.
2. Government Data Quality Framework guidance: companion UK guidance used for the practical quality-dimension and improvement approach; it does not prove a local assessment; recheck at publication.
3. W3C PROV-O: W3C recommendation used for provenance entities, activities and agents; it does not prove a local lineage; recheck version at publication.
4. NIST Privacy Framework: voluntary framework used for privacy-risk controls; qualified authority determines local applicability; recheck current framework at publication.