Journey Measures Count Activity but Not User Progress

What the journey problem looks like and why it affects the business

The dashboard is full of events, but it cannot show whether users progressed, reached value or did avoidable work. Page views and submissions are counted while the user outcome and business consequence remain unclear.

Agree a small set of measures for task completion, value, effort, quality and business consequence before adding more tracking.

This is a governance and learning decision before it is a documentation problem. The primary reader is Customer Director, Analytics Lead or Product Director. The economic buyer is the customer, product or analytics executive who needs trusted evidence about user progress and service value rather than a larger activity dashboard.

The named owner holds decision authority; specialists contribute evidence within their remit, but contribution does not replace accountable ownership.

Agree the outcome and evidence needed for one journey decision. Instrumentation, semantic architecture and enterprise reporting belong to their specialist standards.

What to expect in the working pack

Start the working pack with the charter. Use stable IDs to connect sources, tests, decisions and review records across its eight sheets. Entries need supporting evidence; the workbook alone cannot prove a conclusion:

• Start here: Read the bounded governance or learning decision, evidence labels, sequence and stop conditions.
• Journey charter: Define the user outcome, start and end, routes, exclusions, consequence and authority.
• Journey map: Connect user tasks, context, channels, systems, handoffs, owners and unresolved questions.
• Evidence register: Separate observed facts and statements from interpretations, inferences, proposals and unknowns.
• Friction and failure log: Record symptoms, affected routes, severity, recurrence, consequences and competing causes.
• Intervention and acceptance: Link a proposed governance, measurement or release change to acceptance, guardrails and rollback.
• Outcome and recovery: Observe progression, value, support, recovery, exceptions, harm and durable learning.
• Review and change log: Record versions, evidence changes, approvals, review triggers, replacement and retirement.

Use the same IDs across the workbook so a reviewer can move from a claim to source, definition, consequence, proposed change, outcome and decision. Seed rows are prompts, not local evidence; replace them with authorised organisation-specific material. Do not enter participant names, contact details, recordings, free-text case notes, AI prompts containing identifiers or unnecessary personal information. Store sensitive material in an approved controlled repository and record only a permitted reference. The workbook does not create consent, lawful authority or publication approval.

What evidence distinguishes the real cause

The visible symptom does not establish the cause. Before changing a measure, release gate, owner, definition or AI workflow, build a dated evidence set that distinguishes observed behaviour from interpretation.

Label every material statement. Use VERIFIED only when a current attributable source supports the statement and permitted use. Use OBSERVED for an event or participant statement without turning it into a population claim. Use INFERRED where evidence supports but does not prove an explanation. Use PROPOSED for a future design or decision. Use UNKNOWN when evidence is missing. Use REQUIRES VALIDATION where testing is required before investment, release, scale or public copy.

Test the three competing explanations in the authoring profile: Activity is used as a proxy for progress without testing the relationship. Measures have no shared definition, owner or permitted decision use. Averages hide a quality, effort or route failure that matters commercially or ethically. Record confirming and contrary cases with equal care. A report, metric, release note, map, generated synthesis or consensus can indicate where to investigate, but none proves why a control works or why a journey improved.

The research frame should cover the whole user problem and the routes people actually use. GOV.UK Service Manual: How to set performance metrics and GOV.UK Service Manual: Map a user's whole problem support end-to-end research and joined-up channel thinking; GOV.UK Service Standard: Solve a whole problem for users supports a clear user-needs frame where relevant. GOV.UK Service Manual: How to set performance metrics gives a primary measurement reference. Preserve interfaces across teams and dependencies instead of hiding them in one report.

Accessibility is part of the journey outcome and acceptance test, not a final compliance tick. The Web Content Accessibility Guidelines 2.2 include requirements relevant to complete processes, errors, interruptions, input assistance, redundant entry and accessible authentication. Test with disabled users and people who need assistance where the decision may affect them. Do not infer universal access from automation alone.

Personal information must follow the organisation's approved privacy and research process. POPIA, the Protection of Personal Information Act (https://www.gov.za/documents/protection-personal-information-act), is the primary South African statutory source. This guide is not legal advice; the Information Officer or qualified adviser retains legal authority. Record purpose, permitted use, access, retention and deletion arrangements before collecting identifiable evidence. Minimise workbook content and use controlled references.

Stop and re-charter when the decision is unnamed, the population cannot be bounded, the outcome is only a proxy, evidence cannot be used lawfully, assisted or excluded users cannot be represented, the owner lacks authority, or the work crosses the scope boundary. Record UNKNOWN when evidence is missing.

How to fix it step by step

Step 1: Define the journey decision

Open Start here and Journey charter. Write the decision as a choice an authorised owner must make, not an aspiration such as improve governance. Define the user outcome, the decision consequence and what evidence would permit action. Begin by testing user progress; do not let a familiar report, release status or map stand in for the governed outcome.

For user progress, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Activity is used as a proxy for progress without testing the relationship. Keep first value separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 1: Use E01 to answer “What evidence establishes user progress?” For user progress, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 2: Collect and label the evidence

Create evidence IDs before interpretation. In Evidence register, record source, date, population, route, context, limitation, confidence, lineage and permitted use. Reconcile research, operational, support, product, analytics and commercial evidence without pasting personal data. Collect successful, assisted, interrupted, failed and contrary cases; a clean report cannot prove that governance is working.

For first value, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Measures have no shared definition, owner or permitted decision use. Keep effort and quality separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 2: Use E02 to answer “What evidence establishes first value?” For first value, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 3: Set the population, routes and observation window

Use Journey charter to set population, routes, start, end, exclusions, observation window, review authority and decision boundary. For this guide, make first value visible rather than letting a dashboard, workflow state or launch date stand in for user progress. Keep unknown prevalence, unknown causality and unobserved routes explicit.

For effort and quality, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Averages hide a quality, effort or route failure that matters commercially or ethically. Keep measure definition separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 3: Use E03 to answer “What evidence establishes effort and quality?” For effort and quality, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 4: Map tasks, handoffs and possible causes

In Journey map, map user tasks and context first, then add channels, systems, handoffs, measures, owners and unresolved questions. Connect each transition to evidence IDs and show where effort and quality changes the user's state or the decision's reliability. In Friction and failure log, describe symptoms separately from causes and record competing explanations.

For measure definition, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Activity is used as a proxy for progress without testing the relationship. Keep decision consequence separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 4: Use E04 to answer “What evidence establishes measure definition?” For measure definition, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 5: Check the business consequences and competing causes

Connect the issue to value, harm, cost, delay, quality, trust, retention, capacity or decision risk only where a defensible source exists. If size or causality is unknown, label it UNKNOWN or REQUIRES VALIDATION. Test whether measure definition changes the decision. Do not treat a trend, definition, AI output or stakeholder consensus as proof.

For decision consequence, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Measures have no shared definition, owner or permitted decision use. Keep user progress separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 5: Use E05 to answer “What evidence establishes decision consequence?” For decision consequence, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 6: Name owners, escalation and recovery routes

In Intervention and acceptance, name the accountable owner, contributors, dependency owners, decision rights, exception route and recovery owner. State what receiving teams must accept and what qualified privacy, accessibility, security or technical authorities must decide. A role without evidence access or power to pause, reject or reverse is not sufficient governance.

Step 7: Agree acceptance, guardrails and rollback tests

Write acceptance before implementation: The owner approves a measure set with observable user progress, value, effort, quality, population, period, source, limitation and decision consequence, with unsupported proxy relationships marked REQUIRES VALIDATION. Add accessibility, privacy, value, harm, support, quality, inclusion, trade-off and rollback guardrails. Use Outcome and recovery to record tests over a stated window. The negative case is: A higher completion count is reported as improvement while users repeat work, require more support and fail to realise the promised outcome.

Step 8: Record the decision and review triggers

Record the human decision and version together. In Review and change log, state proceed, adapt, pause, stop or investigate, approver, effective date, next review, evidence change and reopen or retirement trigger. Approval permits action; it does not turn PROPOSED work into VERIFIED evidence. Observe the result, preserve dissent and capture learning.

Worked example: A hypothetical service reports more completed applications after removing a review step. The measurement contract adds successful first use, correction effort, support demand and route-level quality, revealing that the activity increase did not establish better user progress. This is a hypothetical teaching example, not an Abrasim client result and not evidence of a guaranteed outcome. Its purpose is to show how a bounded governance decision, competing explanations, responsible ownership, recovery, value, harm, trade-off and acceptance tests fit together.

How to verify the journey change

Verification asks whether the governance or learning decision improved the user outcome and control quality under stated conditions, not whether a map or workbook was completed. Check user progress, first value, effort and quality, measure definition, decision consequence. Test this possible cause: activity is used as a proxy for progress without testing the relationship. A generic health signal cannot establish whether it is true. Keep the work within this scope: The outcome and evidence contract for one journey. Instrumentation, semantic architecture and enterprise reporting remain separate. The acceptance condition is that the owner approves a measure set with observable user progress, value, effort, quality, population, period, source, limitation and decision consequence, with unsupported proxy relationships marked REQUIRES VALIDATION. Challenge this contrary case: A higher completion count is reported as improvement while users repeat work, require more support and fail to realise the promised outcome. Review the evidence chain in this order:

1. Recheck population, routes, start, end, exclusions and observation window in Journey charter.
2. Sample IDs from Journey map back to Evidence register, source lineage and permitted-use limits.
3. Confirm severe items in Friction and failure log have an owner, disposition, escalation and recovery route.
4. Execute the acceptance, accessibility, privacy, value, outcome, harm, support, inclusion, negative-case, trade-off and rollback tests in Intervention and acceptance and Outcome and recovery.
5. Observe progression, quality, first value, assistance, recovery, exceptions, support demand and adverse outcomes for the agreed window.
6. Reconcile route-level and contrary outcomes; do not use an aggregate to hide material variation or definition drift.
7. Ask the authorised owner to decide proceed, adapt, pause, stop or investigate.
8. Record evidence changes, decision, approver, review date and reopen or retirement trigger in Review and change log.

Use evidence appropriate to the decision. For user progress, reconcile research, operational, analytics, reports, support and commercial sources with their population, period and limitation. Check first value, effort and quality and measure definition and decision consequence against the agreed acceptance conditions. Verification is incomplete when only the easiest route was tested, personal data was used beyond purpose, accessibility was checked only by automation, downstream ownership was absent, or the window ends before value or learning can be observed.

Use this evidence with AI safely. AI may organise authorised, minimised evidence, surface contradictions and draft questions. It must not invent users, prevalence, outcomes or legal conclusions, infer sensitive attributes, make the final decision or receive personal or confidential data outside an approved environment. The NIST AI Risk Management Framework (https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10) is a primary risk reference, not a substitute for local authority. Remove identifiers and use controlled references. Use this prompt:

Role: Journey governance and evidence reviewer. Decision: Define measures that connect task, value, quality, effort and consequence. Scope: The outcome and evidence contract for one journey; instrumentation, semantic architecture and enterprise reporting remain separate. Focus: user progress; first value; effort and quality; measure definition; decision consequence. Evidence questions: E01: What evidence establishes user progress | E02: What evidence establishes first value | E03: What evidence establishes effort and quality | E04: What evidence establishes measure definition | E05: What evidence establishes decision consequence. Competing explanations: Activity is used as a proxy for progress without testing the relationship | Measures have no shared definition, owner or permitted decision use | Averages hide a quality, effort or route failure that matters commercially or ethically. Acceptance and guardrails: The owner approves a measure set with observable user progress, value, effort, quality, population, period, source, limitation and decision consequence, with unsupported proxy relationships marked REQUIRES VALIDATION. Use only authorised, minimised summaries with evidence IDs and labels. Separate VERIFIED, OBSERVED, INFERRED, PROPOSED, UNKNOWN and REQUIRES VALIDATION; cite the supplied ID for every material statement. Identify contradictions, missing routes, definition drift, unsupported causal claims and the smallest next observation. Do not invent users, prevalence, outcomes or legal conclusions. Return an evidence table, risks, questions, decision options, rollback and human approvals required.

Review AI output against source records. If it cannot cite the supplied evidence ID, mark the statement REQUIRES VALIDATION. Human owners retain responsibility for research, privacy, accessibility, commercial, operational, release, governance and recovery decisions.

How to keep the problem from returning

Use three standing controls:

• Tie every headline measure to a decision and user outcome.
• Keep definitions, sources, limitations and route dimensions visible.
• Review proxy measures when behaviour, systems, offers or populations change.

Review after a material offer, channel, system, policy, dependency, operating-model, definition, release, population or AI workflow change; after recurring exceptions; when evidence becomes stale; or when acceptance or guardrails change. Update evidence, decision and change log together. Assign a review owner even when several teams maintain the journey. That owner must convene contributors, preserve dissent, access permitted evidence and obtain an authorised decision.

Continue with the preceding guide when earlier governance or learning evidence is unresolved, then use the next guide when that is the next question you need to answer. Use measurement decision intelligence audit when the adjacent problem owns the wider diagnosis: The existing audit owns the source-to-decision measurement system; this guide defines what one journey needs measured and why.

If the issue crosses teams, budgets, systems or evidence sources and cannot be resolved safely within the guide, Audit the journey measurement contract. An assessment should first agree the decision, evidence access and authority. Technology choices follow the findings.

Primary sources used

Source cutoff: 2 September 2026. Review by 2 March 2027, or sooner after a material change. These dates apply to the external references below; local organisational evidence still needs its own date and limitation.

• GOV.UK Service Manual: How to set performance metrics
• GOV.UK Service Manual: Map a user's whole problem
• GOV.UK Service Standard: Solve a whole problem for users
• GOV.UK Service Standard: Make sure everyone can use the service
• South African Government: Protection of Personal Information Act
• W3C Web Content Accessibility Guidelines 2.2
• NIST Artificial Intelligence Risk Management Framework Checkpoint 6: Use E01 to answer “What evidence establishes user progress?” For user progress, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision. Checkpoint 7: Use E02 to answer “What evidence establishes first value?” For first value, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision. Checkpoint 8: Use E03 to answer “What evidence establishes effort and quality?” For effort and quality, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.