Local Journey Variants Are Hidden inside an Average

What the journey problem looks like and why it affects the business

An average journey can look healthy while a market, device, role, access or assisted route carries most of the failure. Splitting every metric into segments is not the answer; the decision needs evidence about which variations materially change effort, value, harm, capacity or ownership.

Expose decision-relevant journey variation with bounded populations, route evidence and a clear threshold for keeping variants distinct.

This is a governance and learning decision before it is a documentation problem. The primary reader is Customer Director, Analytics Lead or Market Lead. The economic buyer is the customer, analytics or market executive accountable for equitable outcomes, local service performance and informed prioritisation across journey variants.

The named owner holds decision authority; specialists contribute evidence within their remit, but contribution does not replace accountable ownership.

Check variation that could change the journey decision. Do not turn it into automatic segmentation, thin local pages or unsupported demographic targeting.

What to expect in the working pack

Start the working pack with the charter. Use stable IDs to connect sources, tests, decisions and review records across its eight sheets. Entries need supporting evidence; the workbook alone cannot prove a conclusion:

• Start here: Read the bounded governance or learning decision, evidence labels, sequence and stop conditions.
• Journey charter: Define the user outcome, start and end, routes, exclusions, consequence and authority.
• Journey map: Connect user tasks, context, channels, systems, handoffs, owners and unresolved questions.
• Evidence register: Separate observed facts and statements from interpretations, inferences, proposals and unknowns.
• Friction and failure log: Record symptoms, affected routes, severity, recurrence, consequences and competing causes.
• Intervention and acceptance: Link a proposed governance, measurement or release change to acceptance, guardrails and rollback.
• Outcome and recovery: Observe progression, value, support, recovery, exceptions, harm and durable learning.
• Review and change log: Record versions, evidence changes, approvals, review triggers, replacement and retirement.

Use the same IDs across the workbook so a reviewer can move from a claim to source, definition, consequence, proposed change, outcome and decision. Seed rows are prompts, not local evidence; replace them with authorised organisation-specific material. Do not enter participant names, contact details, recordings, free-text case notes, AI prompts containing identifiers or unnecessary personal information. Store sensitive material in an approved controlled repository and record only a permitted reference. The workbook does not create consent, lawful authority or publication approval.

What evidence distinguishes the real cause

The visible symptom does not establish the cause. Before changing a measure, release gate, owner, definition or AI workflow, build a dated evidence set that distinguishes observed behaviour from interpretation.

Label every material statement. Use VERIFIED only when a current attributable source supports the statement and permitted use. Use OBSERVED for an event or participant statement without turning it into a population claim. Use INFERRED where evidence supports but does not prove an explanation. Use PROPOSED for a future design or decision. Use UNKNOWN when evidence is missing. Use REQUIRES VALIDATION where testing is required before investment, release, scale or public copy.

Test the three competing explanations in the authoring profile: Averages conceal an important route because the affected population is smaller or less measured. Some observed differences are noise or collection bias rather than a distinct journey. Teams manufacture segments without a decision, evidence or safe purpose. Record confirming and contrary cases with equal care. A report, metric, release note, map, generated synthesis or consensus can indicate where to investigate, but none proves why a control works or why a journey improved.

The research frame should cover the whole user problem and the routes people actually use. GOV.UK Service Standard: Make sure everyone can use the service and GOV.UK Service Manual: User research support end-to-end research and joined-up channel thinking; GOV.UK Service Manual: How to set performance metrics supports a clear user-needs frame where relevant. GOV.UK Service Manual: How to set performance metrics gives a primary measurement reference. Preserve interfaces across teams and dependencies instead of hiding them in one report.

Accessibility is part of the journey outcome and acceptance test, not a final compliance tick. The Web Content Accessibility Guidelines 2.2 include requirements relevant to complete processes, errors, interruptions, input assistance, redundant entry and accessible authentication. Test with disabled users and people who need assistance where the decision may affect them. Do not infer universal access from automation alone.

Personal information must follow the organisation's approved privacy and research process. POPIA, the Protection of Personal Information Act (https://www.gov.za/documents/protection-personal-information-act), is the primary South African statutory source. This guide is not legal advice; the Information Officer or qualified adviser retains legal authority. Record purpose, permitted use, access, retention and deletion arrangements before collecting identifiable evidence. Minimise workbook content and use controlled references.

Stop and re-charter when the decision is unnamed, the population cannot be bounded, the outcome is only a proxy, evidence cannot be used lawfully, assisted or excluded users cannot be represented, the owner lacks authority, or the work crosses the scope boundary. Record UNKNOWN when evidence is missing.

How to fix it step by step

Step 1: Define the journey decision

Open Start here and Journey charter. Write the decision as a choice an authorised owner must make, not an aspiration such as improve governance. Define the user outcome, the decision consequence and what evidence would permit action. Begin by testing material route variation; do not let a familiar report, release status or map stand in for the governed outcome.

For material route variation, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Averages conceal an important route because the affected population is smaller or less measured. Keep population boundary separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 1: Use E01 to answer “What evidence establishes material route variation?” For material route variation, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 2: Collect and label the evidence

Create evidence IDs before interpretation. In Evidence register, record source, date, population, route, context, limitation, confidence, lineage and permitted use. Reconcile research, operational, support, product, analytics and commercial evidence without pasting personal data. Collect successful, assisted, interrupted, failed and contrary cases; a clean report cannot prove that governance is working.

For population boundary, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Some observed differences are noise or collection bias rather than a distinct journey. Keep access condition separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 2: Use E02 to answer “What evidence establishes population boundary?” For population boundary, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 3: Set the population, routes and observation window

Use Journey charter to set population, routes, start, end, exclusions, observation window, review authority and decision boundary. For this guide, make population boundary visible rather than letting a dashboard, workflow state or launch date stand in for user progress. Keep unknown prevalence, unknown causality and unobserved routes explicit.

For access condition, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Teams manufacture segments without a decision, evidence or safe purpose. Keep outcome difference separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 3: Use E03 to answer “What evidence establishes access condition?” For access condition, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 4: Map tasks, handoffs and possible causes

In Journey map, map user tasks and context first, then add channels, systems, handoffs, measures, owners and unresolved questions. Connect each transition to evidence IDs and show where access condition changes the user's state or the decision's reliability. In Friction and failure log, describe symptoms separately from causes and record competing explanations.

For outcome difference, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Averages conceal an important route because the affected population is smaller or less measured. Keep aggregation decision separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 4: Use E04 to answer “What evidence establishes outcome difference?” For outcome difference, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 5: Check the business consequences and competing causes

Connect the issue to value, harm, cost, delay, quality, trust, retention, capacity or decision risk only where a defensible source exists. If size or causality is unknown, label it UNKNOWN or REQUIRES VALIDATION. Test whether outcome difference changes the decision. Do not treat a trend, definition, AI output or stakeholder consensus as proof.

For aggregation decision, record the affected user or business condition, source, date, owner, limitation and consequence. Test this explanation against successful and contrary routes: Some observed differences are noise or collection bias rather than a distinct journey. Keep material route variation separate from proxy measures and status labels. Mark incomplete evidence UNKNOWN or REQUIRES VALIDATION and name the next observation needed.

Checkpoint 5: Use E05 to answer “What evidence establishes aggregation decision?” For aggregation decision, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.

Step 6: Name owners, escalation and recovery routes

In Intervention and acceptance, name the accountable owner, contributors, dependency owners, decision rights, exception route and recovery owner. State what receiving teams must accept and what qualified privacy, accessibility, security or technical authorities must decide. A role without evidence access or power to pause, reject or reverse is not sufficient governance.

Step 7: Agree acceptance, guardrails and rollback tests

Write acceptance before implementation: The owner can state which journey variations materially change the decision, their populations and limitations, which variants remain combined, and what evidence would reopen the aggregation choice. Add accessibility, privacy, value, harm, support, quality, inclusion, trade-off and rollback guardrails. Use Outcome and recovery to record tests over a stated window. The negative case is: An average completion rate leads to a universal change that improves the dominant route while an accessible or low-bandwidth route loses progress.

Step 8: Record the decision and review triggers

Record the human decision and version together. In Review and change log, state proceed, adapt, pause, stop or investigate, approver, effective date, next review, evidence change and reopen or retirement trigger. Approval permits action; it does not turn PROPOSED work into VERIFIED evidence. Observe the result, preserve dissent and capture learning.

Worked example: A hypothetical service sees a healthy overall completion rate but a severe drop on a low-bandwidth route. The team tests route evidence, avoids inventing demographic explanations, keeps the variant visible and assigns a proportionate performance and accessibility response. This is a hypothetical teaching example, not an Abrasim client result and not evidence of a guaranteed outcome. Its purpose is to show how a bounded governance decision, competing explanations, responsible ownership, recovery, value, harm, trade-off and acceptance tests fit together.

How to verify the journey change

Verification asks whether the governance or learning decision improved the user outcome and control quality under stated conditions, not whether a map or workbook was completed. Check material route variation, population boundary, access condition, outcome difference, aggregation decision. Test this possible cause: averages conceal an important route because the affected population is smaller or less measured. A generic health signal cannot establish whether it is true. Keep the work within this scope: Decision-relevant journey variation. It does not cover automatic segmentation, local-page manufacture or unsupported demographic targeting. The acceptance condition is that the owner can state which journey variations materially change the decision, their populations and limitations, which variants remain combined, and what evidence would reopen the aggregation choice. Challenge this contrary case: An average completion rate leads to a universal change that improves the dominant route while an accessible or low-bandwidth route loses progress. Review the evidence chain in this order:

1. Recheck population, routes, start, end, exclusions and observation window in Journey charter.
2. Sample IDs from Journey map back to Evidence register, source lineage and permitted-use limits.
3. Confirm severe items in Friction and failure log have an owner, disposition, escalation and recovery route.
4. Execute the acceptance, accessibility, privacy, value, outcome, harm, support, inclusion, negative-case, trade-off and rollback tests in Intervention and acceptance and Outcome and recovery.
5. Observe progression, quality, first value, assistance, recovery, exceptions, support demand and adverse outcomes for the agreed window.
6. Reconcile route-level and contrary outcomes; do not use an aggregate to hide material variation or definition drift.
7. Ask the authorised owner to decide proceed, adapt, pause, stop or investigate.
8. Record evidence changes, decision, approver, review date and reopen or retirement trigger in Review and change log.

Use evidence appropriate to the decision. For material route variation, reconcile research, operational, analytics, reports, support and commercial sources with their population, period and limitation. Check population boundary, access condition and outcome difference and aggregation decision against the agreed acceptance conditions. Verification is incomplete when only the easiest route was tested, personal data was used beyond purpose, accessibility was checked only by automation, downstream ownership was absent, or the window ends before value or learning can be observed.

Use this evidence with AI safely. AI may organise authorised, minimised evidence, surface contradictions and draft questions. It must not invent users, prevalence, outcomes or legal conclusions, infer sensitive attributes, make the final decision or receive personal or confidential data outside an approved environment. The NIST AI Risk Management Framework (https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10) is a primary risk reference, not a substitute for local authority. Remove identifiers and use controlled references. Use this prompt:

Role: Journey governance and evidence reviewer. Decision: Expose meaningful market, device, role, access and service variation. Scope: Decision-relevant journey variation; not automatic segmentation, local-page manufacture or unsupported demographic targeting. Focus: material route variation; population boundary; access condition; outcome difference; aggregation decision. Evidence questions: E01: What evidence establishes material route variation | E02: What evidence establishes population boundary | E03: What evidence establishes access condition | E04: What evidence establishes outcome difference | E05: What evidence establishes aggregation decision. Competing explanations: Averages conceal an important route because the affected population is smaller or less measured | Some observed differences are noise or collection bias rather than a distinct journey | Teams manufacture segments without a decision, evidence or safe purpose. Acceptance and guardrails: The owner can state which journey variations materially change the decision, their populations and limitations, which variants remain combined, and what evidence would reopen the aggregation choice. Use only authorised, minimised summaries with evidence IDs and labels. Separate VERIFIED, OBSERVED, INFERRED, PROPOSED, UNKNOWN and REQUIRES VALIDATION; cite the supplied ID for every material statement. Identify contradictions, missing routes, definition drift, unsupported causal claims and the smallest next observation. Do not invent users, prevalence, outcomes or legal conclusions. Return an evidence table, risks, questions, decision options, rollback and human approvals required.

Review AI output against source records. If it cannot cite the supplied evidence ID, mark the statement REQUIRES VALIDATION. Human owners retain responsibility for research, privacy, accessibility, commercial, operational, release, governance and recovery decisions.

How to keep the problem from returning

Use three standing controls:

• Define the decision before selecting dimensions or segments.
• Retain assisted, access and local variants when evidence shows material consequence.
• Do not infer sensitive attributes or target people without lawful purpose and authority.

Review after a material offer, channel, system, policy, dependency, operating-model, definition, release, population or AI workflow change; after recurring exceptions; when evidence becomes stale; or when acceptance or guardrails change. Update evidence, decision and change log together. Assign a review owner even when several teams maintain the journey. That owner must convene contributors, preserve dissent, access permitted evidence and obtain an authorised decision.

Continue with the preceding guide when earlier governance or learning evidence is unresolved, then use the next guide when that is the next question you need to answer. Use aggregate metrics hide failing segments when the adjacent problem owns the wider diagnosis: The analytics guide owns safe metric dimensions and aggregation; this guide owns the user-route differences the analysis must preserve.

If the issue crosses teams, budgets, systems or evidence sources and cannot be resolved safely within the guide, Diagnose the hidden journey variation. An assessment should first agree the decision, evidence access and authority. Technology choices follow the findings.

Primary sources used

Source cutoff: 2 September 2026. Review by 2 March 2027, or sooner after a material change. These dates apply to the external references below; local organisational evidence still needs its own date and limitation.

• GOV.UK Service Standard: Make sure everyone can use the service
• GOV.UK Service Manual: User research
• GOV.UK Service Manual: How to set performance metrics
• W3C Web Content Accessibility Guidelines 2.2
• South African Government: Protection of Personal Information Act
• NIST Artificial Intelligence Risk Management Framework Checkpoint 6: Use E01 to answer “What evidence establishes material route variation?” For material route variation, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision. Checkpoint 7: Use E02 to answer “What evidence establishes population boundary?” For population boundary, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision. Checkpoint 8: Use E03 to answer “What evidence establishes access condition?” For access condition, link the controlled source, owner, evidence status and next action. Preserve contrary findings and reopen the charter if they change the decision.