Analytics and Interviews Disagree about Journey Difficulty

What the journey problem looks like and why it affects the business

Analytics can show a drop while interviews explain a different moment, population or route. The conflict may be real, or created by mismatched definitions. Choosing the source that supports a preferred story is not reconciliation.

Align population, stage, context, period, denominator and outcome definition before deciding whether signals contradict each other.

This guide is for analytics leads, research leads or customer directors. Budget approval sits with the leader who owns the cost of acting on a false diagnosis across growth, service operations, product or trust.

The named owner holds decision authority; specialists contribute evidence within their remit, but contribution does not replace accountable ownership.

Resolve one conflict in journey evidence. Instrumentation repair and the wider persona evidence system remain with their specialist guides.

Agree the user outcome, evidence and decision authority before choosing an intervention. Technology or AI cannot replace that work.

What to expect in the working pack

Use the working pack as you follow the guide. Start with the charter, assign stable evidence IDs and update the decision and review fields at each checkpoint. Its entries need supporting sources; the workbook alone is not evidence:

• Start here: Read the decision sequence, evidence labels, required inputs and stop conditions before changing the map.
• Journey charter: Record the user outcome, start, end, included routes, material exclusions, consequence and authority.
• Journey map: Connect stages to user tasks, context, channels, systems, handoffs, owners and unresolved questions.
• Evidence register: Separate observed facts and participant statements from interpretations, inferences and unknowns.
• Friction and failure log: Record symptoms, affected routes, severity, recurrence, consequences and competing explanations.
• Intervention and acceptance: Link each proposed change to an owner, dependency, test, guardrail, rollback and human decision.
• Outcome and recovery: Observe task and value outcomes alongside assistance, recovery, exceptions and harm checks.
• Review and change log: Govern versions, evidence changes, approvals, reopen triggers, replacement and retirement.

Use the same evidence, issue and intervention IDs across sheets so a reviewer can move from a claim to its source, consequence, proposed change and decision. The guide-specific seed rows show the minimum structure; replace them with authorised organisation-specific evidence.

Do not enter participant names, contact details, recordings, free-text case notes or unnecessary personal information in the workbook. Store sensitive evidence in an approved controlled repository and enter only a permitted reference. The workbook does not create consent or lawful authority.

What evidence distinguishes the real cause

The visible symptom does not establish the cause. Build one dated evidence set before redrawing the journey, commissioning technology or approving an intervention.

Label every material statement. Use VERIFIED only where a current, attributable source supports the statement and its permitted use. Use OBSERVED for a behaviour, event or participant statement without turning it into a population claim. Use INFERRED where evidence supports but does not prove the explanation. Use PROPOSED for a future design or decision. Use UNKNOWN where evidence is missing. Use REQUIRES VALIDATION where the claim must be tested before it may govern investment or public copy.

Three competing explanations deserve deliberate testing: The analytics event and interview task refer to different stages. Populations differ by route, support or eligibility and should not be averaged. Both signals are valid but explain different parts of one outcome. Record contrary cases as carefully as confirming cases. A diagram, analytics trend or stakeholder consensus can indicate where to investigate, but none proves why the condition exists.

The research standard should cover the end-to-end experience and the routes people actually use. GOV.UK Service Manual: User research provides the general research frame, while GOV.UK Service Manual: Researching user experiences helps bound the whole problem rather than only the organisation's visible touchpoint. Where the journey crosses channels or teams, preserve those interfaces instead of hiding them inside a single stage.

For this guide, keep these five decision lenses visible throughout the work: population and denominator alignment, stage and route definition, period and context comparability, outcome and explanation conflict, reconciled decision and instrumentation boundary. Each lens must be supported by an evidence ID or remain REQUIRES VALIDATION; listing a focus term is not evidence.

Accessibility is part of journey evidence and acceptance, not a final compliance check. The Web Content Accessibility Guidelines 2.2 include requirements relevant to complete processes, interruptions, redundant entry, errors and accessible authentication. Test with disabled users and people who need support where the decision may affect them. Do not claim universal access from an automated check alone.

Personal information must be handled under the organisation's approved privacy and research process. POPIA, the Protection of Personal Information Act, is the primary South African statutory source. This guide is not legal advice; the Information Officer or qualified adviser retains legal authority. Record purpose, permitted use, access, retention and deletion arrangements before collecting identifiable evidence.

Stop and re-charter if the decision is unnamed, the population cannot be bounded, the evidence cannot be used lawfully, assisted or excluded users cannot be represented, the owner lacks authority, or the proposed work crosses the scope boundary. Record UNKNOWN when evidence is missing.

How to fix it step by step

Step 1: Define the journey decision

Open Start here and Journey charter. Write the decision as a choice an authorised person must make, not as an aspiration such as “improve experience.” Add the economic consequence, decision date and owner. Then write the user outcome in observable terms: what will the user be able to do, under which conditions, and how will completion be recognised?

Do not move on if stakeholders use different outcomes. Record each competing definition in the Evidence register, identify its source and mark it INFERRED or REQUIRES VALIDATION until an authorised owner resolves it. This prevents the map from manufacturing consensus.

Step 2: Collect and label the evidence

Create evidence IDs before interpretation. For each source, record date, population, context, limitation, confidence and permitted use in Evidence register. Link interviews, observation, analytics, search, CRM, support and operating evidence through controlled references. Do not paste sensitive raw data into the workbook.

Collect unsuccessful, assisted, interrupted and contrary cases as well as successful ones. If the available sample contains only people who completed through the easiest route, record that limitation. It may still support a local observation, but it cannot support a universal journey claim.

Step 3: Set the population, routes and observation window

Use Journey charter to set a start condition, end condition, included population and routes, material exclusions and the point at which the decision stops. Use conditions, not channel names: “the user recognises a need” is a condition; “visits the website” is one possible route.

In Journey map, give every retained stage an observable entry and exit. A stage should represent a change in user state, value, risk or decision, not every click, message or internal action. Keep channels and systems in their own columns. Record exclusions visibly so a neat map cannot imply unsupported completeness.

Step 4: Map tasks, handoffs and possible causes

Map user tasks in Journey map first. Then add context, channels, systems, handoffs and owners. Where internal activity enables or blocks the user, show the interface without replacing the user's task with an internal status. Link each material transition to evidence IDs.

In Friction and failure log, describe the symptom without assuming the cause. Record affected route, severity, recurrence, possible consequence and at least one competing explanation. A high-friction step may be caused by missing information, an eligibility rule, an inaccessible interaction, an ownership gap or an upstream promise; the map should keep those possibilities open until evidence distinguishes them.

Step 5: Check the business consequences and competing causes

Connect the issue to something the business can decide. Quantify revenue, cost, delay, risk, retention or service-demand exposure only where a defensible source exists. Otherwise describe the consequence and mark its size UNKNOWN or REQUIRES VALIDATION. Never invent a saving, conversion uplift or affected population.

Use the Friction and failure log to compare explanations. Ask what evidence would disconfirm the preferred cause. If the issue falls mainly inside a wider operating, persona, analytics or acquisition problem, follow the boundary link rather than stretching this guide into a full transformation audit.

Step 6: Name owners, escalation and recovery routes

In Intervention and acceptance, name one accountable owner, necessary contributors, dependencies and decision authority. A functional contributor is not automatically the journey owner. The six value paths cross functional budgets, evidence and ownership; diagnosis therefore needs cross-functional authority, access and independent challenge, whether led internally or externally.

Specify receiving acceptance for handoffs and escalation for exceptions. Record a rollback route before implementation. If the owner cannot pause, reject or reverse the change, escalate the authority gap instead of recording a false owner.

Step 7: Agree acceptance, guardrails and rollback tests

Write the acceptance test before implementing the change. For this guide, the minimum condition is: The owner accepts a reconciliation table that aligns definitions, records genuine contradictions and names one bounded decision plus any instrumentation work transferred to its owner. Add guardrails for accessibility, privacy, service demand, quality and excluded routes where material.

Use Outcome and recovery to define the observation window, support route, recovery route, exception and harm check. Test the negative case explicitly: The dashboard definition is changed until it agrees with interviews and the conflict is called resolved without testing the route. A local improvement fails acceptance when it damages the wider outcome or hides who was unable to progress.

Make the intervention trade-offs explicit before acceptance. In this guide, weigh population and denominator alignment against reconciled decision and instrumentation boundary, and record what may improve, what may worsen, whose route could be excluded and what evidence would reverse the choice. The decision must remain bounded by this acceptance condition: The owner accepts a reconciliation table that aligns definitions, records genuine contradictions and names one bounded decision plus any instrumentation work transferred to its owner.

Step 8: Record the decision and review triggers

Record the human decision in Intervention and acceptance and the version in Review and change log. State whether to proceed, adapt, pause, stop or commission deeper diagnosis. Include approver, effective date, next review, evidence change and the trigger that would reopen or retire the decision.

Do not convert PROPOSED work into VERIFIED evidence merely because it was approved. Approval authorises action; observation supplies outcome evidence. Keep those states separate through implementation and review.

Worked example

A hypothetical checkout dashboard shows 70% progression from quote to payment while interviews say customers struggle after a quote. The team finds analytics counts opened payment pages while interviews include phone routes; it splits the routes and chooses a bounded follow-up.

This is a hypothetical teaching example, not an Abrasim client result and not evidence of a guaranteed outcome. Its purpose is to show the decision structure: identify the user outcome, preserve competing explanations, assign authority, test the change across the journey and record what would reverse the decision.

How to verify the journey change

Verification asks whether the decision changed the user outcome under the stated conditions, not whether the map was completed. Review the workbook as one connected evidence chain:

1. Recheck the population, routes, start, end and exclusions in Journey charter.
2. Sample evidence IDs from Journey map back to the Evidence register and controlled sources.
3. Confirm that high-severity items in Friction and failure log have an owner and disposition.
4. Execute the acceptance, guardrail, negative-case and rollback tests in Intervention and acceptance.
5. Observe task outcome, first value, assistance, recovery, exceptions and harm in Outcome and recovery for the agreed window.
6. Record contrary evidence and unresolved limitations rather than averaging them away.
7. Ask the authorised owner to decide proceed, adapt, pause, stop or investigate.
8. Record the decision, evidence change and reopen trigger in Review and change log.

Use evidence appropriate to the decision. Behavioural observation can show where people struggle; operational records can show delay and recurrence; analytics can show patterns within its collection limits; participant accounts can explain context; commercial records can show qualified value. Reconcile them instead of treating one source as complete.

Verification is incomplete when only the easiest route was tested, personal data was used outside permitted purpose, accessibility was checked only by automation, a downstream team was not involved, or the outcome window ends before value or recovery can be observed.

Use this evidence with AI safely

AI can help organise authorised evidence, identify contradictions and draft questions. It must not invent missing user evidence, infer sensitive attributes, make the final decision or receive personal or confidential data outside an approved environment. Remove or replace identifiers and use controlled references.

Copy and adapt this prompt:

Role: Journey evidence reviewer. Decision: Reconcile population, stage, context, period and outcome before choosing a cause. Scope: One journey-evidence conflict; instrumentation repair and the wider persona evidence system remain with their owning guides. Evidence questions: E01: Are populations and denominators comparable | E02: Do stages and event names describe the same transition | E03: Do period, channel and context explain the difference | E04: What outcome does each source actually establish | E05: What decision follows and remains unresolved. Competing explanations: The analytics event and interview task refer to different stages | Populations differ by route, support or eligibility and should not be averaged | Both signals are valid but explain different parts of one outcome. Acceptance and guardrails: The owner accepts a reconciliation table that aligns definitions, records genuine contradictions and names one bounded decision plus any instrumentation work transferred to its owner. Use only authorised, minimised summaries with evidence IDs and labels. Separate VERIFIED, OBSERVED, INFERRED, PROPOSED, UNKNOWN and REQUIRES VALIDATION; cite the supplied ID for every material statement. Identify contradictions, missing routes, unsupported causal claims and the smallest next test. Do not invent users, prevalence, outcomes or legal conclusions. Return an evidence table, risks, questions, decision options and human approvals required.

Review the output against source records. If it cannot cite the supplied evidence ID for a statement, treat that statement as REQUIRES VALIDATION. Human owners retain responsibility for research, privacy, accessibility, commercial and operational decisions.

How to keep the problem from returning

Use three standing controls:

• Version event, stage, population and denominator definitions with every decision.
• Pair behavioural data with situated research rather than ranking one source universally.
• Require reconciliation after material tracking, channel, offer or population change.

Review the journey after a material offer, channel, system, policy, operating-model or population change; after recurring exceptions; when evidence becomes stale; or when the acceptance result changes. Do not update the diagram silently. Update the evidence, decision and change log together.

Assign a review owner even when the map is maintained by several teams. The owner does not need to produce every input, but must be able to convene contributors, access evidence, preserve dissent and obtain an authorised decision. If that authority does not exist, record the governance constraint openly.

Continue with the preceding guide when earlier boundary work is unresolved, then use the next guide when that is the next question you need to answer. Use analytics customer research conflict when the adjacent problem owns the wider diagnosis: The persona guide reconciles evidence for a people model; this guide reconciles evidence for a stage-specific journey difficulty.

If the issue crosses teams, budgets, systems or evidence sources and cannot be resolved safely within the guide, Reconcile the journey measurement evidence. An assessment should first agree the decision, evidence access and authority. Technology choices follow the findings.

Primary sources used

Source cutoff: 2 September 2026. Review by 2 March 2027, or sooner after a material change. These dates apply to the external references below; local organisational evidence still needs its own date and limitation.

• GOV.UK Service Manual: User research
• GOV.UK Service Manual: Researching user experiences
• GOV.UK Service Manual: Scoping your service
• W3C Web Content Accessibility Guidelines (WCAG) 2.2
• South African Government: Protection of Personal Information Act