Analytics and Customer Research Tell Different Stories

What the problem looks like and why it affects the business

Analytics suggests that a journey works, while interviews describe severe difficulty or research participants report success while measured progression falls. Teams then choose the source they already trust instead of testing whether the evidence covers the same people, period and question.

Unreconciled mixed evidence can delay action, create false certainty or cause a real customer problem to be dismissed as anecdotal.

Start with the decision this research needs to support. Check whether analytics and research cover the same people and question. The budget owner is normally CMO, Product Director or Analytics Lead; involve research, analytics, sales, service, product, accessibility and privacy specialists where their evidence or authority is needed. The symptom alone does not establish its cause.

This guide helps the owner decide whether the current persona evidence may support that decision, needs correction, should be narrowed, must be validated or should be stopped. It does not choose the organisation's priority customer segment. Use Customer Evidence and Segment Priority Standard when the unresolved decision is whom the organisation should prioritise.

What to expect in the working pack

Start the working pack before research begins. Update its eight sheets as you follow the method:

• Start here: Explain the decision, sequence, evidence labels and stop conditions before the reader begins.
• Decision charter: Bound the decision the persona must support and the authority required to act on it.
• Evidence register: Keep observed facts, participant statements, interpretations, inferences and unknowns distinguishable.
• Research and coverage: Plan research questions, participant contexts, access needs, consent and coverage gaps.
• Persona hypotheses: Represent only evidenced jobs, needs, behaviours, contexts and barriers that may change the decision.
• Roles and journey: Map changing user and buying roles, tasks, criteria, handoffs and unresolved conflicts.
• Activation and tests: Connect persona evidence to a changed decision, an accountable test and observable acceptance evidence.
• Review and change log: Govern versions, approvals, evidence changes, review triggers, replacement and retirement.

Use stable IDs when one sheet refers to another. Replace the illustrative seed rows with authorised organisation-specific evidence. Do not enter names, contact details, recordings or unnecessary personal information into the workbook; link to a controlled source reference instead.

What evidence distinguishes the real cause

Build one dated evidence set before rewriting the persona, launching research or changing a customer journey.

Classify each material statement as VERIFIED, INFERRED, PROPOSED or REQUIRES VALIDATION. Within the evidence register also distinguish an observed fact, a participant statement, an organisational interpretation, an inference and an unsupported assumption. These labels answer different questions and must not be collapsed during synthesis.

Research permission is part of evidence quality. Record purpose, participant information, consent or other authorised basis, access, sharing, retention, withdrawal and deletion routes before collecting personal information. The Protection of Personal Information Act is a primary South African legal source; the guide is not legal advice, and the organisation's Information Officer or qualified adviser retains legal authority.

Stop and re-charter when the decision is unnamed, participation would be coercive, the team cannot protect participant data, the sample cannot answer the question, a preferred outcome controls the method, or no receiving owner can act on the result.

How to fix it step by step

Throughout the eight steps, record each material item's source, population or participant context, date, owner, permitted use and confidence. Keep contrary evidence and unresolved questions visible. Label unknowns, participant statements, internal interpretations and examples separately from verified customer facts. Save source references with the receiving owner, next handoff and conditions for stopping or reopening the work.

Step 1: Charter the decision and exposure

Accountable owner: CMO, Product Director or Analytics Lead.

Restate the decision and define the behaviour or outcome each source is expected to inform.

Use in the working pack: Start here and Decision charter.

Check before continuing: Check that the decision, accountable owner and business consequence are clear. Record who receives the result and any authority gap that prevents the next step.

Step 2: Separate claims from evidence

Accountable owner: Research lead with the decision owner.

Create separate source contracts for analytics and research, including population, period, method, exclusions and confidence.

Use in the working pack: Evidence register.

Check before continuing: A reviewer should be able to trace the recorded claims or inputs to their sources. Keep evidence types, confidence and contrary findings visible before drawing a conclusion.

Step 3: Define coverage and authority

Accountable owner: Research lead with privacy, accessibility and relationship owners.

Test whether the apparent conflict disappears when definitions, cohorts, journey stages and time windows are aligned.

Use in the working pack: Research and coverage.

Check before continuing: Record the coverage, classification and permission limits relevant to this task. Identify what is still missing and whether that gap prevents the next step.

Step 4: Test the competing explanation

Accountable owner: Research and analysis lead.

Investigate collection failure, sample selection, identity loss, recall, social response and researcher interpretation as rival explanations.

Use in the working pack: Evidence register and Persona hypotheses.

Check before continuing: Retain the assumptions, competing explanations and unresolved questions this step exposed. Explain which evidence would change the current interpretation and who will review it.

Step 5: Run the bounded evidence round

Accountable owner: Research lead and participant-access owner.

Locate cases where both sources can be joined safely at an aggregate or session level without exposing unnecessary personal information.

Use in the working pack: Research and coverage.

Check before continuing: Keep the evidence collected or reviewed at this step, its permitted use and important exclusions. Record any remaining gap before treating the result as customer evidence.

Step 6: Synthesise without hiding conflict

Accountable owner: Research lead with cross-functional observers.

Run a bounded triangulation test using observation, source records, targeted research or instrumentation QA.

Use in the working pack: Persona hypotheses and Roles and journey.

Check before continuing: Separate the retained observations from your interpretation. Link the result to sources, preserve conflicting cases and show what the receiving team can safely use.

Step 7: Change and test the decision

Accountable owner: Accountable decision owner with receiving teams.

Classify the result as agreement, complementary evidence, unresolved conflict or source failure and choose the appropriate action.

Use in the working pack: Activation and tests.

Check before continuing: Record the decision change or test, its owner and what remains uncertain. An authorised reviewer must be able to follow the evidence before the next commitment.

Step 8: Approve, defer or reopen

Accountable owner: CMO, Product Director or Analytics Lead.

Record the decision, confidence and review trigger without claiming that either source alone proves cause.

Use in the working pack: Review and change log.

Check before continuing: Record the authorised decision, sources, unresolved limits and next handoff. State when to review, stop or reopen the work so approval cannot be mistaken for permanent certainty.

Worked method example

Completion analytics appears stable, but interviews report repeated difficulty. Cohort review shows that assisted completions are recorded as success while support contacts are not connected to the journey. The team keeps the completion rate, adds assisted-effort evidence and changes the service decision rather than declaring either source wrong.

This is a method illustration, not an Abrasim client result. The reader must replace it with current, permissioned evidence and record any limitation that would change the decision.

How to verify the persona improves the decision

Accept the work when the sources have comparable contracts or an explicit reason they should not agree, rival explanations have been tested, and the decision states which evidence is fit for which use.

Run four tests:

1. Traceability test: select a material persona claim and trace it to the evidence, participant context, permission, interpretation and confidence record.
2. Decision-difference test: present the old and revised evidence to the accountable owner and record whether a real content, product, service, research or commercial choice changes.
3. Contrary-case test: apply the decision to a credible negative, excluded or exception case and confirm that the model exposes rather than hides the difference.
4. Outcome-learning test: after the accepted observation window, record what happened separately from implementation completion and reopen if the expected behaviour or customer outcome does not appear.

Analytics volume does not automatically overrule direct research, and vivid interview evidence does not establish prevalence across the measured population.

Roll back or contain a decision when it increases exclusion, service failure, privacy exposure, unsupported targeting, customer burden or irreversible investment without stronger evidence. A completed interview, workbook or persona card is implementation evidence; it is not proof of a customer or commercial outcome.

How to keep the problem from returning

• Attach population, period and construct definitions to every evidence claim.
• Plan mixed-method reconciliation before research and instrumentation begin.
• Retain unknown and unjoinable populations instead of forcing alignment.

Review the persona when the decision, market, offer, customer context, research coverage, evidence source, regulation or operating model changes materially. Retire it when no active decision uses it or when a replacement has an approved migration and evidence-retention route.

Use the preceding guide before this method when its evidence boundary is unresolved. Continue to the next guide when you have the evidence it requires. Compare the adjacent guide before commissioning overlapping work: The existing guide owns source-to-decision measurement; this guide reconciles persona evidence.

Primary sources reviewed at the 30 August 2026 source cutoff include gov analyse, gov user needs, gao data reliability, popia act. These sources inform the method; they do not prove a condition, prevalence or outcome for a particular organisation.

Where the problem crosses functional budgets, evidence and ownership, Audit the measurement and customer-evidence boundary. The six value paths cross functional budgets, evidence and ownership; diagnosis therefore needs cross-functional authority, access and independent challenge, whether led internally or externally.