The Proof-of-Value Pilot Standard: Designing Automation Tests That Can Earn Production

A pilot should test the proposed change under conditions that resemble real work. Agree the workflow outcome, baseline and acceptance criteria, then check the effects on customers, employees, service, controls and operations before deciding whether it should progress.

This guide helps a COO, CIO and CFO design and govern an automation proof-of-value pilot. It fixes the decision, baseline, population, evaluation set, shadow mode, acceptance, failure tests, rollback, economics and ownership before execution. The output is evidence for a human production decision; the pilot never earns production automatically.

1. The Market Connection Context

The recognisable condition is pilot theatre: a polished happy path is called proof, enthusiastic users substitute for a baseline, errors are repaired off-record, supplier staff operate the intervention, and the successful demo remains connected after the meeting. Leadership then inherits a system whose real exception burden, verification demand, support capacity and customer consequence were never tested.

The value path is:

Accepted workflow and architecture boundary
→ pre-registered pilot question and baseline
→ controlled representative evidence
→ tested value, failure, recovery and operating burden
→ human production disposition
→ separate production design and release gate

The pilot decision contract

Every pilot must resolve a named uncertainty and end in one disposition:

The production decision contract must be written before the pilot starts. Success is not “users liked it,” “the model worked,” or “the automation ran.” It requires separate evidence for outcome, workflow, capability release, quality, reliability, control, affected-person experience, recovery and operating economics.

The proof boundary

A pilot tests an intervention against business-as-usual and, where relevant, a do-minimum alternative. Its evidence contract defines population, period, unit, denominator, exclusions, source, owner, counterfactual and attribution limit. Missing evidence is REQUIRES VALIDATION, not zero. Repaired failures remain failures in the recorded run and receive their own remediation evidence.

NIST states that trustworthy AI depends heavily on reliable measurement and evaluation of technologies and their use (AI test, evaluation, validation and verification, VERIFIED official source). NIST’s ARIA work distinguishes model testing, red-teaming and field testing, demonstrating why one evaluation layer cannot represent all relevant behaviour (NIST Human-Centered AI, VERIFIED official source). This guide uses the layered-evidence principle without claiming that a local pilot implements ARIA.

Representative conditions and safe exposure

Representative does not mean uncontrolled live use. The pilot moves through evidence modes only when accepted gates permit:

1. Offline replay: approved historic, synthetic or specially constructed cases test logic without affecting live work.
2. Shadow mode: the intervention processes a representative stream but cannot write or determine the real outcome.
3. Supervised bounded use: authorised users apply or execute within explicit populations, permissions, limits and immediate fallback.
4. Recovery exercise: failure, degradation, rollback, revocation and operational handover assumptions are tested.

The U.S. GAO Technology Readiness Assessment Guide is identified by the stable official product ID GAO-20-48G. Automated access returned 403 in this review, so the nearby proposition remains REQUIRES VALIDATION pending a dated proof-owner capture. Its maturity levels are not adopted here; the relevant-environment discipline is a proposed local test-design principle, not a GAO conformity claim.

Scope and exclusions

Use the standard only after The Automation and Technology Readiness Assessment: An Executive Standard for Choosing What Should Change readiness, The Workflow Constraint and Root-Cause Audit: Simplify the Work Before Automating It workflow diagnosis and The Automation Architecture Decision: Rules, Platforms, Agents, Custom Software or No Build architecture evidence exist, with The Automation Opportunity Portfolio: A Value, Feasibility, Risk and Learning Playbook portfolio authority where opportunities compete and The AI and Automation Governance Standard: Proportionate Control from Assistance to Autonomous Action controls accepted where applicable. Scope includes charter, hypotheses, baseline, counterfactual, evaluation population, modes, permissions, execution, observation, failures, economics, decision and closure.

It excludes readiness diagnosis, portfolio selection, architecture choice, governance design, full agent delivery, production operations, procurement, enterprise scale and workforce decisions. The Automation Production Operating Standard: Integration, Monitoring, Incidents and Change Control owns production integration, monitoring, incidents and change control. A pilot environment, credential, data copy, supplier access or manual workaround cannot remain active after closure unless separately authorised.

Expected results are a pilot charter, baseline and hypothesis register, evaluation and case register, run/failure/recovery record, economics/decision/closure register and downstream evidence brief. They are not a case study, benefit forecast, production acceptance or guaranteed return.

2. The Leadership vs. Delegation Boundary

The reader recognises that a defensible pilot needs a baseline, evaluation set, shadow mode, acceptance thresholds, failure tests, rollback and operating economics designed before build.

Internal leadership

An internal team can lead when it can pre-register thresholds, preserve failed cases, prevent pilot-to-production drift, access representative workflow evidence and stop a sponsor’s preferred intervention. Evaluation should be sufficiently independent from builders and suppliers to challenge data exclusions, repaired failures and manual support.

Independent leadership

Independent challenge is proportionate when the vendor operates the pilot, the sponsor has announced success, baseline ownership is disputed, sensitive customer or employee effects exist, or production funding depends on the result. An external lead may structure and reconcile evidence but cannot accept risk, fabricate a counterfactual or sign production authority.

Narrow specialists

Use a statistician or evaluation specialist for sample and inference design, a user researcher for representative task evidence, security and privacy specialists for bounded tests, a reliability specialist for failure and recovery, or finance for economic definitions. Specialist work must remain linked to one pilot decision contract.

Stop and escalation

Contain any unauthorised data, participant harm, credential leakage, material service incident, action outside permissions, unreliable fallback or evidence-integrity breach immediately. Escalate outcome and exposure to the sponsor, service consequence to the COO, environment and access to the CIO, economics to the CFO, and specialist matters to qualified authorities.

Questions for evaluating a pilot provider

1. Which production uncertainty will this pilot resolve?
2. What baseline and do-minimum alternative will remain visible?
3. How will cases represent normal, boundary, exception and adverse conditions?
4. Which results will be preserved as failures even after repair?
5. How will shadow mode prevent an output from affecting live decisions?
6. What stops supplier effort from hiding internal operating burden?
7. Which human verification and fallback capacity will be measured?
8. What evidence would cause stop rather than extension?
9. How will credentials, data, environments and claims close after the decision?
10. Where does the handoff end before The Automation Production Operating Standard: Integration, Monitoring, Incidents and Change Control production design?

3. The Step-by-Step Execution Standard

Use the Pilot Charter Baseline and Hypothesis Register, Evaluation Population Case and Acceptance Register, Pilot Run Failure and Recovery Register, and Economics Decision Handoff and Closure Register.

Step 01: Charter one production uncertainty

Purpose and decision served Define the exact question the pilot must resolve and the later human production disposition.

Accountable owner Executive sponsor with COO, CIO and CFO acceptance.

Delegated lead and contributors Pilot lead, operating owner, architect, governance, evaluation, finance, data, users and control owners.

Preconditions, inputs and access Accepted readiness, diagnosis, architecture and applicable governance records plus authority to stop.

Actions

1. State one uncertainty without promising the answer.
2. Bound workflow, population, environment, period and exposure.
3. Define dispositions, exclusions and evidence authority.
4. Record interests, production separation and stop conditions.

Evidence to retain Source handoffs, charter versions, authorities, conflicts and approval.

Decision or scoring rule Proceed only when a negative or inconclusive result can change the decision.

Output and template A2P-CHR-* records in the Pilot Charter Baseline and Hypothesis Register.

Acceptance gate PROCEED, CHANGE, DEFER or STOP.

Leadership checkpoint Leaders confirm pilot activity is not a production quota.

Human control point Only authorised leaders accept exposure and dispositions.

Failure modes and safeguards A technology demonstration is rejected unless it answers the business decision.

Change, stop or escalation condition Stop if production is assumed regardless of result.

Handoff to the next step Pass the charter to Step 02.

Step 02: Contract hypotheses and counterfactuals

Purpose and decision served Make value, quality, reliability, burden and harm claims testable before results exist.

Accountable owner Benefit owner and evaluation owner.

Delegated lead and contributors Finance, process, service, data, customer, employee and control specialists.

Preconditions, inputs and access Charter, baseline sources, business-as-usual, direct-change alternative and proposed intervention.

Actions

1. Write directional hypotheses with units and populations.
2. Define business-as-usual and do-minimum counterfactuals.
3. Separate learning, operating, outcome and guardrail claims.
4. State attribution limits and evidence that would reject each claim.

Evidence to retain Original claims, definitions, sources, alternatives and rejected formulations.

Decision or scoring rule An unfalsifiable or ownerless claim cannot be a pilot success criterion.

Output and template A2P-HYP-* and A2P-CFT-* rows.

Acceptance gate Every decision-critical claim has source, owner, comparator and reject condition.

Leadership checkpoint The CFO accepts that capacity is not cash without a conversion decision.

Human control point Benefit and affected-outcome owners accept definitions.

Failure modes and safeguards Post-result metric selection is prevented through pre-registration.

Change, stop or escalation condition Return claims whose baseline cannot be established proportionately.

Handoff to the next step Pass hypotheses to Step 03.

Step 03: Establish baseline and source contracts

Purpose and decision served Create a decision-fit picture of current workflow outcome, variation, burden, exceptions and harm.

Accountable owner Operating owner.

Delegated lead and contributors Analyst, finance, data owners, frontline users, service and quality specialists.

Preconditions, inputs and access Authoritative workflow, customer, service, finance, incident and capacity sources.

Actions

1. Define population, period, unit, denominator and exclusions.
2. Reconcile systems with observed performed work.
3. Record missingness, selection, seasonality and concurrent changes.
4. Secure baseline owner acceptance and evidence expiry.

Evidence to retain Extract locators, metric contracts, observation notes, reconciliation and approval.

Decision or scoring rule Missing baseline is REQUIRES VALIDATION and never silently reconstructed after the pilot.

Output and template A2P-BAS-* records linked to hypotheses.

Acceptance gate Decision-critical baselines are accepted or a named validation changes the charter.

Leadership checkpoint The COO and CFO accept current performance and evidence limits.

Human control point Source owners verify definitions and authorised access.

Failure modes and safeguards Convenient short periods and excluded failures remain visible.

Change, stop or escalation condition Defer if no ethical and proportionate comparator exists.

Handoff to the next step Pass baseline contracts to Step 04.

Step 04: Design the evaluation population and cases

Purpose and decision served Represent normal work, boundary conditions, exceptions and meaningful adverse conditions.

Accountable owner Evaluation owner.

Delegated lead and contributors Domain, data, user research, accessibility, security, privacy, reliability and affected-person specialists.

Preconditions, inputs and access Hypotheses, baselines, workflow taxonomy, known incidents and authorised data sources.

Actions

1. Define sampling frame and inclusion or exclusion rules.
2. Allocate normal, boundary, exception, adversarial and changed-condition cases.
3. Identify affected groups and rare high-consequence scenarios.
4. Version cases and protect against training or tuning leakage.

Evidence to retain Population rationale, case IDs, source, proof status, exclusions and limitations.

Decision or scoring rule Case count cannot compensate for absent material conditions or populations.

Output and template A2P-POP-* and A2P-CAS-* rows in the Evaluation Population Case and Acceptance Register.

Acceptance gate The evaluation owner and qualified authorities accept representativeness for the bounded decision.

Leadership checkpoint Leaders see what the pilot cannot generalise to.

Human control point Qualified reviewers accept participant, data and affected-person safeguards.

Failure modes and safeguards Happy-path selection and supplier-curated cases are challenged independently.

Change, stop or escalation condition Validate or narrow where a material population cannot be represented safely.

Handoff to the next step Pass cases to Step 05.

Step 05: Pre-register measures and acceptance gates

Purpose and decision served Fix how evidence will be calculated and interpreted before the intervention is run.

Accountable owner Evaluation owner with benefit and control owners.

Delegated lead and contributors Analyst, statistician, finance, operations, quality and qualified control specialists.

Preconditions, inputs and access Charter, hypotheses, baseline, population, case taxonomy and decision consequences.

Actions

1. Define numerator, denominator, unit, exclusions and uncertainty.
2. Set acceptance, rejection, escalation and inconclusive rules.
3. Separate aggregate, group, boundary and guardrail views.
4. Freeze analysis version and change-control route.

Evidence to retain Metric contracts, calculation version, approval, dissent and amendment history.

Decision or scoring rule No average or composite score overrides a mandatory failure or material harm.

Output and template A2P-MET-* and A2P-ACC-* records.

Acceptance gate Every hypothesis and stop condition maps to a pre-agreed measure or qualitative rule.

Leadership checkpoint The decision forum accepts what evidence would produce stop.

Human control point Named owners approve definitions; the delivery team cannot amend them informally.

Failure modes and safeguards Metric shopping is controlled by frozen versions and disclosed amendments.

Change, stop or escalation condition Re-charter a decision-changing measure amendment.

Handoff to the next step Pass acceptance contracts to Step 06.

Step 06: Design modes, permissions and containment

Purpose and decision served Sequence evidence from offline replay through shadow and supervised use without accidental production.

Accountable owner CIO and operating owner.

Delegated lead and contributors Architecture, security, privacy, integration, environment, service and governance owners.

Preconditions, inputs and access The Automation Architecture Decision: Rules, Platforms, Agents, Custom Software or No Build architecture, The AI and Automation Governance Standard: Proportionate Control from Assistance to Autonomous Action controls, cases, identities, interfaces, fallback and incident routes.

Actions

1. Define offline, shadow, supervised and recovery modes.
2. Assign least-privilege identities, data, tools, writes and transaction limits.
3. Separate pilot systems, credentials, data and logs from production authority.
4. Pre-authorise interrupt, rollback, revocation and fallback.

Evidence to retain Environment map, permissions, approvals, isolation, revocation and mode-gate tests.

Decision or scoring rule No mode advances until its prior evidence and containment gate is accepted.

Output and template Mode and control rows linked to case and run IDs.

Acceptance gate Every mode has explicit effect boundary, owner, entry, exit and emergency response.

Leadership checkpoint The CIO confirms shadow output cannot determine live action.

Human control point Named authorities enable each mode and can interrupt it.

Failure modes and safeguards Pilot credentials and manual exceptions cannot become unrecorded production paths.

Change, stop or escalation condition Contain any action outside the accepted mode immediately.

Handoff to the next step Pass approved modes to Step 07.

Step 07: Rehearse workflow, human authority and fallback

Purpose and decision served Verify that people can execute, review, override and recover before consequential exposure.

Accountable owner Operating owner.

Delegated lead and contributors Pilot users, supervisors, service desk, trainers, accessibility, incident and domain specialists.

Preconditions, inputs and access Mode design, user guidance, human-control contract, case set and fallback.

Actions

1. Rehearse task, review, exception, challenge and escalation paths.
2. Observe information, time, competence and workload.
3. Test fallback and manual continuity with real role owners.
4. Record usability, accessibility, workarounds and unplanned support.

Evidence to retain Observed sessions, role acceptance, issues, burden and corrected guidance.

Decision or scoring rule A named human is not a control if practical conditions prevent meaningful action.

Output and template Readiness and human-control run records.

Acceptance gate Pilot roles can execute and recover inside accepted capacity and safeguards.

Leadership checkpoint The COO accepts the actual support and oversight burden.

Human control point Pilot users may stop unsafe or unclear activity without penalty.

Failure modes and safeguards Training success is not confused with workflow or outcome evidence.

Change, stop or escalation condition Return design where fallback or human authority fails rehearsal.

Handoff to the next step Pass readiness evidence to Step 08.

Step 08: Execute controlled runs and preserve observations

Purpose and decision served Generate traceable evidence under the pre-registered modes and cases.

Accountable owner Pilot lead under operating and evaluation authority.

Delegated lead and contributors Operators, observers, analysts, system owners, support and evidence custodian.

Preconditions, inputs and access Accepted mode, authorised participants, frozen versions, case allocations and live stop authority.

Actions

1. Record run, case, component, version, identity and mode IDs.
2. Preserve output, action, human response, timing and support evidence.
3. Record deviations, repairs and excluded cases without overwriting originals.
4. Pause at planned and event-driven evidence gates.

Evidence to retain Immutable run records, logs, observations, deviations, changes and custody.

Decision or scoring rule An undocumented manual repair remains an evidence failure, not successful automation.

Output and template A2P-RUN-* records in the Pilot Run Failure and Recovery Register.

Acceptance gate Runs are authorised, traceable, complete enough for analysis and free of unresolved integrity breach.

Leadership checkpoint Sponsors receive evidence summaries without directing case exclusion.

Human control point Pilot and incident leads can pause or terminate runs.

Failure modes and safeguards Selective screenshots and polished demos cannot replace complete run records.

Change, stop or escalation condition Suspend after integrity, permission, participant or material service failure.

Handoff to the next step Pass all runs and deviations to Step 09.

Step 09: Test failure, abuse, degradation and recovery

Purpose and decision served Determine whether foreseeable failure can be detected, contained and recovered inside tolerance.

Accountable owner Service owner and risk executive.

Delegated lead and contributors Reliability, security, privacy, domain, incident, supplier and operating specialists.

Preconditions, inputs and access Authorised failure catalogue, safe environment, recovery plan and qualified oversight.

Actions

1. Run selected boundary, malformed, unavailable, adversarial and changed-condition cases.
2. Test detection, alert, human escalation, interrupt and revocation.
3. Exercise fallback, rollback, reconciliation and recovery evidence.
4. Record residual exposure and untested scenarios.

Evidence to retain Failure IDs, expected and actual response, trace, impact, recovery and unresolved gaps.

Decision or scoring rule One unacceptable undetected or unrecoverable material failure can stop the pilot.

Output and template A2P-FAL-* and A2P-REC-* records.

Acceptance gate Decision-critical failure and recovery requirements are demonstrated or visibly failed.

Leadership checkpoint Leaders accept residual exposure without netting it against benefit.

Human control point Qualified owners control potentially harmful tests and containment.

Failure modes and safeguards A scenario exercise is not represented as a real incident or production proof.

Change, stop or escalation condition Contain immediately when a test escapes its boundary.

Handoff to the next step Pass failure evidence to Step 10.

Step 10: Reconcile outcome, burden and operating economics

Purpose and decision served Judge whether observed value survives quality, human work, support, failure and lifecycle cost.

Accountable owner CFO and benefit owner.

Delegated lead and contributors Analyst, evaluation, operations, finance, support, change and source owners.

Preconditions, inputs and access Complete baselines, runs, failures, human effort, supplier and technical resource evidence.

Actions

1. Calculate pre-registered measures with uncertainty and exclusions.
2. Separate released, redeployed and absorbed capacity from cash.
3. Include verification, exception, support, integration, evaluation and change burden.
4. Reconcile contradictions, affected-group results and concurrent changes.

Evidence to retain Calculation versions, source data, reconciliations, sensitivity, dissent and limitations.

Decision or scoring rule Do not extrapolate beyond the accepted population, period, mode or operating model.

Output and template A2P-ECO-* records in the Economics Decision Handoff and Closure Register.

Acceptance gate Every decision-critical result retains comparator, burden, proof status and attribution limit.

Leadership checkpoint The CFO and COO accept what would need to change operationally for value to exist.

Human control point Source and benefit owners approve definitions and treatment.

Failure modes and safeguards Supplier labour, hidden review and temporary enthusiasm remain visible.

Change, stop or escalation condition Validate contradictions that could reverse the decision.

Handoff to the next step Pass reconciled evidence to Step 11.

Step 11: Make the pilot disposition and production-design handoff

Purpose and decision served Convert pilot evidence into one controlled human decision without accidental production.

Accountable owner Executive sponsor with COO, CIO, CFO and risk authority.

Delegated lead and contributors Decision secretary, evaluation lead, operating owner, evidence custodian and The Automation Production Operating Standard: Integration, Monitoring, Incidents and Change Control receiving owner.

Preconditions, inputs and access Complete decision pack, acceptance results, failures, economics, dissent and open gaps.

Actions

1. Compare every hypothesis and guardrail with its pre-registered rule.
2. Record rejected alternatives, exceptions and unresolved uncertainty.
3. Assign one pilot disposition and exact evidence boundary.
4. Obtain receiving-owner acceptance for production design or corrective work.

Evidence to retain Signed disposition, decision pack, conditions, dissent, handoff and acknowledgement.

Decision or scoring rule No weighted result offsets a mandatory failure, and evidence support never equals production release.

Output and template A2P-DEC-* and A2P-HND-* records.

Acceptance gate EVIDENCE SUPPORTS PRODUCTION DESIGN, EXTEND EVIDENCE WITHIN CHARTER, CHANGE AND RE-PILOT, VALIDATE A SPECIFIC CLAIM, DEFER or STOP.

Leadership checkpoint The forum states what the pilot proved, did not prove and cannot generalise.

Human control point Named executives decide and The Automation Production Operating Standard: Integration, Monitoring, Incidents and Change Control owner accepts only a design gate.

Failure modes and safeguards Pilot success cannot be relabelled production acceptance.

Change, stop or escalation condition Re-charter any proposed extension that changes exposure or decision.

Handoff to the next step Pass decision and assets to Step 12.

Step 12: Close pilot assets, access, claims and review

Purpose and decision served End temporary authority and preserve a controlled evidence record for the next gate.

Accountable owner Pilot sponsor and CIO.

Delegated lead and contributors Environment, data, identity, supplier, evidence, finance, service and communications owners.

Preconditions, inputs and access Signed decision, asset inventory, credentials, data copies, contracts, communications and retention rules.

Actions

1. Revoke pilot identities, credentials, tools, writes and supplier access.
2. delete or retain data and logs under accepted authority.
3. Close environments, temporary workarounds, claims and spending.
4. Preserve evidence, learning and a trigger for re-charter.

Evidence to retain Revocation, deletion or retention, environment closure, supplier closure and final evidence index.

Decision or scoring rule Nothing remains active unless named in a separately accepted downstream boundary.

Output and template A2P-CLS-* records linked to all pilot assets.

Acceptance gate CLOSED, TRANSFERRED UNDER AUTOMATION PRODUCTION OPERATING BOUNDARY, RETAINED BY AUTHORITY or ESCALATE.

Leadership checkpoint The sponsor accepts the final evidence statement and prohibited claims.

Human control point Asset, data and evidence owners attest closure.

Failure modes and safeguards An idle pilot cannot become shadow production through forgotten credentials.

Change, stop or escalation condition Escalate any asset or access that cannot be accounted for.

Handoff to the next step Return material new questions to a new Step 01 charter.

4. The Evidence & Measurement Checklist

Preflight and authority

• One production uncertainty and one decision forum are chartered. Evidence: pilot charter. Owner: executive sponsor.
• Negative and inconclusive results can change the decision. Evidence: disposition mandate. Owner: decision forum.
• Readiness, portfolio, architecture and governance prerequisites are linked. Evidence: source handoff. Owner: pilot lead.
• Pilot authority is separated from production release. Evidence: scope register. Owner: CIO.
• Sponsor, supplier and evaluator conflicts are disclosed. Evidence: conflict register. Owner: independent challenger.

Evidence access and quality

• Baseline population, period, unit, denominator and exclusions are accepted. Evidence: baseline contract. Owner: source owner.
• Business-as-usual and do-minimum alternatives remain visible. Evidence: counterfactual register. Owner: benefit owner.
• Cases retain source, version, inclusion and proof status. Evidence: case register. Owner: evaluation owner.
• Failed and repaired runs preserve original evidence. Evidence: immutable run record. Owner: evidence custodian.
• Missing evidence remains REQUIRES VALIDATION. Evidence: evidence-gap rows. Owner: decision owner.

Method execution

• Hypotheses and reject conditions are registered before runs. Evidence: hypothesis register. Owner: evaluation owner.
• Normal, boundary, exception, adversarial and changed conditions are represented. Evidence: case allocation. Owner: domain lead.
• Offline, shadow, supervised and recovery modes have separate gates. Evidence: mode register. Owner: CIO.
• Metric amendments are versioned and cannot hide adverse results. Evidence: analysis history. Owner: proof owner.
• Mandatory failures cannot be averaged into a success score. Evidence: decision rules. Owner: risk executive.

Governance, privacy, security and risk

• Pilot identities, data, tools, writes and environments use least privilege. Evidence: permission map. Owner: security owner.
• Participant, customer and employee safeguards are accepted. Evidence: qualified approval. Owner: operating owner.
• Human review, override and fallback are feasible under observed load. Evidence: rehearsal record. Owner: COO.
• Interrupt, rollback, revocation and incident authority are executable. Evidence: recovery test. Owner: service owner.
• Qualified owners decide specialist legal, privacy, security, safety and employment matters. Evidence: referral log. Owner: sponsor.

Deliverables and acceptance

• All four pilot working files use stable linked IDs. Evidence: pack index. Owner: pilot custodian.
• Every run maps to charter, case, mode, version and human response. Evidence: run register. Owner: pilot lead.
• Evaluation includes outcome, burden, failure, recovery and control evidence. Evidence: decision pack. Owner: evaluation owner.
• Economics includes verification, exception, support and change burden. Evidence: economics register. Owner: CFO.
• Receiving owner accepts a bounded production-design or corrective gate. Evidence: handoff receipt. Owner: downstream owner.

Measurement

• Learning, operating, outcome and guardrail measures remain separate. Evidence: measurement register. Owner: benefit owner.
• Measures state baseline, source, definition, owner, cadence and response. Evidence: metric contracts. Owner: analyst.
• Group and boundary failures remain visible beside aggregate results. Evidence: evaluation report. Owner: evaluation owner.
• Capacity is classified as released, redeployed or absorbed before financial treatment. Evidence: benefit record. Owner: CFO.
• Attribution and generalisation limits accompany every result. Evidence: decision pack. Owner: proof owner.

Decision and handover

• Every hypothesis receives an accepted, rejected or inconclusive result. Evidence: hypothesis disposition. Owner: decision secretary.
• Pilot disposition, resource decision and production authority remain separate. Evidence: signed decision. Owner: sponsor.
• Dissent, failures, exclusions and rejected alternatives are retained. Evidence: decision record. Owner: independent challenger.
• The Automation Production Operating Standard: Integration, Monitoring, Incidents and Change Control receives evidence for design, never an implied release. Evidence: handoff boundary. Owner: CIO.
• Public claims cannot exceed accepted evidence boundaries. Evidence: claim control. Owner: proof owner.

Review, change and stop controls

• Material scope, population, mode or measure changes require re-charter. Evidence: change record. Owner: pilot sponsor.
• Material incidents suspend pilot cadence for containment. Evidence: incident record. Owner: service owner.
• Pilot credentials and supplier access have closure evidence. Evidence: revocation register. Owner: CIO.
• Data and logs are deleted or retained by named authority. Evidence: custody closure. Owner: information authority.
• No pilot asset remains active without separate downstream acceptance. Evidence: final asset reconciliation. Owner: pilot sponsor.

Measurement contracts

Measure A: Evaluation population coverage

Measure B: Accepted task outcome performance

Measure C: Human verification and exception burden

Measure D: Recovery evidence

Measure E: Accepted service or market outcome movement

Measure F: Material harm or control exception

Evidence classifications used in this guide

• VERIFIED: supported by retained decision-fit organisational evidence or authoritative material within scope.
• INFERRED: a reasoned interpretation linked to evidence with alternatives and limitations visible.
• PROPOSED: a hypothesis, case, threshold, mode or method awaiting authorised acceptance.
• REQUIRES VALIDATION: a material claim or condition that cannot support the pilot or production decision yet.

Sources and evidence notes

1. NIST: AI test, evaluation, validation and verification supports reliable measurement and evaluation of AI technology and use. VERIFIED official source.
2. NIST: Human-Centered AI describes ARIA evaluation layers of model testing, red-teaming and field testing. VERIFIED official source; this guide does not claim ARIA conformance.
3. NIST: ARIA Pilot Evaluation Report reports an exploratory sociotechnical pilot and its limitations. VERIFIED official source.
4. U.S. GAO: Technology Readiness Assessment Guide is a stable official publication identifier for the relevant-environment discipline. REQUIRES VALIDATION pending dated proof-owner capture because automated access returned 403; local pilot decisions are not GAO TRAs.
5. GOV.UK Service Manual: Using moderated usability testing supports task-based observation with actual or likely users and appropriate test conditions. VERIFIED official guidance within its context.
6. NIST SP 800-218: Secure Software Development Framework supports integrating secure development practices and supplier communication into software lifecycles. VERIFIED official source.

The dispositions, twelve-step method, working tools, measurement contracts and visual structures are PROPOSED Abrasim methods. They are not external certification, production evidence or a guarantee of value.