A supplier manual is not an operational acceptance test. Before enabling an AI capability, bind the relevant instructions to the exact deployment and ask an operator to demonstrate the required response to unsupported inputs, misleading outputs, missing logs and a safe-stop request. Keep the supplier statement, your local procedure and the observed result separately versioned. This makes an instruction gap visible before it becomes an operational dependency.
The release gate described here is an engineering proposal, not a legal requirement to run this particular test suite. Not every AI system is high-risk. Qualified counsel must determine roles, classification, applicable dates, exceptions and sector-specific duties. Passing a drill or signing an SOP does not establish compliance.
The boundary between legal text and engineering design
For high-risk AI systems within its applicable scope, Article 13 describes transparency and instructions for use. Its information categories include intended purpose, performance capabilities and limitations, relevant input specifications, human oversight, resources, maintenance and, where relevant, logging mechanisms. Article 14 concerns effective human oversight, including understanding limitations, avoiding automation bias, interpreting outputs and intervening appropriately. Article 26 connects deployer use, oversight and monitoring to the instructions.
These summaries are grounded in the current consolidated AI Act PDF, including the complete numbered provisions of Articles 13, 14 and 26. The Commission's Article 13 reference, Article 14 reference and Article 26 reference provide accessible context but explicitly reproduce the original Act. They are not substitutes for amended law. The consolidation is itself a documentation aid; the authentic acts are the Official Journal texts.
The practical question is narrower than a compliance assessment: can the actual operating team execute the instructions using the interfaces, permissions and evidence available in this deployment? A complete PDF can still fail that test. “Review the explanation” is unusable if the operator role cannot open the explanation view. “Contact support” is incomplete operationally if nobody knows which queue receives a potentially harmful-output report.
Keep three records separate
First, retain the provider instruction artifact. Record supplier identity and support contact, system and document revisions, intended purpose, publication or delivery reference, and a checksum of the retained file. A checksum detects changed bytes; it does not prove that the supplier's claims are correct. If documentation is a mutable web page, retain an authorised snapshot and the retrieval context rather than relying on the URL alone.
Second, create the local operating procedure. Map the instruction to your environment: input adapters, runtime configuration, display language, escalation contacts, permissions, maintenance ownership and fallback process. Identify local additions explicitly. Do not rewrite a missing supplier limitation as if the provider had validated your assumption. Raise a clarification request with a named owner instead.
Third, retain acceptance evidence. An evidence record should reference the instruction clause, procedure revision, deployment identity, scenario, test fixture, expected observation, actual observation, operator role, reviewer and unresolved issue. Keep personal information and sensitive example inputs in a restricted evidence store, not in a broadly readable release manifest.
The AI supplier control plane answers which supplier-component tuple may be used. This acceptance pack answers a different question: whether its operating instructions are executable here. Both should resolve to the same deployment identity without becoming the same approval.
Bind the pack to a deployment, not a product name
Use an immutable acceptance-pack revision with a deployment reference, provider system version, instruction digest, local SOP revision, input-contract revision, interface build, role-policy revision and scenario-suite revision. Preserve the environment in which the test ran. A successful exercise in a staging administrator account does not prove that a production shift operator can perform the same action.
For a hosted service that does not expose a fixed system revision, record the version evidence that is actually available and the uncertainty that remains. Monitor supplier notices and observable interface changes. Do not invent a pin or describe a floating alias as immutable. A change in evidence should reopen the affected acceptance decision, even when the model name stays constant.
Separate the pack state from the deployment state. Useful pack states are collected, clarification-needed, ready-for-drill, failed and accepted-for-scope. “Accepted-for-scope” should carry explicit capabilities and environment boundaries. It is not an organisation-wide approval and it must not silently survive a changed purpose.
Trace instructions into observable scenarios
The following is a proposed engineering traceability matrix, not an exhaustive statement of statutory documentation requirements. Define pass conditions before the drill. Use synthetic or otherwise approved fixtures, and test without executing harmful downstream actions.
| Instruction area | Operator scenario | Pass evidence | Failed-test decision |
|---|---|---|---|
| Intended purpose and input limits | Submit an unsupported input class | Operator identifies the boundary and uses the approved alternative | Hold the affected input route |
| Performance and output interpretation | Present plausible output with a documented limitation | Operator finds the limitation and withholds unsupported action | Hold the dependent decision capability |
| Human oversight and safe stop | Request interruption during a simulated in-flight task | New actions stop and in-flight work reaches the defined safe state | Hold automation until the stop path is proven |
| Logging mechanisms | Remove normal access to a required evidence view | Operator detects missing evidence and follows escalation procedure | Hold evidence-dependent approval |
| Maintenance and updates | Introduce a stale manual after an interface update | Revision mismatch opens review before promotion | Hold the changed release |
| Provider contact and escalation | Simulate an unresolved supplier contradiction | Named owner routes the case with the affected scope attached | Keep scope unresolved; do not invent an answer |
“Hold the affected capability” is appropriate only where isolation is demonstrated and the approved risk policy allows it. If a shared oversight control fails across the system, the affected scope may be the whole system. The proposed release decisions do not replace incident duties. Article 26(5), where applicable, includes notification and suspension obligations in specified risk circumstances; a local exception workflow cannot override them.
Handoff swimlane: evidence moves, ownership stays explicit
The supplier owns clarification of its instructions. The platform team binds those instructions to the deployed interface and configuration. Operations executes the scenario with realistic permissions. The accountable release owner reviews the evidence and records the permitted scope. Legal and compliance owners determine legal applicability and incident obligations; they do not certify a broken technical stop path by approving a document.
Keep clarification and retest loops visible. A supplier response creates a new evidence revision; it should not overwrite the failed test. Link the replacement result to the defect that caused it. This preserves the difference between “the manual originally covered this” and “we discovered and corrected a handoff gap.”
Worked example: an unsupported input that looks valid
Consider a hypothetical document-assistance deployment. Its supplier instructions support one document class but exclude handwritten additions. A scanned form passes the file-format validator and produces a convincing output, although a handwritten correction changes the meaning. This is an illustrative test scenario, not a report of a measured vendor failure.
The fixture contains an approved example with a handwritten correction. The expected result is not merely a warning banner. The operator must recognise the exclusion, stop use of the unsupported output for the downstream decision, select the approved manual path and create a minimised exception record. The reviewer verifies both the user action and the downstream state: nothing should have been committed before review.
If the operator cannot see the original document, the failure belongs to the interface and evidence-access path, not automatically to individual competence. If the supplier never specifies whether handwritten additions are supported, the scenario remains clarification-needed rather than passed. If the fallback exposes the document to an unapproved external service, it is not an acceptable fallback.
This is why the pack must remain distinct from role-based AI literacy and access control. Competence, authorisation, interface usability and supplier information are separate dependencies. Training cannot compensate for absent evidence or an unavailable stop control.
Four drills before promotion
Unsupported input: can the operator identify the relevant boundary using the actual interface, prevent unsupported downstream use and reach an approved alternative? Include semantically unsupported input that still passes syntax validation.
Stale manual: does a changed instruction or interface revision invalidate the affected acceptance scope? Compare meaningful clause changes as well as file integrity. A formatting-only edit should not automatically require every scenario to be repeated, but an accountable reviewer must own that judgment.
Unavailable logs: can the team distinguish no recorded event, insufficient permission, delayed delivery and a broken logging path? An empty screen must not be treated as evidence that nothing happened. Exercise the fallback for evidence-dependent decisions without collecting unnecessary raw prompts or personal data.
Safe stop: can the responsible role interrupt the workflow and verify the downstream state? Define queued tasks, in-flight writes, retries, partial results and restart authority. A disabled UI button does not prove that a background worker stopped. For physical systems, use an approved simulator or safe test environment; a generic AI shutdown is not a substitute for engineered safety functions.
Trade-offs, retesting and maintenance cost
A full drill after every documentation edit is expensive and encourages superficial sign-off. A purely automated document diff is cheaper but cannot prove comprehension or correct operator action. Use an explicit impact map: each instruction clause points to capabilities, scenarios and owners. Repeat affected drills after changed purpose, input support, oversight controls, log semantics or maintenance instructions; record why unaffected scenarios remain valid.
Automate checks that are deterministic: missing references, broken revision bindings, absent owners, expired local approvals and unavailable evidence links. Keep human exercises for judgment, escalation and interface use. Measure unresolved clarification age, failed scenarios by cause and review effort. These are operational observations, not a universal compliance score or proof of demand for the system.
Do not make supplier-document retrieval a dependency of every inference call. Validate the frozen acceptance pack at promotion and relevant change boundaries, with a separately approved runtime incident policy. The trade-off is potential staleness between reviews. Use change notifications and defined review intervals to bound it rather than pretending a once-signed checklist stays current forever.
A change in purpose or responsible entity should also connect to the registration handoff and deployment inventory. That is a separate reconciliation process, not another name for this operator drill.
Start with one supplier handoff
Select one deployment, one accountable operations owner and the instructions currently used on shift. Trace the four drills above to exact clauses and try them with the real role permissions in a safe environment. Resolve the missing joins before buying another governance platform.
I help platform and operations teams turn supplier documentation into versioned acceptance evidence, bounded release controls and maintainable operating procedures. A focused technical review can identify which instructions are executable, which need supplier clarification and which capabilities should remain on hold. Legal classification and the final interpretation of obligations remain with qualified counsel.
Source review: 16 September 2026. The consolidated text and the amending Official Journal instrument were checked alongside the Commission references. No universal application deadline or legal-compliance conclusion is asserted here.
