AIG-023 AI System Override and Safe-State Mechanisms
Description
Each production AI system has a documented mechanism by which an authorised operator can reject or override an individual output, suspend AI-assisted processing and fall back to a manual procedure and deactivate the system into a defined safe state. The procedures name who may invoke each mechanism and are accessible to those operators. Each mechanism is exercised at defined intervals and the test record states the outcome, the time taken and whether the action was completed by the organisation's own operators or required the supplier.
Rationale
A system that cannot be stopped by the people responsible for it is not governable. The dependency that bites is the supplier: an override that means raising a support ticket is not an override during an incident. Recording whether the supplier was needed turns that into an observable the tier model and the profile can set a threshold against, in place of a tier reference inside the control. Overriding an individual output carries no penalty for the operator, which is a management commitment rather than a system property. AIG-022 holds the oversight role that exercises it. BCM-002 holds the manual fallback procedure the suspension falls back to.
Applicability (9 profiles)
The mechanism is built by the provider. Naming who may invoke it, exercising it at intervals and recording whether the supplier was needed are the deployer's.
Art.14(4) names the stop mechanism as one of the capabilities the system provides to the oversight person, intervening or interrupting operation and bringing the system to a safe state, which puts the mechanism inside what is examined for conformity rather than in operational readiness. Art.15(4) supplies the other half, resilience to errors, faults and inconsistencies with technical and organisational measures that may include redundancy and fail-safe mechanisms, plus minimisation of biased feedback loops where the system keeps learning after deployment. So the safe state has to be defined and reachable and not only the control present.
The mechanism is built by the provider. Naming who may invoke it, exercising it at intervals and recording whether the supplier was needed are the deployer's.
Framework Mappings (15)
| GRC-15 | Human supervision | informative |
| TVM-13 | Guardrails | informative |
| EU-AI-Art.14.2 | Human Oversight — Capabilities Assigned to Oversight Persons | full |
| EU-AI-Art.15.2 | Accuracy, Robustness and Cybersecurity — Resilience and Fail-Safe Design | partial |
| CP-12 | Safe Mode | informative |
| SI-17 | Fail-safe Procedures | informative |
| GV-1.3-007 | Risk Management Activity Level Determination | GV-1.3-007 | partial |
| GV-1.7-001 | AI System Decommissioning Processes | GV-1.7-001 | full |
| GV-6.2-006 | Third-Party Failure Contingency Processes | GV-6.2-006 | informative |
| MG-2.4-002 | AI System Deactivation and Override Mechanisms | MG-2.4-002 | informative |
| MG-2.4-004 | AI System Deactivation and Override Mechanisms | MG-2.4-004 | partial |
| MS-2.6-005 | AI System Safety Risk Evaluation | MS-2.6-005 | informative |
| MANAGE 2.4 | AI System Deactivation and Override Mechanisms | full |
| ASI08 | Cascading Failures | partial |
| ASI10 | Rogue Agents | informative |
Evidence (2)
Documented and tested override, suspension, and safe-state deactivation procedures for each production AI system, including test results confirming the mechanisms function as designed.
Example: AI System Override Test Record · Content Moderation Engine v3 (Confluence): override test performed 2026-01-15 by Ops Lead, individual output rejection confirmed functional, fallback to manual queue confirmed operational within 2 minutes, full deactivation test result: endpoint offline in 45 seconds, rollback verified
Test: Request the override and safe-state test records for a sample of production AI systems. Verify: (1) all three mechanisms were tested, namely individual output override, suspension with manual fallback and deactivation into the safe state, (2) the test falls within the defined interval, (3) the record states the outcome and the time taken for each mechanism, (4) the record states whether deactivation was completed by the organisation's own operators or required the supplier, (5) the procedures are documented and accessible to the operators named as authorised.
Event log of override, suspension and deactivation actions on production AI systems, showing each exercise of the mechanism with its actor and timing.
Example: AI control-plane action log, 2026-01-01 to 2026-08-31: 12 override events, 3 suspensions, 2 deactivation tests, each with actor, system, start and completion timestamps
Test: Export the override, suspension and deactivation events for the period. Verify: (1) each production system in the inventory has at least one recorded exercise of each mechanism within the defined interval, (2) each event records the actor, the system and the start and completion timestamps, (3) the elapsed time is recorded so the test record's stated duration can be checked against it, (4) each event records whether the action completed under the organisation's own credentials or required the supplier, (5) a system with no event in the period is reported rather than omitted.
Questions (2)
Is there a documented override and safe-state procedure for each production AI system?
AI systems that cannot be safely stopped or overridden are ungovernable. Override, suspension, and deactivation procedures must be documented, accessible to operators, and tested at least annually, not left to vendor support.
Which of the following override and safe-state capabilities have been tested within the defined interval for your production AI systems?
The test record is the evidence, not the documented procedure. The fourth item is the one that decides whether the mechanism works during an incident: a deactivation that requires a supplier support ticket is not available at the moment it is needed.