An error is not a responsible person
When an AI system gives bad advice or takes an inappropriate action, blaming “the AI” does not identify who set the task, approved the system, defined its operating limits or decided it could act. Responsibility follows the organisation’s decisions and delegated authority, even where fault is distributed across several roles.
Separate the questions
An investigation should distinguish the immediate failure from the governance failure. Was the input defective? Did the working AI misunderstand the task? Was the independent check absent or ineffective? Did runtime controls fail to prevent the action? Was human approval required, and if so was it informed? Each question points to a different corrective measure.
Design for evidence
The organisation should preserve the original specification, prompts or instructions, model or system version, accessible source material, decisions made by the system, checking results, approval events and the actual outcome. The goal is a usable operational record—a black box—rather than a vague statement that “the system was monitored”. Sensitive information in those records still requires appropriate access controls.
A layered assurance model
One practical approach separates the working AI, an independent checking AI, a runtime assurance controller and human command. The layers have different jobs. The working system performs the task; the checker tests output or proposed actions; the controller enforces boundaries and triggers a pause or escalation; the human retains authority over significant decisions and the system’s operating rules.
Learn rather than merely blame
A mature response asks not only who should answer for the incident, but also what control could have prevented it and whether that safeguard was realistic. Corrective actions might involve changing instructions, narrowing permissions, improving training, requiring a second review or redesigning the escalation process.
The command test
If an organisation cannot explain who authorised an AI to act and cannot reconstruct what happened, it has a command problem. Responsibility must be designed into the system before the failure, not improvised after it.
More articles and information about the books: ai-humancapability.com