An organisation may keep a human responsible for an AI-assisted decision while giving that person too little information or time to check it. Responsibility alone does not make oversight effective.
Open Rights Group’s October investigation into AI tools used in Home Office asylum casework illustrates the concern. Its report draws on parliamentary answers, ministerial correspondence and freedom of information responses.
ORG notes that the Home Office’s pilot evaluation found 9% of generated interview summaries were inaccurate or missing information and were removed from the pilot. It also reports that specialist subject matter experts checked just 0.3% of summaries generated between April and July 2026.
The figures describe different stages and different forms of checking. The specialist-checking figure does not establish that all remaining summaries went unread by caseworkers. Nor does a defective summary automatically establish that a final decision was wrong. But the findings raise a serious question about how errors are detected before they influence decisions.
Check the evidence behind the summary
A fluent summary can make a case easier to process while concealing an omission. Someone reviewing only that summary may never see the detail needed to challenge it.
My view is that meaningful oversight requires access to the original evidence, a clear method for checking significant claims and enough time to investigate discrepancies. The reviewer must also be able to reject the output, request further evidence or pause the decision without being penalised for slowing the process.
Checks should reflect the consequences of error. A routine internal note and a decision affecting someone’s rights require different levels of assurance.
Preserve the capability to challenge
There is also a longer-term management responsibility. If staff increasingly receive finished AI outputs, employers must deliberately preserve opportunities to practise the reasoning needed to assess them. This is a risk organisations should address, rather than an outcome proved by the asylum report.
That means supervised work with original evidence, discussion of disputed cases and occasional assessment without AI assistance. Productivity measures should recognise the quality of checking as well as the speed of completion.
This connects the command arrangements in Orchestration and Command in the Age of AI with the human capability concerns explored in Succession. A named human remains effective only while that person has the competence, information and authority to intervene.
For any consequential AI-assisted decision, ask: what would enable the reviewer to discover that this output is wrong?
Source
More on human command and capability: ai-humancapability.com