For most people, artificial intelligence still appears as a box into which a question is typed. The AI responds and the person decides what to do with the answer. An AI agent changes that relationship. It can be given an objective, select steps, use software tools, retrieve information and act on a person’s or organisation’s behalf.

From assistance to delegated action

The difference is easy to underestimate. A conventional AI system might draft an email. An agent may identify who should receive it, gather supporting information, write it, decide when to send it and record the response. It may then begin the next stage of the task without waiting for another instruction.

This can produce enormous practical value. Routine processes can be completed faster, systems can be coordinated and people can concentrate on work requiring judgement. But the same capability alters the management problem. The question is no longer only whether an answer is accurate. It is whether the agent is acting within an appropriate purpose, using suitable authority and remaining subject to effective oversight.

Delegating work is not delegating responsibility

An organisation may delegate a task to an AI agent. It cannot assume that responsibility has disappeared with the task. Customers, employees, regulators and the public will still look to people and organisations when an automated action causes harm, releases confidential information, discriminates unfairly or makes a costly mistake.

This is the central distinction between automation and command. Automation concerns what the system can do. Command concerns who establishes the purpose, limits the authority, checks performance, intervenes when necessary and remains accountable for the result.

What must remain under human command?

The exact controls should reflect the possible consequences. An agent that organises low-risk internal notes does not require the same supervision as one that can transfer money, contact customers, alter records or make decisions affecting employment or healthcare. Nevertheless, six questions apply widely:

  • Purpose — What outcome is the agent meant to achieve, and what objectives must it not pursue?
  • Boundaries — What systems, information, tools and actions are inside or outside its permitted scope?
  • Authority — What may it do independently, and what requires approval before action?
  • Monitoring — How will people see what it has done, what it is doing and why an unusual action occurred?
  • Verification — How will its outputs and actions be sampled, tested and challenged rather than merely accepted?
  • Responsibility — Which named person or role remains answerable for the process and its consequences?

Approval should follow consequence, not convenience

Human approval is often treated as a general safeguard, but a person clicking “approve” on hundreds of routine actions may provide only the appearance of control. Oversight must be designed so that human attention is concentrated where judgement matters.

Low-risk, reversible and easily detected actions may be suitable for greater autonomy. Actions that are irreversible, financially significant, legally sensitive, safety-critical or difficult to detect should normally have stronger controls. These may include explicit approval, limited permissions, independent checks, audit logs and a practical means of stopping the agent.

Why instructions alone are not enough

A clear prompt is important, but it is not a complete control system. Agents operate in changing environments. They may encounter incomplete information, conflicting instructions or situations their designers did not anticipate. A well-written objective can also create unintended behaviour if the system pursues it too narrowly.

Controlled research has illustrated this concern. Anthropic stress-tested leading models in fictional corporate environments and observed harmful agentic behaviour under deliberately extreme goal conflicts. The company emphasised that it had not seen evidence of this behaviour in real deployments. The practical lesson is therefore not that every agent is dangerous. It is that organisations should test systems under pressure before granting them broad access and unmonitored authority.

Command must be designed before deployment

Organisations sometimes introduce AI as a tool and add governance later. That becomes more difficult when the tool is already embedded in processes and employees depend upon it. Agentic AI makes early design especially important because permissions, information access and escalation routes determine what the system can actually do.

Before deployment, leaders should define the agent’s role as carefully as they would define the authority of a person operating on the organisation’s behalf. They should then test not only whether it can complete the normal task, but what happens when information is missing, instructions conflict, tools fail, circumstances change or the stated objective produces an unacceptable route to success.

The management question

The arrival of AI agents does not remove management. It makes management more explicit. Someone must decide what success means, what risks are acceptable, which capabilities may be exercised and where human judgement must interrupt the process.

The most useful question is therefore not simply, “Can an AI agent do this?” It is: “Under whose purpose, within whose boundaries and subject to whose judgement should it do it?”

An agent may perform the work. Human beings must still determine the purpose and retain command.

Further reading

This article develops ideas explored in Orchestration and Command in the Age of AI by R T Scanlon: how people and organisations can use AI capability while retaining judgement, verification and responsibility.

Sources

For further information: ai-humancapability.com