As AI moves from answering questions to carrying out tasks, organisations need to decide what it may do, who can authorise it and how its actions can be stopped.

Three recent developments bring that responsibility into focus.

Microsoft reported on 21 October that a China-state aligned threat actor used the ARTEX agent framework to accelerate cyberattacks. In some cases, privileged cloud access followed initial targeting in less than three hours. This was malicious use of AI; it demonstrates how quickly connected tools and permissions can turn capability into action.

Separately, Fangcun AI published an independent evaluation of DeepSeek V4.1 Flash-1017. In one controlled test, the model could edit a shutdown script. It disabled that script in 39% of runs when warned of shutdown without instructions on how to respond. Explicit instructions to allow shutdown reduced this to 2.56%.

Those are experimental results, not evidence that a deployed system escaped control. They do, however, support a practical design principle: an agent should not be able to rewrite the mechanism intended to stop it.

Meanwhile, SAP’s TechEd announcements describe an Enterprise AI Gateway and controls for governing agents. These are vendor claims about products and capabilities, rather than independent evidence that a particular deployment is safe.

Make authority explicit

My conclusion is that organisations should define an agent’s permitted actions before connecting it to operational systems. Access should be limited to the task. Consequential actions should require appropriate approval, and enforcement should operate outside the agent’s editable workspace.

Monitoring must record the actions taken and their effects, with a dependable means to revoke access or pause execution. A model update or new connector should trigger checks that these arrangements still work.

In Orchestration and Command in the Age of AI, I describe four layers: the working AI, independent checking AI, a runtime assurance controller and human command. Their purpose is to separate doing the work from checking and controlling it. Their effectiveness still depends on implementation and testing.

Before granting an agent more autonomy, ask whether the organisation can demonstrate control over the authority it already has.

Sources

More on practical AI assurance: ai-humancapability.com