When AI stops answering and starts acting, a mistake can land before anyone checks it. For the teams putting agents into regulated work, here is what changes, and how to govern AI agents so you keep the automation without losing control.
By Marianna Printziou, Head of Product, TheFutureCats | 7 min read
Key takeaways – An AI agent does not just answer, it acts: it advances steps, makes decisions, and triggers systems. That is where the value is, and where the risk moves. – Industry research expects over 40% of agentic AI projects to be canceled by the end of 2027, largely because of inadequate risk controls, not weak technology. – An agent inherits the mistakes of the large language model (LLM) it runs on, so what keeps it safe is the governance built around it, by design, not added on afterwards. – Govern AI agents with four controls: scoped permissions, approval before critical actions, a full action log, and a human who can stop or undo. Automate the routine, hold the line where a wrong action would be costly.
When an AI answers, a person still decides what to do with it. When an AI agent acts, the decision is already made.
To govern AI agents, start from what makes an agent different: it acts. Instead of only answering a question, it advances a step in a process, makes a decision, or triggers another system. That is where the real automation lives, and also where control gets hard, because what matters is no longer only what the AI says but what it does. You get that automation safely by governing what the agent is allowed to do, and the controls below are how.
Industry research expects over 40% of agentic AI projects to be canceled by the end of 2027, due to escalating costs, unclear business value, and inadequate risk controls.* The projects that fail mostly do not fail on the model; they fail on the governance around it.
What follows is a short set of controls that let an agent run on its own where it is safe, and a scorecard you can hold any agentic deployment against.
In this article – What changes when AI acts instead of answers – Why agentic AI projects fail – How to govern an AI agent in regulated work – What regulators expect from AI agents – How to keep automation without losing control – How to evaluate an agent’s governance – FAQ
What Changes When AI Acts Instead of Answers?
The unit of risk changes. With a chatbot, a mistake is a wrong answer, and the person still decides what to do with it. With an agent, by contrast, the same mistake can be an action that has already happened: a record updated, a payment released, a case closed, a message sent, and you cannot always go back and correct it. The checkpoint that used to sit between the answer and the action can disappear. In a regulated process, that checkpoint is often the exact control an auditor asks about.
AI acts, the mistake has already happened.
Chained workflows teach this quickly: a silent error in an early step does not stay where it happened, it travels. The next step treats it as input, the one after builds on it, and by the end you have a clean, confident, completely wrong result that no single stage flags.
There is a deeper reason to worry. An agent is only as reliable as the model it runs on, and even the best LLMs are wrong more often than most teams assume: Stanford’s 2026 AI Index puts hallucination rates across 26 top LLMs between 22% and 94% (Stanford HAI, 2026). Grounding the model in your own internal knowledge with retrieval helps, of course, but it does not verify the output. So an agent acting on an unverified answer is acting on something no one has checked, which is why the controls that follow are not optional.
Why Do Agentic AI Projects Fail?
Because the governance is treated as an afterthought, and in regulated settings the risk-control gap is the decisive one. An agent that can act but cannot be limited, checked, or explained is not something a risk function will sign off, so it never leaves the pilot. The teams that get past that do the opposite: they build the guardrails first, and let the autonomy grow inside them.
How Do You Govern an AI Agent in Regulated Work?
You put four controls in place before the agent runs on anything that matters:
- Scoped permissions. The agent can act only inside an explicit boundary of systems, data, and actions, and anything outside that boundary is simply not available to it.
- Approval before critical actions. Low-risk steps run automatically, while the moves that are costly to get wrong pause for a named person to approve, on the record.
- A complete action log. Every action the agent takes is recorded in an immutable trail, so you can reconstruct later what it did, when, and on what basis.
- A human who can stop or undo. Someone can stop the agent mid-run and reverse what it did, and the thresholds that trigger a stop are defined in advance, not improvised.
Together, these four controls turn an agent from a black box that acts on its own into a glass box you can see into, so that every action is limited, visible, and reversible.
Four controls that let an agent act safely.
How you build these matters as much as which ones you pick. Governance designed into the agent from the start is a different thing from a guardrail added at the end to block a bad action after the model has already decided on it. An add-on like that can only react once the decision is made, while governance built in from the start decides, at each step, whether the agent should be acting at all, which makes it far harder to bypass and far easier to defend.
What Do Regulators Expect From AI Agents?
These controls line up with where regulation already points. The EU AI Act expects high-risk systems to keep logs and stay under human oversight, and DORA holds financial firms accountable for the resilience and traceability of the systems they run, agents included. And you cannot hand the accountability to the tool. Under the EU AI Act the obligations fall on the provider and the deployer, not on the model, and you cannot outsource them to your AI vendor, so “the AI did it” is not a defence to a regulator. That is the real reason a person has to own the critical calls. Under the EU’s Digital Omnibus, those high-risk obligations now apply from December 2027 for use-based systems and August 2028 for product-embedded ones (Council of the EU, 2026), which is time to build the controls in, not a reason to wait.
How Do You Keep Automation Without Losing Control?
You run the agent on the principle of control by exception. The routine, low-risk work runs end to end without a person, which is the point of automation and where the time and cost savings come from, while the exceptions, the actions where a confident wrong move would be expensive, route to a human who confirms or overrides. You are not trading autonomy for safety, you are getting automation with a net you control.
Control by exception: the routine runs on its own, the exceptions go to a human.
An agent that can act without limits is not a productivity gain, it is an incident waiting for a timestamp.
How to Evaluate an Agent’s Governance
Before you let any agent act in a regulated process, hold it against this scorecard.
What to check | What good looks like | Red flag |
Permissions | Explicit, scoped boundary of systems and actions | Broad or default access |
Approval | Critical actions pause for named human approval | Agent executes everything autonomously |
Action log | Immutable, reconstructable record of every action | No trail, or logs that can be edited |
Stop and undo | A human can stop and reverse, with defined thresholds | No kill switch, no rollback |
Escalation | Clear rules for when the agent must ask, not act | The agent decides its own limits |
If an agentic deployment cannot pass this scorecard, it is not ready to act on anything a regulator, an auditor or a board might later question.
The Real Takeaway
The shift from answering to acting is the moment AI stops being something you consult and becomes part of the process itself. That is worth a great deal, and it is the point where governance stops being paperwork and becomes the thing that lets you ship. Give the agent a boundary, a checkpoint on the costly moves, a full log, and a stop button, and you can automate with confidence instead of hoping nothing breaks.
Always keep in mind that something might break, and when it does, the question will not be why the agent did that, but why you let it. Governance is how you answer that question before it is asked, not after, and the teams that can answer it are the ones who get to let their agents act at all.
Putting agents into a regulated process? Book a working session with our team to run the five checks against your own agents and set the bar each one has to clear for your obligations.
FAQ
What is an AI agent, and how is it different from a chatbot? A chatbot answers questions, while an AI agent takes actions: it advances steps in a process, makes decisions, and triggers other systems. The difference matters because an agent can change the state of your business, not just inform it.
Why do most agentic AI projects fail? Industry research expects over 40% to be canceled by the end of 2027, largely because of unclear value and inadequate risk controls. In regulated settings the deciding factor is usually governance: an agent that cannot be limited or explained does not get approved to scale.
How do you govern an AI agent without killing the automation? Use control by exception. Let low-risk, routine actions run automatically, and require human approval only for the costly moves. Combined with scoped permissions, a full action log, and a stop-and-undo control, you keep the speed and keep the oversight.
What do the EU AI Act and DORA expect around autonomous agents? Broadly, a clear boundary on what a high-risk system can do, human oversight, an immutable log of what it did, and accountability for its resilience and traceability. Under the EU’s Digital Omnibus, those high-risk obligations now apply from December 2027 for use-based systems and August 2028 for product-embedded ones, and the four controls above are how you produce that evidence in practice.
*The projected cancellation rate is from Gartner (2025). Full reference in Sources below.
Marianna Printziou is Head of Product at TheFutureCats, where she builds AI products for regulated organisations and trains the teams that use them.
Sources – Gartner, “Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027” (2025): https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027 – Stanford HAI, 2026 AI Index Report, Responsible AI chapter (hallucination rates across 26 top LLMs range from 22% to 94%): https://hai.stanford.edu/ai-index/2026-ai-index-report/responsible-ai – Council of the EU, “Artificial intelligence: Council and Parliament agree to simplify and streamline rules” (7 May 2026), high-risk deadlines moved to December 2027 and August 2028: https://www.consilium.europa.eu/en/press/press-releases/2026/05/07/artificial-intelligence-council-and-parliament-agree-to-simplify-and-streamline-rules/ – EU AI Act, Article 26, obligations of deployers of high-risk AI systems (deployers cannot outsource responsibility to the AI vendor): https://artificialintelligenceact.eu/article/26/
