When an AI Agent Can Act, Who Holds the Keys?

A quiet change in the meaning of “assistant”

Imagine planning a short business trip. You ask an AI assistant to find a convenient flight, check your calendar and suggest a hotel. A conversational system can give you a comparison and a draft itinerary. An agent connected to tools might open booking pages, create calendar events, email a colleague or move money to pay for a reservation. From the user’s perspective these experiences can look almost identical until the final step, when a suggestion becomes a commitment.

This is what makes agents different from a chatbot that only produces text. NIST describes contemporary agent systems as models supported by software that lets them use tools and take actions in their environments [1]. The language model may decide which tool to call, examine the result and take a follow-up step. An apparent conversation is therefore capable of becoming a chain of operations across multiple services.

The same underlying model can be harmless in a read-only search experience and powerful in an application connected to private files, email and payment systems. What matters is not only how often the model gets the answer right. It is how much the surrounding software lets it do when the answer is wrong.

What “access” really means

A permission screen often compresses a complex question into a few familiar labels: read, write, send, publish or delete. Read access may expose sensitive documents. Write access can alter a shared spreadsheet. Send access can contact a customer in your name. Publish access can put an unreviewed statement on a public website. Delete access may be irreversible if there is no recovery system.

There is also the difference between access to one item and access to an entire account. An agent asked to find one meeting does not need to read every personal email. One preparing a newsletter does not necessarily need rights to modify the company website. In security engineering this is the principle of least privilege: give a process only the abilities required by the task, ideally only for the time they are needed. OWASP identifies excessive permissions, functions and autonomy as the three common roots of “excessive agency” [2].

Scope matters as much as the permission type. Editing one document is not the same as editing every document in a drive. Sending a draft to its author is not the same as mailing a whole contact list. A responsible design should make those boundaries visible to the person giving consent.

The essential distinction: a useful suggestion is not the same as permission to act.

The danger hiding inside an ordinary webpage

Here is a more surprising risk. Suppose a travel-planning agent opens a hotel page to check cancellation conditions. Buried inside that page is text instructing the AI to ignore its original task and send the user’s itinerary to an unfamiliar address. To a person reading the page, the instruction may look absurd. To a system processing outside text as part of its working context, it creates a potential conflict between trusted instructions and untrusted information.

The technique is known as indirect prompt injection. The attacker is not talking to the assistant as the legitimate user. Instead, the attacker places instructions in something the assistant was supposed to inspect as data: an email, search result, repository file or website. NIST has demonstrated how such material can redirect tool-using agents into actions not requested by the user [3].

Not every attempt succeeds, and the outcome depends on the model, the task and the controls around it. But the important lesson is architectural. The source of a piece of text is part of its trustworthiness. A webpage describing a hotel room cannot also become the authority that grants permission to send mail. If a system confuses those roles, external content may become a route to actions that were never intended.

Why clever wording is not enough protection

A developer might tell an AI system, in a firm instruction, never to obey commands found on a webpage. That can help, but it is not a substitute for technical controls. The agent still has to process unpredictable content. Attackers can hide instructions inside apparently relevant information or disguise them as routine work. NIST’s research shows why evaluating a single, familiar test attack is not enough: new attacks may behave differently, and repeated attempts can change the risk picture [3].

The strongest protection limits what can happen even when the model makes a mistake. If the agent can only read an itinerary, it cannot use the email tool to send it. If it may prepare a payment but cannot submit it, an injection cannot directly make the purchase. OWASP recommends restricting tool access, validating the intended operations and applying least privilege rather than relying on prompt wording alone [2][4].

This turns a vague promise of safety into a concrete boundary. The rule should live in the application and its credentials, not only inside the conversation. An approval button has little value if another available tool or token can perform the same action without approval.

Make “draft” and “do” two separate stages

One of the most useful patterns is a clear pause before significant actions. In stage one, an agent gathers evidence, compares options and prepares exactly what it proposes to do. In stage two, the person reviews the recipient, price, date, destination and any lasting change. Only after approval does the application execute the step. A genuine confirmation screen is not just a visual courtesy; it must stop the underlying operation until permission is received.

Consider a simple email. An agent may draft a polite answer to a customer in seconds. But the customer’s name might resemble another person’s, or a sensitive attachment might have been selected by mistake. Displaying the actual address, message and attachments creates a practical checkpoint. The same logic applies to making a purchase, submitting a form or deploying a new version of a website.

Automation need not be all or nothing. Organising items in a reversible folder may be safe to automate freely. Sending a public statement or deleting records deserves more scrutiny. The level of supervision should match the cost of a mistake, not how impressive the demonstration looks.

The invisible safety net: records, limits and recovery

Good systems leave an understandable trail. If an agent searches a database, edits a file or sends a message, the user or an administrator should be able to see what it did and when. An action history helps identify mistakes, investigate unusual activity and undo changes when recovery is possible. It also makes it easier to distinguish “the AI recommended this” from “the software actually performed this”.

Limits are another important tool. A rate limit can prevent an email agent from sending hundreds of messages in one unexpected burst. A spending limit can constrain a purchasing assistant. A narrow credential can be revoked without disrupting unrelated services. Separate tokens for reading and publishing reduce the risk of one task gaining rights meant for another [4].

None of those measures makes an agent infallible. They make failures smaller, more visible and more manageable. In everyday software, dependability is often the result of many mundane safeguards working together rather than one dramatic security feature.

How much autonomy is sensible?

Think of autonomy as a dial. At the lowest setting, the system explains options. Next it drafts changes for you to approve. Above that, it performs small, reversible operations within a defined scope. At the highest setting, it may operate across accounts and complete entire workflows without immediate supervision. Every increase in autonomy should be paired with stronger safeguards, clearer logging and better recovery options.

A research assistant reading public reports might reasonably work unattended. An agent handling legal documents, private information or financial transfers has a different risk profile. Even for a well-designed system, the correct choice depends on the sensitivity of the information, the reversibility of an error and how reliably the software can check its own work.

There is a reason the most trustworthy automation sometimes asks you a question. Stopping at the right moment is a feature, not a failure of intelligence.

Approval is not the same as oversight

Approval can fail if the interface hides the most important part of a transaction. A traveller might agree to a flight while overlooking the date, the final price, a passenger name or a cancellation condition. A person reviewing an automated email must be able to see its recipients and attachments, not merely a reassuring summary. The checkpoint only works when the user can inspect the real action the software is about to take.

That makes product design an important part of AI safety. Strong systems distinguish the request, the proposed action, the destination and the permission being exercised. They also show whether the result has actually happened. A green success message is not proof that a reservation was confirmed or a payment settled; confirmation needs to come from the external service involved. The principle is simple: verify the consequential step, not just the agent’s confident description of it.

Five questions to ask before switching an agent on

Before authorising any connected assistant, ask what it can see, what it can change and which outside services it can contact. Then ask whether each capability is really needed for this task. Does it have access to the entire mailbox or just a selected message? Can it only draft an email or can it send one? Can it spend money or publish publicly? Is there an audit trail? Can access be revoked quickly?

The answers should be specific. “The agent is secure” does not tell you much. “It can read selected calendar events, cannot send mail, and must obtain confirmation before booking” is far more useful. Security becomes understandable when it is described in terms of actions and limits rather than marketing language.

AI agents may become a normal part of work and daily life. The best products will be those that do the tedious work while keeping the right decisions with the right people. The real mark of a capable agent is not that it never asks. It is that it knows precisely when it must.

Sources & Further Reading

1. NIST, Lessons Learned from the Consortium: Tool Use in Agent Systems (2025) (https://www.nist.gov/news-events/news/2025/08/lessons-learned-consortium-tool-use-agent-systems)
2. OWASP, LLM06:2025 Excessive Agency (https://genai.owasp.org/llmrisk/llm062025-excessive-agency/)
3. NIST, Strengthening AI Agent Hijacking Evaluations (2025) (https://www.nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations)
4. OWASP, LLM Prompt Injection Prevention Cheat Sheet (https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html)

Related stories