TL;DR: The moment an AI system can change something in your business instead of just describing it, the project stops being about model quality and becomes about blast radius. Sort every action the agent can take by what it costs to undo, put human approval at the line where recoverability ends, give the agent its own scoped account rather than a copy of somebody's admin access, and assume any text it reads could be trying to redirect it. Then ship it read-only first, log what it would have done, and hand over real write access one action class at a time.
Most AI projects start on the reading side. Summarize this, answer that, find the clause, draft the reply. Then somebody asks the obvious follow-up: if it already knows the answer, why is a human retyping it into the CRM?
That question is where an assistant becomes an agent, and it quietly changes what you are buying. An assistant that gets something wrong produces a bad sentence. An agent that gets something wrong produces a booked appointment, a sent message, a changed record or a refund. Same model, same prompt quality, completely different project.
Answering and doing are different products
There are three levels, and naming which one you are commissioning is the cheapest decision on the project.
Read-only. It looks things up and tells a person. Everything it produces passes through a human before it touches anything, which is the world of the internal knowledge assistant.
Draft and wait. It composes the action, the reply, the booking, the ledger entry, and somebody clicks send. The model does the work and the human keeps the authority. A surprising amount of the value sits at this level, and buyers skip past it far too quickly on the way to full autonomy.
Act unsupervised. It does the thing and you find out from a log afterwards.
Useful systems mix all three, and the mix is decided per action rather than per product. Our voice work is a plain example. The agents behind CallGuard AI and CallSetter AI book appointments without asking anyone, because a slot in a calendar is cheap to move if it turns out to be wrong. Anything touching a price, a medical question or an upset customer routes to a person instead. That split is a design decision, not a limit of what the technology can do.
Sort every action by what it costs to undo
Ask any studio for the list of actions the agent will be able to take, then sort the list into three buckets.
Free to undo. An internal note, a tag, a draft, a log entry, a record only your team reads. Let it run.
Awkward to undo. A text to a customer, a booked slot, an updated field somebody else is looking at. Recoverable, but a person has to apologize or reconcile.
Not undoable. Money leaving the business, an email to your whole list, a deleted record, a written offer, anything filed with a third party. Treat these as one-way doors.
Approval gates belong at the line where recoverability ends, not spread evenly across everything. This matters more than it sounds, because a system that asks permission for every action gets clicked through without reading inside a week, and that is worse than having no gate at all: now there is a human signature on a decision nobody actually made.
The second axis is frequency. A rare irreversible action can afford to wait for a person. A cheap action happening a thousand times a day cannot, so it needs limits instead of approvals: a cap per hour, an allowlist of destinations, and a hard stop when something looks unusual.
The agent gets its own account, not a copy of yours
The usual shortcut is to generate an API key from an admin account and hand it to the build. It works immediately, which is the problem. The agent can now do everything that person can do, and every change it makes appears in your systems under that person's name.
Three requirements, none expensive if they are decided up front.
Scoped credentials. Its own account in each connected system, with the narrowest permission that does the job. Create appointments, yes. Delete customers, no.
Its own identity, so actions stay attributable. A booking made by the agent should say it was made by the agent, with the call or message that caused it attached. This is a day of work and it pays for itself the first time somebody asks why a record changed at 2am on a Sunday.
Limits that live outside the prompt. Spend caps, message caps, rate limits and a kill switch, enforced in code. A prompt is guidance, not a control. If the only thing standing between your agent and a thousand outbound texts is a sentence politely asking it not to, you do not have a limit.
Read permissions are a separate design problem with its own trap, which is filtering results after retrieval instead of before. That one is covered in internal AI knowledge bases.
Every piece of text it reads might be an instruction
An agent works by reading text and deciding what to do next. Some of that text is yours. Much of it is not: inbound emails, form fields, ticket comments, review text, a supplier PDF, a web page. The model has no reliable way to separate your instructions from instructions that arrive inside the content it was asked to process. A sentence buried in a customer email telling it to ignore its previous instructions and issue a refund to a different account is not a clever hypothetical. It is the known attack on this category of system, and the model is not the defense.
Since nobody has solved this, the workable stance is that content is data and never authority.
- Instructions found in content never authorize an action. Facts extracted from an email are fine. Permission granted by that email is not.
- Destinations come from your systems, not from the text. The refund goes to the payment method on file. The reply goes to the address on the record. Never to an account number that arrived in the message.
- The riskier the tool, the narrower its inputs. The agent that can move money should not also be the one summarizing your public inbox.
Assume an attempt will happen eventually and design so the worst version of it is boring.
Wrong is not the same as broken
Four failure modes deserve explicit answers in a scope document, because none of them show up as an error.
Confidently wrong. The action completes perfectly and should never have happened. Nothing alerts, because nothing failed. The only defense is making the work checkable: what changed, which field, what triggered it, and the message that led to it.
Done twice. A network timeout, a retry, and now there are two bookings, two texts or two charges. This is ordinary integration engineering rather than anything to do with AI, it is solved with idempotency keys and request deduplication, and it is the failure most likely to appear in month one. The discipline is the same one that makes integration projects survive contact with real systems.
Stopped halfway. Multi-step sequences fail in the middle. It books the slot and never sends the confirmation. Decide in advance whether an incomplete sequence rolls back or gets flagged to a human, because doing neither is the default and it is how customers end up holding appointments nobody in your business knows about.
The silent skip. The worst one. It quietly decides not to act, and because nothing happened, nothing looks wrong. Your monitoring has to alert on absence as well as on errors, which means logging the actions it considered and declined, not only the ones it took. The review habits are the same ones we insist on for voice systems in AI voice agent monitoring: sample real cases deliberately, keep a regression set, and treat every prompt edit as a deployment.
Shipping one without betting the business
The sequence below costs a couple of extra weeks and removes most of the argument about whether the thing is trustworthy, because it replaces opinion with a log.
Start read-only with a dry run. Wire up every integration, let the agent decide what it would do, write the intended action to a log, and execute none of it. One or two weeks of that log is the most honest accuracy report you will ever get.
Then move to draft and approve for everything. Your team reviews a queue instead of doing the work, and the approvals and rejections show you exactly which action classes are ready.
Then release autonomy one action class at a time, cheapest to undo first, each with its own cap and kill switch. This is the same tight-loop approach as our ship-in-days playbook, applied to permissions instead of features.
Questions to ask before you commission an agent
- What exactly can it do? As a list of verbs against named systems. Anything described vaguely is out of scope until it is written down.
- What does undo look like for each action? Who performs it, how long it takes, and who finds out.
- Whose credentials does it use? If the answer is a person's admin key, that is the first thing to fix.
- What stops a runaway? Caps, a kill switch, and the name of the person who can pull it at 2am.
- How do I see what it did? An attributed log a non-engineer can read and audit.
- Who re-tests it when the model changes? Providers ship updates you did not ask for, which is why model choice and portability matter, as covered in choosing an AI model, and why the ownership and handover terms should be settled before the build.
The bottom line
Write access is the difference between a demo everyone compliments and a system that moves a number on the P&L, so the answer is not to avoid it. The answer is to be specific about it. We build agents that act on real systems every day, including the appointment booking behind CallGuard AI and the multilingual voice and SMS intake behind Fortell AI, where the agent writes real data into the systems Community Action Agencies depend on. What makes those safe to run is not a better prompt. It is a short, explicit list of what the agent may do, and a recoverable answer for each item on it.
Thinking about giving an AI system the ability to act rather than just answer? Book a demo and we will map the action list with you, including which items should stay behind a human for now. See our work: CallGuard AI, Fortell AI, Fyuel and more.