WritingAugust 2026
The Hard Part of Building an AI Agent Is Deciding What It Is Allowed to Do
Getting an LLM to call a function is increasingly table stakes. The difficult problem is deciding where the model's authority ends.
When people talk about AI agents, the conversation usually starts with what the model can do.
It can understand a request. It can call tools. It can browse the web. It can remember things. It can chain several actions together.
But while building the agent inside Xpend, I've become more interested in a different question:
What is the model actually allowed to do?
That question becomes much more important when the agent has access to something with real consequences.
If an assistant gets a question wrong, you can usually ask again. If an agent gets a ₦50,000 transfer wrong, the problem is very different.
This changes how I think about the boundary between an assistant and an agent.
Modern models can produce a structured tool call reasonably well. What matters more is what happens after the tool call.
The model shouldn't be the transaction system
In Xpend, a user can type something like “send ₦20,000 to John.”
The model interprets that request and can select a structured preview_transfer tool. It has to turn the conversational request into actual parameters that the application understands.
But that tool call does not move ₦20,000.
The tool produces a domain result. The application validates the recipient, checks the relevant rules, builds the preview and presents the next step to the user.
The actual transaction happens somewhere else.
A PIN confirmation eventually reaches the transaction execution layer, which is responsible for moving the money. The model doesn't get access to a confirmWithPin tool. It can't simply decide that because the user said “send it,” the transfer should happen.
That separation is intentional.
The model is good at interpreting messy human language. It is not the place where we want the final financial authority to live.
An agent is more than chat with tools
This is where the assistant-versus-agent distinction gets useful.
A chatbot can answer: “What's my balance?”
An agent can take an instruction, gather the information required to act, call the appropriate tools and move through a task lifecycle.
In our implementation, that can mean collecting missing information across multiple turns, resolving a recipient, creating a transfer preview, waiting for authorization and eventually handing the transaction to deterministic execution code.
The conversation and the transaction are related, but they aren't the same thing. The chat message is temporary. The transaction has its own state.
Real actions don't always finish inside the same request that started the conversation. A transfer can be waiting for a PIN. A deposit can be waiting for a blockchain confirmation. A scheduled payment can execute later. An external food order can have its own lifecycle.
Once you start building around those realities, “agent” stops meaning “LLM with function calling.” It starts meaning a system that can move through real tasks while respecting the boundaries of the application.
Structured intent is not authorization
One of the simplest mistakes would be treating the model's structured output as permission.
Suppose the model produces recipient = John and amount = 50000. That tells us what the model thinks the user wants. It does not tell us that ₦50,000 should leave the account.
Those are two different decisions. The first is interpretation. The second is authorization.
We enforce that distinction in the architecture.
Structured schemas validate the arguments coming from the model. Domain services validate the actual operation. Recipient resolution doesn't simply take the first matching person. Financial previews are rendered directly rather than allowing the model to rewrite them into a statement that money has already moved.
And for the normal main-wallet transfer path, the transaction still needs a separate authorization step.
The model can ask for a transfer. It cannot authorize the transfer.
That difference sounds small, but it changes the architecture considerably.
Then we had to deal with autonomy
There is an interesting exception.
Xpend also has an agent-wallet path where a transfer can happen without asking for a PIN every time.
At first glance, that seems to contradict the principle above. It doesn't.
The important difference is where the authority comes from.
The model doesn't receive unrestricted permission to spend money. The transfer has to pass through an autopay policy and risk evaluation, and the client presents a countdown before execution. If the policy doesn't allow the action or the risk checks require stronger authorization, the flow falls back to the normal PIN path.
So the system can be autonomous without making the LLM the authority.
Autonomy isn't the same as unrestricted access. You can give a system permission to act within a defined boundary without giving the model unlimited authority to define that boundary itself.
The failure cases are part of the agent
Another thing that changed my perspective was realizing how much of the agent is defined by what happens when things go wrong.
What happens if the recipient name is ambiguous? The system shouldn't guess.
What happens if the user hasn't provided enough information? The agent needs to collect it rather than invent it.
What happens if a crypto deposit needs a network? The model is explicitly prevented from making one up.
What happens if a transfer requires authorization? The conversation moves into a transaction state instead of pretending the action succeeded.
What happens if a provider fails? The transaction system reports a failure rather than letting the language model narrate a successful payment.
These aren't secondary implementation details. They're part of what makes an agent usable in the real world.
An agent that can act but cannot reliably stop is not particularly useful for financial software.
The model is one component of the system
This has also made me less interested in architectures where the LLM is treated as the center of everything.
In a financial agent, I want the model to be responsible for the things models are good at: understanding what the user means, choosing an appropriate capability, extracting structured information, asking for missing information, and deciding which tool might help next.
I want deterministic systems to be responsible for other things: whether the arguments are valid, whether the recipient exists, whether the transaction is allowed, whether the user has authorized it, whether the balance is sufficient, whether a provider can execute it, and whether the transaction actually succeeded.
Those responsibilities don't need to belong to the same component. In fact, I think they are safer when they don't.
The agent needs a control plane
The result is that the conversational layer starts looking less like the financial system and more like a control plane for it.
The user speaks naturally. The model translates that language into structured intent. The application decides what that intent means in terms of actual domain operations. Policies determine what the agent is allowed to do. Authorization determines when an operation can cross the boundary into execution. Then deterministic services perform the actual side effect.
The LLM is involved in the decision-making process, but it doesn't become the source of truth for the transaction.
That separation gives us somewhere to put rules that shouldn't depend on whether the model happens to phrase something correctly.
This changes what “smarter” means
There is a temptation to think that making an agent better means making the model better at completing more tasks. That is part of it.
But in a system that can touch money, a better agent also needs better boundaries.
It should know when it has enough information. It should know when it needs the user. It should know when an action requires authorization. It should be able to operate within a policy without being able to redefine the policy.
And when something goes wrong, the application should be able to tell the user what actually happened without relying on the model to reconstruct the truth.
That's a different kind of intelligence. It is less about giving the model more power and more about giving the whole system a reliable way to use the power the model already has.
The hard part is deciding what the agent is allowed to do
The more I build agentic systems, the less I think the difficult problem is getting an LLM to call a function.
That's increasingly table stakes.
The difficult problem is deciding where the model's authority ends.
A useful agent needs enough authority to accomplish something on the user's behalf. A trustworthy system needs enough control to make sure that authority doesn't become arbitrary.
Those two requirements have to coexist.
At Xpend, we're still figuring out the right boundaries. But the architecture has made one thing clear to me:
The model doesn't need to be the system that moves the money to be the system that makes the product feel agentic.
The agent can understand the request, coordinate the work and move a task toward completion. The financial system can remain the authority over what is actually allowed to happen.
That boundary is probably one of the most important parts of building AI agents that can do things in the real world.