A customer asks:
“How much can I safely spend today?”
An LLM can understand the question.
It can explain the customer’s account.
It can summarize recent transactions.
It can even reason about what information might be relevant.
But there is a boundary that financial systems cannot afford to blur:
The LLM can interpret financial information. It should not become the authority for financial truth.
That distinction sounds obvious.
In practice, it becomes complicated very quickly.
Because modern AI systems can reason, plan, call services, summarize information, and make decisions that previously required explicit application logic.
The temptation is therefore to put the LLM in the middle of the financial system and let it “figure things out.”
That is where the architectural boundary matters.
The LLM Boundary
A useful mental model is:
User Intent
|
v
LLM
|
| Interpretation
| Reasoning
| Recommendation
v
Business Rules
|
| Validation
| Authorization
v
Financial Systems
|
v
Authoritative State
The LLM can operate on the left side of the boundary.
Financial systems must remain authoritative on the right side.
The LLM may say:
“The customer appears eligible for this payment.”
The financial system must determine:
“The customer is authorized to make this payment under these rules.”
Those statements are not equivalent.
One is probabilistic reasoning.
The other is a business decision with financial consequences.
That difference is the boundary.
Understanding Is Not Authorization
This is one of the easiest mistakes to make with AI.
Suppose a user says:
“Transfer $50,000 to the supplier we normally use.”
The LLM may correctly understand:
- which supplier the user means
- which account is being referenced
- what amount is intended
- what operation the user is asking for
But understanding the request does not authorize it.
The system still needs to establish:
Who is the user?
|
v
What are they allowed to do?
|
v
Which account can they use?
|
v
Is this transaction permitted?
|
v
Are additional controls required?
An LLM can assist with interpretation.
It should not replace the authorization boundary.
Authorization cannot be probabilistic.
Probabilistic Reasoning Meets Deterministic Money
LLMs are extremely good at dealing with ambiguity.
Financial systems often have the opposite requirement.
Consider:
“Pay the latest approved invoice.”
An LLM might identify the invoice that appears to be the latest approved invoice.
But a financial system needs to establish precisely:
- which invoice
- which version
- which legal entity
- which currency
- which amount
- whether it is still approved
- whether it has already been paid
- whether the user is authorized
- whether the payment is permitted
The difference is subtle but fundamental.
The LLM can help answer:
“What does the user appear to mean?”
The financial system must answer:
“What is actually true?”
Those are different questions.
Why the LLM Should Never Be the Source of Truth
This becomes even more important when dealing with balances.
Imagine an LLM is given:
Account Balance: $10,000
It reasons about the customer’s spending.
A few seconds later, another transaction posts.
The actual balance is now:
$7,500
The model does not own the balance.
It should not.
The ledger or authoritative financial system owns it.
The LLM can consume that state.
It can explain it.
It can reason about it.
It can answer questions about it.
But it should never become a second ledger merely because it has been given financial data.
This principle extends far beyond balances.
The authoritative source should remain outside the model for:
- account balances
- transaction status
- payment status
- settlement status
- available funds
- customer identity
- authorization state
- limits
- accounting entries
- regulatory status
The LLM can interpret those facts.
It should not redefine them.
The Difference Between Recommendation and Commitment
This is perhaps the most important distinction in an AI-enabled financial architecture.
Consider:
LLM
"Based on the available information,
I recommend approving this payment."
That is a recommendation.
Now compare it with:
Financial System
Payment P-98231
Status: COMPLETED
Amount: $50,000
That is a commitment.
The first can be wrong.
The second must represent what actually happened.
The architecture should therefore preserve the boundary:
Recommendation
|
v
Validation
|
v
Authorization
|
v
Execution
|
v
Financial Commitment
The LLM can participate in the first stage.
It should not silently collapse all five stages into one.
What Should an LLM Actually Do?
There is a lot that an LLM can safely contribute to financial systems.
It can help interpret human intent.
It can summarize complex information.
It can explain why a transaction failed.
It can identify missing information.
It can classify incoming requests.
It can help an operator investigate an exception.
It can suggest possible next steps.
It can compare information across multiple sources.
It can explain financial processes in language a customer understands.
It can help navigate a large operational environment.
These are valuable capabilities.
None require the LLM to become the financial authority.
A useful distinction is:
LLM
|
+-- Understand
|
+-- Explain
|
+-- Summarize
|
+-- Recommend
|
+-- Assist
|
v
Deterministic Boundary
|
+-- Validate
|
+-- Authorize
|
+-- Commit
That boundary is where the architecture becomes safe.
Where the LLM Must Stop
There are certain operations where I would want the LLM to stop at recommendation.
For example:
“Should this payment be made?”
The LLM can provide context.
But the financial system should determine whether the payment is permitted.
Similarly:
“Should this customer’s credit limit be increased?”
The LLM can summarize relevant information.
But the actual credit decision should remain governed by the appropriate business and risk controls.
And:
“Should we reverse this transaction?”
The LLM can explain the circumstances.
But reversal must follow deterministic business rules and authorization.
The common pattern is:
LLM proposes
|
v
Policy evaluates
|
v
Authorized operation
|
v
Financial system commits
The LLM does not disappear.
It simply stops at the right boundary.
The Dangerous Shortcut: “The Model Knows Enough”
One of the most dangerous assumptions is:
“The model has all the information, so it can make the decision.”
Having information is not the same as having authority.
A model may have:
- transaction history
- customer information
- account information
- business rules
- previous decisions
- external context
And still not know everything that matters.
There may be:
- a transaction happening concurrently
- a state change that occurred seconds ago
- an authorization that has expired
- a policy exception
- a regulatory restriction
- a pending transaction
- an operational incident
- a reconciliation issue
The model’s context is a representation of the world.
It is not the world itself.
That distinction is fundamental to distributed financial systems.
State Changes Faster Than Reasoning
This is a familiar problem in distributed systems.
A system reads some state.
It makes a decision.
The underlying state changes before the decision is committed.
AI does not eliminate this problem.
It can make the gap more visible.
Consider:
10:00:00
LLM reads:
Available Balance = $100,000
10:00:01
Another transaction consumes:
$40,000
10:00:02
LLM recommends:
Approve $80,000 payment
The reasoning may have been perfectly consistent with the information available at 10:00:00.
It is still no longer safe to execute the payment.
The financial boundary must validate the current state again.
This is an important architectural principle:
A recommendation based on yesterday’s state cannot automatically become today’s financial commitment.
The LLM Should Not Maintain Financial State
There is another temptation:
“The agent already knows what happened.”
It doesn’t.
The agent may have a conversational history.
That is not authoritative business state.
Consider:
Agent Memory
"I believe payment P-123 was completed."
versus:
Payment System
P-123
Status = COMPLETED
Only the second should determine what happens next.
The agent can forget.
The process can restart.
The model can produce a different answer.
A service can fail.
A conversation can disappear.
None of those events should change the underlying financial truth.
This is exactly why financial state needs an authoritative owner.
Financial Systems Have Always Needed This Boundary
This isn’t really an AI invention.
Financial systems have always separated:
Request
|
v
Validation
|
v
Authorization
|
v
Transaction
|
v
Ledger
AI introduces a powerful new reasoning component before these boundaries.
The mistake is assuming that because the reasoning component is more capable, the boundaries are no longer necessary.
They become more important.
The more flexible the reasoning layer becomes, the more deterministic the commitment boundary needs to be.
The LLM Should Not Become the Ledger
This deserves explicit emphasis.
A ledger answers questions such as:
What was posted?
When was it posted?
Which account was affected?
What was the amount?
What is the resulting balance?
What is the transaction status?
An LLM answers a very different class of questions:
What does this information mean?
What might the user be asking?
What could explain this situation?
What actions might be appropriate?
Those capabilities complement each other.
They should not be confused.
A useful architectural separation is:
LLM
"Explain the account."
Ledger
"Define the account state."
The LLM explains.
The financial system defines.
When the Model Is Wrong
Eventually, the model will be wrong.
That should not be a controversial statement.
The architectural question is:
What happens when it is wrong?
Suppose an LLM incorrectly interprets:
“Pay the invoice from last month.”
It identifies the wrong invoice.
If the model has direct authority to execute the payment, the error becomes a financial event.
If the model only proposes:
Candidate Invoice: INV-4821
Amount: $42,000
the deterministic financial boundary can still validate:
Is this invoice valid?
Is it approved?
Is it unpaid?
Is this user authorized?
Is this amount correct?
Is payment permitted?
The model can therefore be wrong without necessarily making the financial system wrong.
That is a very powerful architectural property.
Containment Is More Important Than Perfection
A financial architecture should not assume:
“The model will always make the right decision.”
A better assumption is:
The model will sometimes be wrong, and the architecture must contain the consequences.
That changes how we design the system.
Instead of asking:
“How do we make the model perfect?”
we should also ask:
“What happens when the model is wrong?”
That leads to:
- validation boundaries
- authorization boundaries
- limits
- approval workflows
- deterministic business rules
- audit trails
- reversible operations where possible
- reconciliation
- exception handling
These controls are not evidence that AI failed.
They are evidence that the architecture understands the nature of AI.
Auditability Becomes More Important
Financial systems already require strong auditability.
AI adds another layer.
Suppose an operator asks:
“Why was this payment initiated?”
It is not enough to say:
“The AI decided to do it.”
We need to understand the chain:
User Request
|
v
Interpretation
|
v
Recommendation
|
v
Business Rules
|
v
Authorization
|
v
Execution
|
v
Financial Result
The system should be able to reconstruct what happened.
Not necessarily every internal detail of model reasoning.
What matters operationally is the business decision trail:
- what was requested
- what information was used
- what action was proposed
- what controls were applied
- what was authorized
- what was executed
- what financial state changed
This distinction is important.
Explainability is not the same as financial accountability.
A beautifully explained model decision is still not an authoritative transaction record.
Human Approval Is Not a Failure of Automation
There is sometimes an assumption that a truly intelligent system should be able to make every decision automatically.
I don’t agree.
In financial systems, there are situations where the correct architecture is:
LLM
|
v
Recommendation
|
v
Policy
|
v
Exception
|
v
Human Approval
|
v
Execution
For example:
- unusually large transactions
- unusual account behavior
- conflicting business information
- ambiguous ownership
- policy exceptions
- unresolved reconciliation
- high-risk financial operations
The goal of AI should not be:
Remove humans from every decision.
The goal can instead be:
Give humans better information so they make fewer and better decisions.
That is a much more realistic model for financial infrastructure.
What About Autonomous Agents?
This becomes even more important when agents are allowed to operate for longer periods.
Imagine:
Goal
|
v
Agent
|
+--> Observe
|
+--> Decide
|
+--> Act
|
+--> Observe
|
+--> Decide
|
+--> Act
|
+--> ...
The longer the loop continues, the more important the boundary becomes.
At every consequential action, the system should know:
Is this action permitted?
Is the underlying state still valid?
Has the business condition changed?
Is this operation reversible?
Does it require additional authorization?
Autonomy should not mean unrestricted authority.
A useful definition of safe autonomy is:
The ability to operate independently within clearly defined business boundaries.
The Principle of Least Authority Applies to AI Too
Traditional security architecture has long understood the principle of least privilege.
An AI agent should follow the same principle.
If the agent needs to:
- read invoice information
it does not automatically need authority to:
- modify invoices
If it needs to:
- recommend a payment
it does not automatically need authority to:
- execute the payment
If it needs to:
- investigate an exception
it does not automatically need authority to:
- resolve the exception
The architecture should expose only the capabilities required for the task.
The agent should operate within a constrained business surface.
This is not about distrusting AI.
It is about designing systems so that no single component has more authority than necessary.
The Boundary Should Be Explicit
One thing I would avoid is an architecture where the boundary exists only as an informal convention.
For example:
“The agent normally doesn’t execute payments directly.”
“Normally” is not a boundary.
The boundary should be enforced by the architecture.
Something closer to:
AI Layer
|
Recommendation
|
X
|
No Direct Commitment
|
v
Business Boundary
|
+-------+-------+
| |
Validation Authorization
| |
+-------+-------+
|
v
Execution
|
v
Financial State
The system should make the unsafe path difficult or impossible.
The Real Source of Truth
Every important financial concept should have an authoritative owner.
For example:
Customer Identity
|
v
Identity System
Account Balance
|
v
Ledger
Payment Status
|
v
Payment System
Settlement Status
|
v
Settlement System
Authorization
|
v
Authorization Boundary
The LLM can bring these pieces together.
It can create a useful understanding of them.
But it should not become the owner of any of them simply because it can reason across them.
This is particularly important when different systems disagree temporarily.
The answer should not be:
“Ask the model which one looks correct.”
The answer should be:
Know which system is authoritative for that piece of state.
AI Should Interpret the World, Not Define It
This is the architectural principle I keep coming back to.
An LLM can create an interpretation:
“This appears to be a duplicate payment.”
The financial system can determine:
“Payment P-78219 is a duplicate according to the defined business rule.”
The first is useful.
The second is authoritative.
Similarly:
“This customer appears to have sufficient funds.”
versus:
“Available balance at authorization time is $82,000.”
Or:
“This transaction looks suspicious.”
versus:
“This transaction has been placed into the required review state.”
The distinction between interpretation and state is critical.
A Boundary I Would Trust
For a consequential financial workflow, I would want something conceptually like:
User
|
v
AI / LLM
|
Understand Intent
|
Propose Action
|
v
+-------------------------+
| Deterministic Boundary |
| |
| Validation |
| Authorization |
| Business Rules |
| Risk Controls |
+------------+------------+
|
v
Financial Operation
|
v
Authoritative State
|
v
Audit / Events
|
v
Reconciliation
The AI layer is powerful.
But it is not sovereign.
The deterministic boundary decides whether the proposed action is actually allowed.
The financial system records what actually happened.
Reconciliation handles what cannot be immediately established.
And auditability allows the organization to reconstruct the journey.
The Most Important Boundary Is Not Technical
It is tempting to describe this as an AI architecture problem.
I think the deeper issue is a responsibility boundary.
Who is responsible for:
understanding intent?
The AI can help.
Who is responsible for:
deciding whether an action is allowed?
The business rules and authorization system.
Who is responsible for:
recording the financial truth?
The authoritative financial system.
Who is responsible for:
resolving uncertainty?
The workflow and reconciliation process.
Who is responsible for:
exceptional decisions?
Potentially a human, depending on the business rule.
Once these responsibilities are explicit, the architecture becomes much easier to reason about.
The LLM Does Not Need to Be the Smartest Component
There is another interesting consequence.
A financial architecture does not become better simply because the LLM is given more authority.
In fact, the opposite may be true.
The strongest architecture may be one where:
LLM
|
| high flexibility
|
v
Business Boundary
|
| high determinism
|
v
Financial System
|
| high authority
|
v
Financial Truth
The LLM can be highly capable while still being deliberately constrained.
That is not a contradiction.
It is good architecture.
Where AI Adds the Most Value
I don’t think the answer is to keep AI away from financial systems.
Quite the opposite.
There are enormous opportunities.
AI can sit at the human-system boundary and make complex financial infrastructure easier to interact with.
It can help operations teams understand incidents.
It can help customers understand financial activity.
It can help engineers investigate distributed workflows.
It can help identify patterns across large volumes of information.
It can help operators navigate exceptions.
It can help translate business intent into structured actions.
It can help explain why something happened.
These are powerful capabilities.
The mistake is crossing the final boundary:
turning interpretation into authority.
The LLM Boundary Is a Business Boundary
The most important question is therefore not:
“Which model should we use?”
Nor:
“How intelligent is the agent?”
Nor even:
“How autonomous should the system be?”
The more important question is:
Where does probabilistic reasoning end and deterministic financial authority begin?
That boundary should be explicit.
It should be enforced.
It should be observable.
And it should be auditable.
Final Takeaway
Financial systems have always dealt with uncertainty.
They have always dealt with distributed state.
They have always dealt with partial failures, authorization, reconciliation, retries, and business exceptions.
AI does not remove those problems.
It introduces a new component capable of interpreting intent and making remarkably useful recommendations.
That capability is valuable.
But it also creates a temptation to give the model more authority than it should have.
An LLM can misunderstand a request.
It can reason from stale information.
It can make an incorrect recommendation.
It can interpret an ambiguous business situation incorrectly.
The architecture should assume that possibility.
The answer is not to demand that the model become perfect.
The answer is to establish a clear boundary around what the model is allowed to influence.
The LLM can:
understand.
interpret.
summarize.
reason.
recommend.
assist.
But when the system reaches:
authorization,
financial commitment,
accounting state,
transaction completion,
and financial truth,
the responsibility must move to deterministic, authoritative systems.
The most important architectural principle is therefore simple:
Let AI reason about the financial system. Do not let AI become the financial system.
The LLM can explain what should happen.
The business rules determine what is allowed.
The financial systems determine what actually happened.
And the ledger remains the ledger.
That is the boundary I would trust in a financial system.