Enterprise Adoption2 min read

Why 80% Accurate AI Works for Email but Fails at Month-End

By , Senior AI ConsultantPublished

An AI that is right 80% of the time is fine for drafting email, but chained across five finance steps it gets the whole job right only about one time in three.

SAP, and Infosys, the consulting firm that builds SAP systems for other companies, are pushing customers to put AI agents inside their main business systems, working from one set of connected company data, instead of running scattered pilots. Behind the sales pitch is a point about accuracy that applies to any team thinking about where to use AI first.

An email draft and a ledger are different jobs

Most people's first use of AI is writing or summarizing: an email, a meeting recap, a first draft of a slide. An answer that is right four times in five works well there, because you are the checker. You read it, fix the one wrong line and send it. The AI saves you the blank page and you keep the last word.

Finance and supply chain work is built differently. The result of one step goes straight into the next, and the people who would catch a mistake are looking at the final report, not the middle. An error that nobody sees early is still there when the books are signed.

Errors multiply along a chain

This is arithmetic, and it is harsher than most people expect. Take a close with five steps, each handled by an agent. If every step is right 80% of the time, the chance that all five are right is 0.8 multiplied by itself five times, which is about 33%. At 95% a step it is about 77%. At 99% a step it is about 95%.

So a number that sounds excellent for one task sounds much worse for a process. Ten steps at 95% a step is right only about 60% of the time. The more steps you hand over without a person looking, the closer each step has to be to perfect, and that is a much higher bar than the one a drafting tool has to clear.

What this changes for your own work

For example, a mid-sized distributor could let an agent handle supplier invoices in four steps: read the invoice, match it to the purchase order, choose the account code and schedule the payment. At 95% a step, about 81% of invoices come through right, so with 500 invoices a month, around 90 carry some error, and some of them are payments that have already gone out.

Now put a person at one point only: after the matching step, where the agent shows a short list of invoices whose price or quantity does not match the order. The accountant reviews a few dozen lines instead of 500 invoices, and the later steps start from checked input. The agent still does most of the work, and the person's attention goes where a mistake would cost money.

This is also why the vendors keep talking about data. An agent that looks up the real contract price in the company's own records makes fewer wrong matches than one that guesses from patterns. Higher accuracy at each step is what makes a long chain workable at all. They also stress control: a registry of every agent and what data each one can reach, because a chain you cannot list is a chain you cannot check. The big ERP vendors are already adding agents to their systems, so these questions will reach ordinary finance teams soon.

When anyone offers you an agent for a multi-step job, the accuracy figure for a single step tells you little. Ask how many steps the work has, what each one hands to the next, and where a person looks at the result before it moves on. Where nobody looks, the agent needs to be far better than 80%.

Share this

STAY INFORMED

Get AI intelligence like this delivered to your inbox.

Free forever · Unsubscribe anytime


You May Also Find Valuable