Stories

AI analysis needs a company's own definitions more than it needs a smarter model

Goldman Sachs limits Claude Code to its engineers, and Grok Bot users ran out of their weekly allowance days early.

By , Senior AI ConsultantEdition of

6stories
4minute read
In This Edition

These agents are good at describing what happened in the data. Recommending what to do is where they fall short: fewer than half of their findings lead to an action that improves the business. The most common reason is missing business context. The agent can read every table, but it does not know what the company means by its own words.

A better model cannot close that gap, because what your company means by an on-time delivery or an active customer is not in any training data. It is a decision the company made, and it lives in the head of whoever builds the weekly report. A stronger model simply picks the textbook meaning and defends it more convincingly.

For example, a logistics firm asks an agent which depots to fix first for late deliveries. The agent counts a delivery as on time if it arrived on the promised day. The firm's biggest customer counts it as on time only if it arrived inside a two-hour slot. So the agent ranks the wrong depots first, with a clear chart and a sensible reason.

The fix is one page of definitions: twenty terms, each with its formula, the system field it comes from and the cases it leaves out. Once that page is in the agent's standing instructions, every later question gets the company's answer instead of the textbook one. The person who owns the weekly report can write it in an afternoon, before anyone pays for a bigger model.

Share this story

The tool forecasts demand and tells the crew when to grill chicken or simmer beans. Chipotle plans to use it across the United States early next year.

A kitchen is a good first job for an AI forecast because a wrong number costs little and shows within the hour: a cook sees the chicken running low and grills more before anyone waits. AI forecasting will pay first in jobs that work this way, such as a bakery's morning batch, and later in the monthly purchase order, where a wrong guess stays on the shelf for weeks.

Share this story

Grok Bot, the AI agent from SpaceX, carries out tasks for paying subscribers. When one user asked for a reset of the allowance, Elon Musk replied that serving the agent cost too much relative to revenue.

A flat monthly price works while a person's day limits use, because nobody types questions all night. An agent does work all night, so one heavy user can cost as much as fifty light ones. Unlimited phone data plans ended for the same reason and became allowances, with extra charges after them.

ChatGPT is moving the same way. From December, its chat with the advanced models will share one allowance with Codex, OpenAI's coding agent. A long coding task on Monday can then leave a user waiting for a reset before asking an ordinary question on Friday.

Share this story

A person at a checkout knows that a missing delivery slot means calling the branch. An agent knows only what the page states, and it stops there. Some shops block agents without meaning to: dozens of retailers do it by accident, through old rules written to keep data scrapers out of checkout pages.

Give a browsing agent, such as the one in ChatGPT, a real order to place in your own shop, and see where it stops. Then look for the order in the order list, because web analytics often count an agent's visit as a bot.

Share this story

Traders and bankers had used them for months to build small tools of their own, such as ones that watch client trading.

When spreadsheets that traders built caused costly errors, banks did not take Excel away. Instead, any spreadsheet that money or a report depended on got a named owner and a second person to check it.

The same split fits AI-built tools. A tool that only its builder uses can stay free. A tool that others rely on gets the second reader, and most bans on staff building with AI will end up as that rule.

Share this story

A reviewer who sees the AI's verdict on an insurance claim before forming a view reads the case looking for reasons to agree. Even expert pathologists dropped correct judgments for wrong AI advice 7% of the time, although the AI helped them on average.

The fix takes twenty seconds: before opening the verdict, the reviewer writes one sentence saying what they think and why. Those sentences also show a manager how often staff disagree with the AI. If they never do, nobody can tell whether the review adds anything.

Share this story

THE DAILY BRIEF

Get the next edition in your inbox.

A five-minute read, every weekday morning.

Free forever · Unsubscribe anytime

Share this

Other Editions

Newest first


THE DAILY BRIEF

Read the next one first.

A five-minute read, every weekday morning.

Free forever · Unsubscribe anytime