Stories

Managers check AI less carefully when it is called a coworker, and Shopify lets AI agents pay

Only 23 companies spread an AI gain from their top factories to three or more plants, and Gartner expects over 40% of agent projects to be canceled by 2027.

By , Senior AI ConsultantEdition of

5stories
3minute read
In This Edition

Managers catch 18% fewer errors in AI work when they are told it comes from an "AI employee" instead of an AI tool. That is the main result of a randomized study of 1,261 managers led by Emma Wiles, a business professor at Boston University.

Every group saw the same documents, with the same mistakes planted in them. One budget memo claimed a new contract would cut costs while the spreadsheet attached to it showed expenses rising. A job description for an entry-level post asked for more than ten years of experience. Only the label changed from group to group.

Under the employee label, managers also took less personal responsibility for the output, and they were 44% more likely to send questionable work to their own manager for another review instead of correcting it themselves. Managers checking the same work from a human employee flagged more errors on their own. So the weak checking is specific to AI with a job title: managers give it a colleague's trust without the supervision they give a person on their team.

About one in five organizations now list AI agents on their org charts. One participant's company has an agent called Kevin on its chart, and when something goes wrong, the team says "Kevin made a mistake" and blames software that cannot be held responsible for anything.

The human label did not make managers any more willing to use AI. Those most ready to adopt it were the ones whose own bosses encouraged the tools and used them openly.

Share this story

Capital One wants AI agents that take real actions for customers, such as booking a test drive or handling a request. Evident's global ranking puts Capital One second among banks for AI maturity. The bank says these agents are safe only if strict data and security rules are in place before they go live.

Separate research from MIT and S&P Global finds that most AI projects fail because nobody is assigned to steer the AI and check its work.

Having no owner is more expensive with an agent than with a chatbot. A chatbot's wrong answer is a draft someone can delete. An agent's wrong decision is a completed act: the refund paid, the appointment booked, the email sent in the company's name.

Steering an agent comes down to three questions, and each one needs a named person to answer it: what data the agent may read, which actions it may take without a human approving them, and who reviews what it did afterwards. Gartner predicts that more than 40% of company AI agent projects will be canceled by 2027.

Share this story

Shopify stores now accept orders that an AI agent completes and pays for. The agent runs inside the shopper's own web browser, so a person can tell a browser assistant what to buy and let it finish the purchase on a Shopify merchant's site.

To the store, the order looks like any other sale. A dispute begins if the agent buys the wrong item, or if the shopper later says they never approved the charge. In online card payments the merchant usually pays for a disputed order: the refund plus a chargeback fee. And the merchant sees only the order, never the instructions the shopper gave the agent.

Amazon has chosen the other way. It is fighting a similar agent, Perplexity's Comet browser, in court to keep it out of its own checkout.

Share this story

A single hacker using AI can now break into more than a dozen organizations in weeks. The strongest AI defense tools, meanwhile, are restricted to big companies.

That leaves small hospitals, nonprofits and shops open to the new attacks without access to the best new protection. When one of them is broken into, it pays for the damage itself.

Share this story

When AI improves quality at one plant, how often does the same company get that gain at its other plants? The World Economic Forum looked at 223 of the world's most advanced factories, all top performers with AI improvements to show.

A system that predicts which parts will fail inspection learns from one production line: its sensor readings, its maintenance history, the way that plant's inspectors record a fault. A sister plant may have older machines and its own codes for faults. There the same system meets records unlike anything it learned from, so engineers at that plant organize the data again from the beginning.

Among the companies behind those 223 factories, only 23 had spread an AI improvement to three or more of their plants.

Share this story

THE DAILY BRIEF

Get the next edition in your inbox.

A five-minute read, every weekday morning.

Free forever · Unsubscribe anytime

Share this