Stories

AI numbers need agreed definitions, and AI courses need someone to redesign the work

Crisis rehearsals work best when a real supplier disappears, and HackerRank's AI interviewer asks developers how they checked the AI.

By , Senior AI ConsultantEdition of

5stories
3minute read
In This Edition

Ask a company's AI assistant for last month's revenue, and it first has to decide what the word means. Does it include sales tax and delivery charges? Are returned orders taken off? The data does not answer these questions. Even a small online shop has four different revenue figures in four of its systems.

So the sales director gets one answer, the finance director's monthly report shows another, and the person who knows the real number decides the tool is wrong. After two or three of those moments, they go back to the spreadsheet.

The AI did not create the disagreement. Each department always had its own version of revenue, kept in its own spreadsheet, where nobody compared them. An assistant that answers everyone from the same data puts those versions side by side.

The fix is about thirty years old: write down once what each disputed word means, and make every tool use that definition. AI can now draft the definitions from existing reports, so the real work is the meeting where people agree.

For example, a finance director could list the five numbers that cause the most arguments, such as revenue, gross margin and on-time delivery, and name one owner for each. The agreed definitions go into the instructions that the assistant reads. Then someone asks the assistant each question and compares its answers with last month's board report. A week of short meetings buys a tool that people believe.

Share this story

The day that hurts most is usually the day one ordinary dependency disappears: the only supplier of a part, or the one person who knows the bank login.

So pick a real one, decide that it is gone from Monday at 9am, and take the team through the first two days. It costs a few hours of staff time.

AI helps only when it is given structure. Asked what could go wrong, it repeats the usual list. Given the real supplier list and asked to play out each hour, it raises the questions that nobody planned for.

Share this story

Schneider Electric, the French energy technology group, has made basic AI training compulsory for all 160,000 staff. It also trains a group that is harder to identify: the people who own a process and decide how it runs.

That second group is where the gains come from. When factories first replaced steam engines with electric motors, they kept the old layout, built around one long turning shaft, and output barely moved. It rose when factories were rebuilt around the order in which the work was done.

After a course, a clerk has a new tool and still handles each invoice in the same seven steps. So choose one task, give it to the two people who do it best, and let them design it again from the beginning.

Share this story

A manager notices that a forecast looks wrong because she built forecasts like it for years and remembers where they usually break. When AI drafts that work, the people who will check it later never build that memory.

Pilots met this problem first. Autopilot made flying safer and gave them less practice, so the US aviation regulator recommends that they fly manually more often.

An office can do the same. On one task a week, each person writes their own answer before opening the AI's. The differences show what the AI missed, and what the person would have missed.

Share this story

HackerRank already let candidates use an AI assistant in live interviews while a human interviewer watched. Chakra, its AI interviewer, now runs the interview itself. The candidate works on real code with AI available, and Chakra asks why they chose their approach and how they checked what the AI produced.

When every candidate has the same assistant, the finished answer says little about the person. A manager hiring a buyer could hand over three supplier quotes, allow ChatGPT, and spend the interview on two questions: which of its figures did you check, and where did you overrule it?

Share this story

THE DAILY BRIEF

Get the next edition in your inbox.

A five-minute read, every weekday morning.

Free forever · Unsubscribe anytime

Share this

Other Editions

Newest first


THE DAILY BRIEF

Read the next one first.

A five-minute read, every weekday morning.

Free forever · Unsubscribe anytime