When a new digital tool underdelivers, the usual verdict is that people did not adopt it, so the company runs more training. That verdict skips a question: was the information in the tool bad, or was the tool hard to use? Those are different problems, and AI is about to make the mix-up more expensive.
Better information and easier screens are different jobs
Good information makes people think harder, and that is what you want. When the data in front of someone is complete and relevant, they spend their effort on the decision itself. A screen that is hard to use does the opposite. It spends their effort on clicks, scrolling, and alerts that need no action, and none of that goes into the decision. The research behind this, done on how doctors cope with hospital records software, treats these as two separate problems that need separate diagnosis and separate money.
The mix-up costs money in both directions. A company can pay for much better data and put it behind a clumsy screen, and the total effort on each person goes up even though the information improved. Or it can redesign the screen and leave the data patchy, and the tool becomes pleasant to use and no better at helping anyone decide. In both cases people complain, the company blames training, and the real cause stays where it was.
What this looks like with an AI tool
The research was done on doctors and hospital software, not on AI. But the logic carries over, because an AI tool is a data source with a screen on the front. Take a shift manager at a warehouse whose new AI system predicts which orders will ship late. The predictions are good. The system flags 80 orders a shift, and only 6 of them need the manager to do anything. The other 74 take about 20 seconds each to look at and dismiss, which is roughly 25 minutes a shift and about two hours a week. Within a month the manager mutes it, and the six orders that mattered go unseen along with the rest.
The same thing happens when a recommendation arrives with no reason attached. A recruiter whose tool ranks candidates has to open three screens to find out why one person scored higher than another. Checking costs more than ignoring the ranking, so she does her own sort, and the model's quality never reaches a decision.
Hospitals show where this ends. The allergy warnings were meant to catch dangerous prescriptions. Over a decade, doctors overrode more of them each year, even those about life-threatening reactions, because the tool trained them to. Alert fatigue is not a flaw in the people. Every alert threshold is a choice about when to interrupt someone, and a threshold set too low teaches people to stop listening.
Check the two problems separately
The reverse also holds, and it is the encouraging part. AI tools that take work off the screen, such as note-taking tools that listen to a doctor's visit and write the note, improved doctors' reported burnout and workload in a randomized trial, although the time saved on notes was modest. An AI tool succeeds when it removes steps, not when it adds a smart step.
So a review of any AI tool needs two checks. First, are the answers good enough that people should spend their attention on them? Second, how many steps and interruptions stand between the answer and the decision? The second check needs no technical skill. Ask one person to make a normal decision with the tool while you count the clicks, and look at how many alerts people dismiss without acting. If most alerts are dismissed, raise the threshold until the tool speaks only when someone has to do something. That change is cheap, and it can do more for the tool than a better model would.
The companies that get the most from AI will probably not be the ones with the smartest tools. They will be the ones that treat alert settings and screen layout as decisions about people's attention, and keep those decisions out of the hands of whoever installed the software.