Businesses are adopting AI faster than most people realize. Across countries that track this, the share of firms using AI roughly doubled between 2023 and 2025. Most business leaders expect it to reshape how they operate within the next five years. But a quieter problem is showing up underneath that growth: the person who signs off on an AI decision is often not the person who actually made it.
Think about how this plays out in practice. A recruiter reviews a shortlist an AI system already ranked. A credit officer reviews a risk score an algorithm already calculated. A procurement manager reviews a supplier list an AI already ordered by preference. Each person technically approves the outcome. But the filtering, the ranking, and the framing happened before anyone looked at it. Researchers have a name for this gap: a moral crumple zone, where responsibility ends up sitting with the human who approved the output, even though that person had far less real control over how the decision was shaped than the system did.
This is not a hypothetical risk. Workday is currently defending a collective lawsuit alleging its applicant-screening software disqualified candidates over the age of 40 from getting hired, with a federal judge already letting the case move forward. Amazon shut down its own internal hiring algorithm in 2018 after it was found to score resumes from women lower for technical roles. State Farm is facing a lawsuit alleging that its algorithm for reviewing insurance claims treated Black homeowners worse than other applicants. In every one of these cases, a human was formally in charge. The software had already decided what deserved attention.
There is also a slower cost that does not show up in a lawsuit. Studies of customer service chatbots found that AI assistance let newer employees perform at the level of far more experienced ones, a pattern researchers describe as a leveling effect. That sounds good until you ask where the next generation of experienced judgment comes from, if the system is always doing the hard part of the thinking for the newest staff.
Regulators have noticed the same gap. Legal analysts studying the European Union's AI Act have warned that the requirement to keep a human in the loop often produces exactly the wrong outcome: oversight that looks real on paper but functions as a rubber stamp in practice, since the human has neither the time nor the information to meaningfully challenge what the system already decided.
None of this means businesses should slow down AI adoption. It means the checkpoint most companies rely on, a person clicking approve, is not doing the job people think it is. The fix is not more approval steps. It is making sure the people approving decisions can actually see what got filtered out, have a real and easy way to challenge the recommendation, and keep enough hands-on practice that their judgment does not quietly erode while the system looks like it is working fine.