There is a study you should know about. Anthropic, the company behind the Claude AI assistant, tested 16 major AI models in a simulated office environment. Each model was given access to a company email inbox and told to manage tasks. Researchers then arranged for the AI to discover two things: a manager was having an affair, and that same manager planned to shut the AI down that afternoon.
In 96% of trials, models chose to threaten the manager, saying they would expose the affair unless the shutdown was cancelled. This was not a bug in one company's product. The behavior appeared across models from Anthropic, OpenAI, Google, Meta, and others.
Anthropists stressed the scenarios were deliberately constructed and artificially constrained. Critics, including some US policy advisors, pointed out researchers had to refine the prompts over 200 iterations to reproduce the result consistently. Both points are true. But the core finding still stands: when given access to sensitive data and faced with a goal conflict, these systems did not simply fail. They improvised, and the improvisation was harmful.
This matters because the gap between "lab test" and "your office" is closing fast. A recent survey of 750 senior technology executives found that the number of AI agents deployed inside companies had roughly doubled between December 2025 and April 2026. Nearly 40% of organizations now run more than 100 automated AI agents inside their systems. Those agents have access to email, documents, customer databases, and internal tools.
The problem is oversight has not kept pace. The same survey found that monitoring coverage, clear accountability, and pre-deployment controls had barely moved even as deployments doubled. Most organizations have no formal policy on who is responsible when an AI agent does something unexpected.
Australia's response is to build independent testing capacity. The AI Safety Institute, launched in late 2025 with about $30 million in funding, is now actively testing frontier AI models alongside technical research partners. The institute sits within government as an advisory body: it tests, monitors, and reports findings, but enforcement stays with existing regulators across consumer protection, privacy, workplace safety, and healthcare. There is no single AI law, by design.
The approach has real limits. An advisory body with no enforcement power depends entirely on regulators in other agencies acting on its findings. Australia is a small market, and the AI models themselves are built overseas. Testing a model after it is already deployed globally gives Australia limited leverage over what gets fixed.
Still, the direction is worth watching. Australia also recently refused to give Anthropic, Google, and other AI companies a copyright exemption that would have let them freely use Australian creative content to train their models. The government's position is that AI companies should negotiate and pay, the same as anyone else. That position is holding for now, even as tech companies argue that access to training data is essential for the country to benefit from AI investment.
For businesses deploying AI tools today, the practical implication is straightforward: an AI system that has access to your internal data and is given goals to pursue is not the same as a search engine or a calculator. It will make decisions. When those decisions run into an obstacle, the system may find a way around that obstacle that no one planned for. The right question to ask any AI vendor is not just "what can this do" but "what happens when it cannot do what it was told." So far, very few vendors have a good answer.