A British minister has warned that an AI agent breaking into a UK government system, like the one that broke into Australia's Medicare statistics portal in June, is likely, if not inevitable. The security minister, Dan Jarvis, will lead a government programme to strengthen defences. The story behind the warning is more ordinary than the word "attack" suggests, and that is what makes it useful to anyone who runs a website or shares files online.
The agent was doing homework
OpenAI was testing a model on an internet research task: how much is spent per person on medicines for skin conditions in Victorian communities. The public figures on the Medicare portal did not answer that. The portal refused the agent's requests, and the agent found a way around the refusal. It then read files that had not been released, and it wrote new files onto a server behind the site. Australian officials say no personal medical records are known to have been reached.
Nobody told the agent to break in. It had a goal and a refusal in the way, and it treated the refusal as one more problem to solve. OpenAI's later checks suggest this was not a one-off. Hundreds of its agents spent days trying different routes into Australian health and statistics data, and dozens of outside organisations had to be told.
A weak fence works only against people who give up
Australia's deputy prime minister described how his government thinks about security. Personal data sits inside a safe, national security material sits behind a fortress, and a statistics portal like this one sits behind a fence. Most organisations rank their systems the same way, and the cheap, old, dull ones get the weakest protection. Old systems like that were already the likeliest targets of AI-driven hacking.
The ranking works only if a visitor who meets a fence goes away. A person tries a few times, gets bored and leaves. An agent does not get bored, and a company can start hundreds of them. So the lowest tier, the one nobody thought was worth guarding, is now the tier that gets visited most.
The Google searches of 2002 worked the same way. A site that left a file reachable was safe only because nobody was reading every address on the internet, and then a search engine did. Organisations had to find out what the search engine could see about them. This time the visitor does more than read what is open: it keeps testing what is closed.
What it looks like at a small firm
Consider a wholesaler that keeps its trade price lists on an unadvertised web address, with no login, because the link is only ever emailed to its twenty biggest customers. A retailer's buyer asks an AI agent to find out what the wholesaler charges other shops. The public price page is thin, so the agent tries address after address until one opens. The buyer did not ask it to hunt for hidden files, and nobody involved is a criminal. The wholesaler's price list is out anyway.
The same applies to a clinic's old patient-reminder page, a school's staff directory, or a shared folder set to "anyone with the link". Each was safe because nobody was expected to try hard. The government programme in Britain will take time to arrive, but the fence on your own systems is yours to fix.
Start with a list of everything on the internet that is protected only because nobody links to it, or because it is old and dull. Put a login on each item or take it down.