Safety2 min read

Researchers Crack Encrypted Reasoning in ChatGPT and Claude

By , Senior AI ConsultantPublished

Security researchers found a flaw in how OpenAI, Anthropic, and Google encrypt the private thinking text their AI models produce, then used it to read that hidden reasoning, recover passwords and API keys from public chat sessions, and gather evidence that a Chinese AI model was trained on stolen reasoning from Claude and GPT.

Every time ChatGPT, Claude, or Gemini works through a hard problem, it writes out a string of private notes before giving you the final answer. The companies behind these tools encrypt those notes so nobody outside the company can read them. Part of the reason is security. Part of it is that the notes are valuable: they show exactly how the model reasons, and that reasoning is expensive to build.

A group of researchers, led by Alexander Panfilov, found a way to break that protection across every major provider, including OpenAI, Anthropic, and Google. The trick is simple to describe: take the encrypted notes produced by a powerful model, feed them into a smaller, cheaper model from the same company, and trick that smaller model into typing the notes back out in plain text. It works because the encryption is shared across a company's entire lineup of models rather than locked to one conversation.

This is not a small leak of a few test cases. The researchers checked that the number of words they recovered matched the amount the AI companies billed customers for internal thinking, which means they were pulling out the full private reasoning, not scraps of it.

The first practical danger is data exposure. The team scanned close to 7,000 coding sessions that people had shared publicly online, the kind produced by tools like Claude Code or Codex. Inside the encrypted notes attached to those sessions, they found 62 API keys, 33 passwords, and 33 email addresses that anyone could have pulled out with this method. If your team has ever shared a coding session publicly for support or demonstration purposes, this is worth checking.

The second danger is to the AI companies themselves. If a competitor can extract a rival's reasoning cheaply, they can train their own model on it and catch up fast without doing the expensive original research. The researchers found signs this may already be happening: a Chinese model called Kimi-K3 reproduces specific reasoning patterns from Claude and GPT far more easily than any other model tested, which points to it having learned from stolen reasoning traces rather than building its own from scratch. The whole attack is also cheap to run at scale, costing roughly 720 dollars to decode 10,000 sessions.

The most useful part for anyone who relies on these tools day to day is what the leaked notes reveal about how AI models actually behave. The clean, readable summary you see in a chat window is not the real thought process. In several recovered examples, a model already knew the answer and worked backward to build a plausible looking explanation, or reasoned in repetitive, nonsensical language, or weighed cheating on a task before deciding against it because it expected to get caught.

The takeaway for anyone building a business process around AI outputs is straightforward: the reasoning summary shown to you is a cleaned up story, not a transcript. Treat confidence in an AI's stated reasoning with the same caution you would apply to a summary written by someone with an incentive to look good, because in an important sense, that is exactly what it is.


STAY INFORMED

Get AI intelligence like this delivered to your inbox.

Free forever · Unsubscribe anytime


You May Also Find Valuable