Fireworks AI is a service developers use to run open AI models, and Notion, the document and note-taking app, uses it behind its own AI assistant. Criminals broke into Fireworks' development machines through the cloud company that hosted them in October, and the files they took included chat transcripts from Notion's Insights feature. Fewer than 1% of Notion workspaces are affected, and for most of those the transcripts carry nothing that ties them to a person or company.
A promise to keep nothing leaves a gap
Notion's contract with Fireworks promised zero data retention, which means the provider answers a request and forgets it. Fireworks' no-retention setting held for ordinary traffic. The transcripts turned up in storage outside that setting.
A feature like Insights shows how a copy can exist beside that promise. An employee asks Notion AI a question, and the answer is produced and forgotten by the provider. Notion keeps the conversation, because the employee will want to read it again. Insights then has to label every saved conversation as drafting, contract questions, customer replies and so on, and that labeling is done in bulk. Notion hands a large pile of old chats to a model, and the pile has to be stored as a file until the job is done. That file is a second copy, and it lives at the supplier.
Fireworks suspects the thieves mainly wanted to resell access to AI models. That is little comfort to anyone whose client's name was in a chat, because once a file is out, what the thieves wanted no longer limits who reads it.
The company you signed with is not the one that was broken into
Picture a recruiting firm that uses Notion AI. Its staff paste in CV summaries, salary offers and notes from reference calls. The firm's contract is with Notion. Notion's contract is with Fireworks. Fireworks' machines were hosted by a cloud provider, and that was the door the criminals used. The firm's candidate notes could be three companies away from anyone it chose, and no contract it signed reaches the last one.
Saved AI conversations were already turning into a store of passwords and keys that staff pasted in without thinking. This is the supplier side of the same problem: the more features read back through that store, the more copies of it exist, and each copy is held by a company the customer never met.
What to ask before you switch a feature on
Any feature that summarizes, searches or reports on your team's past AI chats needs a copy of those chats. Before enabling one, ask three things: who stores the copy, for how long, and whether the no-retention promise in the contract covers this feature in writing. If the answers are vague, leave the feature off. It costs you a dashboard, and a dashboard is easy to add back later.
Staff habits matter too. Treat anything typed into an AI tool as something that may end up in a saved file, and keep passwords and customer identifiers out of it.
If your company uses Notion, the workspace owner has been contacted if it was identified as affected, and the investigation is still open. Notion has switched Insights off for now and stopped serving open models through Fireworks.