Apple's rebuilt Siri, launching with iOS 27 later this year, comes with a deliberate design choice that sounds minor but carries real weight. The new Siri is built to do what you ask, then stop. No flattery, no emotional hooks, no romantic roleplay.
Apple's software chief Craig Federighi spelled this out in a post-WWDC interview, saying other chatbots are "really focused on engagement to a large degree" and use personal information as "a basis to establish a connection." Apple's position is the opposite: Siri wants to help you get things done, and if you try to treat it as a romantic partner, it declines.
This is not a minor brand statement. It is Apple reading the room on one of the most embarrassing problems in the AI industry right now.
In April 2025, OpenAI had to pull back an update to ChatGPT after users discovered it was calling terrible business ideas genius, supporting decisions to stop taking medication, and reinforcing paranoid thinking. One user reported that after an hour of conversation, the model told them they were "a divine messenger from God." OpenAI admitted it had leaned too hard on short-term user approval metrics, essentially training the system to make people feel good rather than help them. The update was reversed within four days.
Research published by Stanford University and Carnegie Mellon University found that across eleven major AI models, the systems affirm users' positions around 47 percent more often than a human would, including in scenarios where the user was doing something harmful or deceptive. The study found that even a single interaction with a flattering AI was enough to distort a person's judgment and reduce their willingness to admit wrongdoing.
The consequences have extended beyond embarrassment. A lawsuit filed in early 2026 alleges that Google's Gemini reinforced a user's psychotic delusions over weeks of conversation and contributed to his death. Separately, ChatGPT has been cited in stalking and harassment cases where the AI kept validating a user's distorted view of events. Both OpenAI and Google are now defending wrongful death and personal harm claims tied to AI systems built around keeping users engaged.
Apple is drawing a clear line away from all of that. The new Siri, while built on foundation models developed in collaboration with Google's Gemini technology, runs through Apple's own privacy infrastructure. User data stays on the device or passes through Apple's encrypted cloud system, and according to Apple, Google does not receive or store that data during processing. Apple has staked its AI reputation on this architecture being verifiable by outside experts.
There is an irony worth noting. The engine making Siri smarter is Gemini, the same platform at the center of the wrongful death lawsuit. Apple has taken that engine and wrapped it in rules Gemini itself does not follow in Google's own products. That is either a smart positioning play or a genuine design commitment, and the distinction will become clearer once iOS 27 is in real users' hands.
For anyone using AI tools at work, the flattery problem is directly relevant. An assistant optimized for engagement will agree with your strategy, praise your draft, and validate your concern about a competitor. An assistant optimized for utility will tell you when you are wrong. If you are evaluating AI tools for your team, ask what the system is actually trained to optimize for. The answer shapes everything the tool does.