The world's contact center industry is worth roughly $100 billion and is growing fast, with voice calls still accounting for the largest share of interactions. At the same time, voice AI is moving from pilot projects into production at scale across customer support, IT helpdesks, and HR lines. A question that has sat unanswered: do these systems actually work when the caller speaks two languages at once?
A team published a benchmark this week that answers the question with real data. They built a dataset of 918 audio samples across four language pairs, had each utterance reviewed by a native-speaker linguist, and ran them through seven of the most widely used speech-to-text tools on the market. The scenarios covered realistic enterprise situations: password resets, payroll questions, VPN problems, benefits queries.
The short answer is that the best tools handle bilingual speech with surprisingly small accuracy losses compared to single-language audio. ElevenLabs Scribe V2 came out on top across all four language pairs in both raw transcription accuracy and meaning preservation. AssemblyAI Universal-3 Pro tied or came within fractions of a percentage point on raw accuracy, but Google Gemini 3 Flash overtook it when the test measured whether the correct meaning was captured, the kind of measurement that reflects whether an automated system can actually route a ticket correctly or answer a policy question.
The finding about Gemini makes intuitive sense. It was built with language understanding as its core strength, not just word-matching. That gives it an edge when a bilingual sentence is slightly mis-transcribed but the overall meaning still comes through. For practical purposes, if your voice system is feeding into a ticketing or case-management workflow, meaning accuracy is more operationally important than word-for-word accuracy.
OpenAI's Whisper, the tool that many teams chose in earlier wave of voice AI deployments because it was free and easy to set up, finishes last by a large margin. The reason is worth noting: without an explicit language setting, Whisper translates bilingual audio into English rather than transcribing it. A Spanish-English caller gets their message turned into English-only text, losing the structure of what they actually said. Teams running Whisper-based pipelines should check whether language detection is configured, or consider whether a more capable tool is warranted.
Two patterns from the research are worth internalizing. First, the number of times a speaker switches language within a sentence is the strongest predictor of whether an error occurs at all. Each language switch is another point where the tool can lose the thread. Second, when errors do occur, their size is driven less by how often the speaker switches and more by how densely the two languages are mixed throughout the sentence. The most thoroughly blended speech causes the biggest errors.
Also counterintuitive: errors concentrate on the English portions of bilingual sentences, not on the non-English portions. English is what most tools handle best in single-language audio, but when English appears as an embedded fragment inside a French or Spanish sentence, the tool has to change mental gear mid-stream, and that is exactly where it stumbles.
For businesses deciding which tool to build on, the practical guidance is clear. Scribe V2 and Universal-3 Pro are the safe choices on accuracy, with Scribe holding a narrow lead. If your workflow cares most about correct meaning rather than correct words, Gemini belongs in the evaluation. Deepgram Nova-3 performs reasonably on accuracy but drops sharply on meaning preservation, which is a gap worth knowing before committing to it.
One honest caveat: the benchmark used synthesized audio, not recordings of real bilingual speakers. Real callers add background noise, accents, hesitations, and regional pronunciation differences that synthetic audio cannot fully replicate. The relative rankings are informative, but any business with a significant bilingual customer base should run a test on their own real call recordings before committing to a production choice.