An AI scribe is simple to describe: it listens to a doctor-patient conversation, transcribes it, and produces a draft medical note automatically. The appeal is obvious. Doctors spend enormous amounts of time on paperwork, and burnout driven by administrative overload is a serious problem in healthcare systems worldwide. A tool that handles note-taking lets a doctor focus on the patient in front of them.
The adoption numbers reflect that appeal. In Australia, GP use nearly doubled between mid-2024 and late 2025. The US, UK, and Canada are seeing the same curve. Globally, the market for these tools was valued at around $2 billion in 2025 and is on track to reach well above $10 billion within a decade.
But adoption has outrun oversight by a significant margin, and the problems that creates are not hypothetical.
The first problem is where patient data goes. Most AI scribes are cloud-based, meaning the audio of a private medical consultation is sent to a server, often operated by a third-party company. Australia's federal health department has flagged that some vendors are apparently unaware their platforms route data outside the country. Under Australian privacy law, that matters. Under Europe's data protection rules, it can be outright illegal. The UK's NHS has issued detailed guidance requiring healthcare providers to complete formal privacy impact assessments before deploying any AI scribe, and to establish a legal basis for processing patient voice recordings, which count as a sensitive category of personal data.
The second problem is accuracy. AI scribes are built on the same large-scale language models that power tools like ChatGPT: powerful, but prone to generating plausible-sounding errors. Research from a major US hospital system found that the most common safety-related issue, accounting for nearly one in five safety complaints, was incorrect medication names, dosages, or instructions. In other cases, scribes recorded physical exams that never happened or invented diagnoses. Studies estimate hallucination rates at somewhere between 1% and 3% of generated content, which sounds low until you consider that these tools are being used for millions of consultations. A clinician survey found that over 90% of respondents had encountered AI-generated errors in medical contexts, and around 85% thought those errors were capable of causing patient harm.
The legal position is clear but uncomfortable: the doctor is responsible for reviewing and correcting the AI's output before it becomes part of the permanent record. The problem is that one of the main selling points of these tools is speed, and speed creates pressure to accept notes without careful review.
The third problem is consent. In Australia, government documents reveal wide variation in how doctors ask patients for permission to use a scribe, or whether they ask at all. In Melbourne, at least one psychiatrist has refused to accept patients who decline. Patients in several countries are reporting that they are being told to find another provider if they object. A class-action lawsuit was filed in the US in early 2026 against a major health system after a patient alleged his appointment was recorded using an AI scribe without his knowledge.
Regulatory responses are forming, but slowly. Australia is still deciding whether these tools should be classified as medical devices subject to formal oversight. The UK NHS endorsed their use in April 2025 but attached detailed compliance requirements around data protection and human review of AI output. Canada's provinces are adding their own guidance. None of these frameworks is yet binding in a way that meaningfully constrains how vendors sell or deploy these products.
For anyone whose business touches healthcare, whether as an employer managing occupational health, an insurer handling medical claims, or an operator running any service where staff interact with the public about health matters, the direction of travel is clear. Regulation is coming. The question is whether the first binding rules will arrive before a high-profile patient harm incident forces the issue.