In a Boston-based trial involving 98 patients, a Google-developed AI chatbot gathered symptom histories ahead of scheduled physician appointments, and none of the overseeing doctors found it necessary to intervene in any conversation on safety grounds. Those findings, carried in The Lancet's Oct. 8 edition, draw on data gathered when a patient-facing diagnostic AI tool was deployed and evaluated within an active clinical setting — among the earliest trials of this kind conducted outside a laboratory.

Conducted within the outpatient primary care clinic at Beth Israel Deaconess Medical Center, the investigation drew its leadership from a team spanning that hospital and Google. Before their in-person appointments, a group of patients scheduled for time-sensitive consultations held remote conversations with an AI system named AMIE from their own homes.

Turning to a chatbot to enter symptoms before booking a GP appointment has become a routine step for a notable portion of the population. The research posed a more bounded question: whether such a system could hold conversations with actual patients without harm, provided physicians remained present to oversee the interactions. Provisions for oversight of safety were embedded within the system from the outset of its design.

A preprint of the study describes how seven physicians holding board certification in internal medicine monitored each text-based exchange as it unfolded, watching through a secure video connection in real time. At no point did patients have direct access to the supervisors, nor did any supervisor hold the role of the patient's own primary care physician.

Four conditions under which a conversation could be halted were established beforehand: any indication of risk to the patient or another person, notable distress arising from the AI interaction, a clinical concern flagged by the supervising clinician, or the patient's own choice to withdraw. Across all 98 conversations examined, not one satisfied even a single one of those criteria.

Following each conversation, a review session allowed the supervisor to go over what had been said with the patient, addressing any inaccuracies that had come up. Among the instances the team documented, one involved the AI generating a claim that had no factual basis in the underlying data.

In five of the documented exchanges, supervisors stepped in to add clinical clarification, while one physician flagged a particular interaction as mildly harmful on the grounds that the patient may have grown anxious after AMIE introduced lymphoma among the potential diagnoses under consideration, a broadcast news outlet reported.

Dr. Adam Rodman, who leads AI programs at BIDMC's Carl J. Shapiro Institute for Research and Education, argued that any assessment of whether such tools save time should come only after they have been shown to function safely and acceptably within real clinical care workflows, while the study separately examined how useful AMIE's outputs proved to be in practice.

In 90 out of 100 cases examined, the differential diagnoses generated by the AI tool included the same conclusion a physician ultimately reached, based on figures drawn from Google's account of the research. When blinded evaluations were conducted, AMIE and primary care physicians reached comparable conclusions on both diagnosis and treatment planning, with the physicians drawing ahead on care that accounted for real-world practicalities and cost considerations.

Of the doctors who consulted AMIE's pre-visit summaries, roughly three in four reported that the material aided their preparation. Across a majority of the reviewed cases, physicians reported that the AI findings prompted them to reconsider how they would manage a patient's treatment.

Opinion on the matter split among those receiving care. Attitudes toward the AI tool lifted following the conversation and held at that elevated level through the subsequent visit, with a majority of participants indicating they would turn to AMIE again. Fewer than half of those surveyed expressed confidence that the system would protect their data or operate with honesty and transparency.

Research findings cited by Rodman point to patients' existing bonds with their human care providers as the foundation upon which trust in medical AI tends to rest. Presenting a vision of three-way collaboration among patients, human clinicians, and artificial intelligence, Dr. Marc Cohen — who holds the clinical chief of primary care role at BIDMC and contributed as a co-senior author — described that model as the direction his team sees for delivering care grounded in evidence.

Constructed to evaluate feasibility and safety rather than to establish whether AI delivers health benefits, the trial had a narrower remit than outcome-focused research. The study drew participants from a single Boston clinic, lacked any control group, and enrolled only those who volunteered to take part.

Among the 1,452 urgent visits logged across the study window, 140 patients met the eligibility criteria and gave their consent, with 98 of that group going on to complete both the chat component and their clinic appointment. Among those barred from taking part were pregnant patients, individuals seeking mental health treatment, and anyone whose condition required emergency attention.

Without access to patient records and lacking the ability to conduct a hands-on examination, the AI operated under two fundamental constraints on its diagnostic reach. The study group leaned towards younger adults who were at ease with digital tools, and the act of observation itself may have shaped how those individuals responded.

Enrollment in ClinicalTrials.gov placed the study within the prospective observational cohort framework. Alphabet — the parent company of Google — provided the funding for the research, and Rodman held a visiting researcher position at Google for a portion of the work, BIDMC disclosed.

Whether AI diagnostic tools such as AMIE genuinely improve patient outcomes remains an open question, one that Google notes can only be resolved through expanded clinical trials of greater scale. Developed purely for research purposes, AMIE has neither received regulatory approval for clinical use nor been made accessible to the general public.

The study findings do not alter patients' approach to seeking care. People who turn to consumer chatbots for help making sense of their symptoms are generally advised by health professionals to treat the output as a starting point for further research rather than a definitive medical assessment.

Symptoms such as chest pain, breathing difficulties, sudden weakness, loss of orientation, or uncontrolled bleeding are among the conditions that health authorities indicate warrant a visit to emergency services rather than an online consultation tool. Patients whose clinic deploys an AI intake tool are in a position to ask who will review the data it collects, where that data is held, and whether a qualified clinician will assess it before any care-related determination is reached.