a bold experiment on the verge of failure or the beginning of a new era in healthcare?

Google has developed a new artificial intelligence tool that can help people get answers to questions about their health. This is Med-PaLM 2, a specialized version of the PaLM 2 language model that underpins the Google Bard chatbot.

Med-PaLM 2 has been in trials since April this year at several research hospitals, including the Mayo Clinic, informs The Wall Street Journal. According to Google, this tool can be especially useful in countries with limited access to doctors. Med-PaLM 2 was trained on hand-picked examples of medical consultations, which Google says makes it better for health conversations than generic chatbots like Bard, Bing and ChatGPT.

However, The Wall Street Journal also mentions the Google study, published in May (pdf) which shows that Med-PaLM 2 still suffers from some of the accuracy problems we are used to seeing with large language models. In this study, physicians found more inaccuracies and out-of-date information in the responses provided by Med-PaLM and Med-PaLM 2 than in other physicians’ responses.

However, in almost every other measure, such as having a rationale, meeting consensus, or not showing signs of misunderstanding, Med-PaLM 2 performed about as well as real doctors.

Customers testing Med-PaLM 2 will have control over their data, which will be encrypted and Google won’t have access to it.

Med-PaLM 2 is still in the early stages of development, according to Google senior director of research Greg Corrado. Corrado said he wouldn’t want him to be part of his own family’s “medical journey”, but he believes Med-PaLM 2 “increases the number of places in healthcare where artificial intelligence can be useful, ten times.”