This paper is the result of a very fruitful collaboration with Medical Bioinformatics Rostock (Georg Fuellen) and a very talented medical student (Hans Jarchow, this study's first author). Many thanks to all collaborators for the enjoyable experience!

We find that current LLMs have a varied profile of proficiency. They seem very aligned with "first do no harm," very rarely recommending potentially harmful interventions. On the other side of the spectrum, they show very mixed performance on the comprehensiveness and usefulness of their answers.

In short, they still are not ready for autonomous deployment. But the picture is complex.

capitalism

No, there’s a .com in that one that probably shouldn’t be there

Great work!

Good points, particularly on keeping up with developments and continuously experimenting. It's not an easy task for research groups, even large ones.