I am very excited to share the pre-print study on Atropos Health ChatRWD beta results(link to paper below). ChatRWD beta outperformed ChatGPT 3.5, Google Gemini 1.5 Pro. and Anthropic Claude 3 on answers to clinical questions when reviewed by independent physicians. OpenEvidence also outperformed the Big Tech LLMs by a wide margin. Most important to us, ChatRWD performed highest on Physician trust and Best Answer and was able to answer questions asked 94% of the time. As the Gen AI cycle continues to move at a rapid pace we firmly believe that "Trust" and "Accuracy" are going to be critical, and that publishing your performance on both is table stakes. It is also clear to me that in the quest to be able to answer 100% of questions with Clinical grade accuracy training LLMs solely on the literature will not be enough. We need to produce new evidence in the form of high-quality transparent studies using appropriate methodology. We are adding beta users post this announcement, so please sign up to try it, and we will have more to come on full launch this summer. Exciting times and hope everyone has a great July 4th. https://lnkd.in/es5gWK84 #generativeAI #LLM #Realworldevidence
Interested in having my physician wife test this and also see how well it replicates some of the hypothesis iteration I’ve done in RWD
Saif Al, MBA
2mo
Brilliant work, Brigham Hyde! I like how the study quantifies (RWD + PICO) * (RAG + Literature) = 60% actionability 👍
This is an important milestone in moving forward the inevitable automation of real world data science in healthcare, and the generation of automated RWE. None of the LLM's that we have tested have yet been able to produce meaningful and reliable answers to clinical questions. Therefore, this was a seriously challenging problem, and the investigators should be congratulated.
Heather Rosso
14h
That's fantastic!!!
Malcolm Baracho
1mo
Exciting news! Brigham Hyde ChatRWD beta's top performance in physician trust and accuracy underscores its innovative approach in healthcare. Looking forward to its industry impact!
Impressive
Maggie Sprenger
2mo
This is exceptional and transformative for the democratization of health care. Huge congrats to you and your team!
Congratulations
very impressive, Brigham!
CEO - Atropos Health
2mohttps://arxiv.org/abs/2407.00541