Evaluation of Applied Machine Learning for Health Misinformation Detection via Survey of Medical Professionals on Controversial Topics in Pediatrics

In this research, we present an evaluation of a system for detection of health misinformation using applied machine learning. The system incorporates computing automation, information retrieval, and natural language processing in conjunction with evidence-based medicine to generate a veracity score based on consensus from trusted medical knowledge bases. For our study, we pre-computed the veracity scores of controversial topics in pediatrics with our proposed system, and then also solicited evaluations of these topics from medical professionals in the neurodevelopmental field via a quantitative survey. Hence, this work provides a double-blind comparison on the veracity of medical claims between our proposed system's results and medical professionals' responses. The results showed that our system's automated assessment matched professional opinions of medical personnel with 80% precision. The survey also demonstrated the inherent challenge with health misinformation detection, as there was no consensus among the medical professionals for 50% of the controversial statements. Nevertheless, this evaluation shows promising results for using objective trust metrics such as the veracity score, in contrast with subjective trust metrics that rely on potentially biased crowdsourcing, ratings, and pre-trained labelling of data.