EVALUATION OF THE DIAGNOSTIC ACCURACY OF TWO AI MODELS FOR ANALYZING MAMMOGRAPHIC IMAGES

Abstract

Early diagnosis of breast cancer is crucial for improving prognosis and reducing mortality. Mammographic screening is a standard method for detecting malignant lesions, but its effectiveness depends on accurate interpretation and clear communication with patients. In recent years, artificial intelligence (AI) has shown promising results in medical diagnostics, yet its ability to analyze mammographic images and provide reliable patient guidance remains insufficiently studied. This pilot study evaluates the diagnostic accuracy and informational quality of two AI models—ChatGPT and X-Ray Insight. To assess their performance, we use a standardized database of mammographic images, including benign and malignant findings. Additionally, patient inquiry scenarios are used to evaluate the AI-generated explanations, recommendations, and overall clarity in response to common patient concerns. The results are compared against expert radiologist assessments. The AI models are evaluated based on diagnostic accuracy, clarity and correctness of explanations, usefulness of recommendations, and potential risks. Both models performed equally in determining the BI-RADS category, Differential Diagnosis, and Next Clinical Steps. The study underscores the challenges of integrating generative AI models into medical diagnostics, revealing both accurate insights and notable inaccuracies compared to expert interpretations. These discrepancies highlight the need for further training and refinement to improve the reliability and applicability of AI in clinical settings.