A Novel CAD Framework with Visual and Textual Interpretability: Multimodal Insights for Predicting Respiratory Diseases
Küçük Resim Yok
Tarih
2024
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Institute of Electrical and Electronics Engineers Inc.
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
Generating textual interpretability using recent advancements in large language models (LLMs) is crucial for enhancing the efficiency of comprehensive computer-aided diagnosis (CAD) systems. This improves transparency between medical staff, intelligent CAD systems, and end-users by creating a trustworthy and effective intermediate medical diagnosis environment. In this paper, an innovative explainable throughout CAD system is introduced, designed to predict diseases from Chest X-rays (CXR) in a comprehensive scenario. The primary goal is to undertake multiple tasks that reduce the burden on medical staff and enrich CAD outcomes, including classification, visual explanations (heatmaps), and textual report generation. The proposed CAD system is developed through eight key steps: Data Collection and Annotation, Data Preparation, Text Vectorizations (Indexing), Visual Encoder, RAG-Fusion, Structural Prompt, XAI LLmTextual Reasoning (LLM Model), and Final Output (LLM textual report, image classification, and heatmap localization). The AI-based CAD system is trained and evaluated using the public benchmark MIMIC-CXR dataset with 14 different classes. The classification performance achieved an overall accuracy of 70 %, precision of 70 %, and F1-score of 0.60 %, while for text report generation, the system obtained an average BERTScore precision of 0.83, RougeL 0.16, and a Meteor score of 0.28. These promising results suggest the potential for further improvement of the CAD system and its applicability to real-world medical tasks. © 2024 IEEE.
Açıklama
8th International Artificial Intelligence and Data Processing Symposium, IDAP 2024 -- 21 September 2024 through 22 September 2024 -- Malatya -- 203423
Anahtar Kelimeler
Comprehensive CAD system, Large language model (LLM), Retrieval Augmented Generation (RAG), Text embedding, visual and textual interpretability
Kaynak
8th International Artificial Intelligence and Data Processing Symposium, IDAP 2024
WoS Q Değeri
Scopus Q Değeri
N/A
