A Novel CAD Framework with Visual and Textual Interpretability: Multimodal Insights for Predicting Respiratory Diseases

Basit Öğe Kaydı
dc.contributor.authorMukhlis, Raza
dc.contributor.authorSaleem, Saied
dc.contributor.authorKwon, Hyunwook
dc.contributor.authorHussain, Jamil
dc.contributor.authorAydin, Ahmet Arif
dc.contributor.authorAl-Antari, Mugahed A.
dc.date.accessioned2026-04-04T13:18:59Z
dc.date.available2026-04-04T13:18:59Z
dc.date.issued2024
dc.departmentİnönü Üniversitesi
dc.description8th International Artificial Intelligence and Data Processing Symposium, IDAP 2024 -- 21 September 2024 through 22 September 2024 -- Malatya -- 203423
dc.description.abstractGenerating textual interpretability using recent advancements in large language models (LLMs) is crucial for enhancing the efficiency of comprehensive computer-aided diagnosis (CAD) systems. This improves transparency between medical staff, intelligent CAD systems, and end-users by creating a trustworthy and effective intermediate medical diagnosis environment. In this paper, an innovative explainable throughout CAD system is introduced, designed to predict diseases from Chest X-rays (CXR) in a comprehensive scenario. The primary goal is to undertake multiple tasks that reduce the burden on medical staff and enrich CAD outcomes, including classification, visual explanations (heatmaps), and textual report generation. The proposed CAD system is developed through eight key steps: Data Collection and Annotation, Data Preparation, Text Vectorizations (Indexing), Visual Encoder, RAG-Fusion, Structural Prompt, XAI LLmTextual Reasoning (LLM Model), and Final Output (LLM textual report, image classification, and heatmap localization). The AI-based CAD system is trained and evaluated using the public benchmark MIMIC-CXR dataset with 14 different classes. The classification performance achieved an overall accuracy of 70 %, precision of 70 %, and F1-score of 0.60 %, while for text report generation, the system obtained an average BERTScore precision of 0.83, RougeL 0.16, and a Meteor score of 0.28. These promising results suggest the potential for further improvement of the CAD system and its applicability to real-world medical tasks. © 2024 IEEE.
dc.description.sponsorshipNational Research Foundation of Korea, NRF; MSIT, (RS-2022-00166402, RS- 2023-00256517); Türkiye Bilimsel ve Teknolojik Araştırma Kurumu, TÜBİTAK, (123N325); Türkiye Bilimsel ve Teknolojik Araştırma Kurumu, TÜBİTAK
dc.identifier.doi10.1109/IDAP64064.2024.10710824
dc.identifier.isbn979-833153149-2
dc.identifier.scopus2-s2.0-85207866263
dc.identifier.scopusqualityN/A
dc.identifier.urihttps://doi.org/10.1109/IDAP64064.2024.10710824
dc.identifier.urihttps://hdl.handle.net/11616/108050
dc.indekslendigikaynakScopus
dc.language.isoen
dc.publisherInstitute of Electrical and Electronics Engineers Inc.
dc.relation.ispartof8th International Artificial Intelligence and Data Processing Symposium, IDAP 2024
dc.relation.publicationcategoryKonferans Öğesi - Uluslararası - Kurum Öğretim Elemanı
dc.rightsinfo:eu-repo/semantics/closedAccess
dc.snmzKA_Scopus_20250329
dc.subjectComprehensive CAD system
dc.subjectLarge language model (LLM)
dc.subjectRetrieval Augmented Generation (RAG)
dc.subjectText embedding
dc.subjectvisual and textual interpretability
dc.titleA Novel CAD Framework with Visual and Textual Interpretability: Multimodal Insights for Predicting Respiratory Diseases
dc.typeConference Object

Dosyalar

Koleksiyon

Scopus İndeksli Yayınlar Koleksiyonu