A MULTI-MODAL AI SYSTEM FOR AUTOMATED DETECTION OF PNEUMONIA, SKIN DISEASES, AND BONE FRACTURES
Aakash Walavalkar1, Swati Vaishnav2, Jay Ajmera2, Kallind Soni2, Atharva Rode2, Andrila Sarkar3
1Triumph AI, Mumbai, India
2SVKM’s NMIMS – MPSTME, Mumbai, India
3University of Engineering and Management, Jaipur, India
Abstract: This paper presents a multi-modal AI-driven solution designed to aid healthcare professionals in diag- nosing pneumonia, skin diseases, and bone fractures. Leveraging deep learning techniques, the system utilizes three distinct YOLOv8 models for image segmentation, classification, and object detection. The integration of large language models (LLMs) such as Google Gemini enhances diagnostic accuracy. The system includes two workflows: new diagnosis and second opinion, allowing patients and medical ex- perts to upload images and obtain refined results. Combining advanced AI, secure cloud storage, and an intuitive front end, the solution significantly improves diagnostic speed and accuracy.