FractureSight-HCI: A Human-Centered Interactive System for AI-Assisted X-ray Fracture Visualization
DOI:
https://doi.org/10.65405/ke61pw60الكلمات المفتاحية:
Human–Computer Interaction, Image Processing, X-ray Fracture Detection, YOLOv8, Object Detection, Medical Image Analysis, Clinical Decision Support, Human-in-the-loop, AI-Assisted Visualization, Visual Explanationالملخص
Fracture detection in X-ray images is a critical task in medical image analysis, where accurate local-
ization of suspected fracture regions can support faster, clearer, and more focused clinical interpretation.
In this work, we present FractureSight-HCI, a human-centered interactive system for AI-assisted X-ray
fracture visualization and clinical decision support. The proposed system combines an image process-
ing and deep learning pipeline with an interactive Human–Computer Interaction interface to transform
model predictions into clear visual outputs that can be reviewed by physicians. The technical compo-
nent of the system was developed using a YOLOv8-based object detection model trained on more than
20,000 radiographic fracture samples. The training process was conducted using Google Colab and con-
tinued for more than three hours, reflecting a practical experimental workflow rather than a simple static
demonstration.
The trained model was evaluated using standard object detection metrics. The preliminary exper-
imental results achieved a Precision of 89.5%, Recall of 79.5%, mAP50 of 89.6%, and mAP50-95 of
49.3%. These results indicate promising fracture localization capability, especially for identifying suspi-
cious regions that require visual clinical review. However, the contribution of this work is not limited to
model performance. A major focus is the design of an interpretable and physician-centered interface that
displays the original X-ray image, presents the AI-assisted fracture localization result, and highlights the
suspected fracture region using a bounding box. This visual presentation reduces ambiguity in the AI
output and supports a more understandable decision-support workflow.
From the HCI perspective, the system was designed according to a human-in-the-loop approach,
where the AI model provides visual assistance while the physician remains responsible for the final
medical decision. The interface supports image selection, visual analysis, highlighted fracture visual-
ization, and clinical review in a simplified workflow. By integrating large-scale model training, image
processing, object detection, visual explanation, and human-centered interaction, FractureSight-HCI
demonstrates a practical and structured approach for presenting medical AI results in a clinically mean-
ingful and user-friendly manner. The proposed system highlights the importance of combining technical
accuracy with interface clarity, interpretability, and human control in AI-assisted medical imaging ap-
plications.
التنزيلات
المراجع
[1] E. Nagy, M. Janisch, F. Hržić, E. Sorantin, and S. Tschauner, “A pediatric wrist trauma X-ray
dataset (GRAZPEDWRI-DX) for machine learning,” Scientific Data, vol. 9, Art. no. 222, 2022,
doi: 10.1038/s41597-022-01328-z.
[2] E. Nagy, M. Janisch, F. Hržić, E. Sorantin, and S. Tschauner, “GRAZPEDWRI-DX,” figshare,
2022, doi: 10.6084/m9.figshare.14825193.
[3] F. Hardalaç et al., “Fracture Detection in Wrist X-ray Images Using Deep Learning-Based Object
Detection Models,” Sensors, vol. 22, no. 3, Art. no. 1285, 2022, doi: 10.3390/s22031285.
[4] R.-Y. Ju and W. Cai, “Fracture Detection in Pediatric Wrist Trauma X-ray Images Using YOLOv8
Algorithm,” Scientific Reports, vol. 13, Art. no. 20077, 2023, doi: 10.1038/s41598-023-47460-7.
[5] Z. Su, X. Liu, and Z. Sun, “Skeletal Fracture Detection with Deep Learning: A Comprehensive
Review,” Diagnostics, vol. 13, no. 20, Art. no. 3245, 2023, doi: 10.3390/diagnostics13203245.
[6] J. Husarek et al., “Computer-assisted artificial intelligence fracture detection: a systematic review
and meta-analysis of diagnostic test accuracy studies,” Scientific Reports, vol. 14, Art. no. 22475,
2024, doi: 10.1038/s41598-024-73058-8.
[7] National Institute for Health and Care Excellence, “Artificial intelligence technologies to help de-
tect fractures on X-rays in urgent care: early value assessment,” NICE HealthTech Guidance, 2025.
[8] C. Farmer et al., “Artificial intelligence software to help detect fractures on X-rays in urgent care:
an early value assessment,” NIHR Health Technology Assessment / NCBI Bookshelf, 2026.
[9] P. Rajpurkar et al., “MURA: Large Dataset for Abnormality Detection in Musculoskeletal Radio-
graphs,” arXiv preprint arXiv:1712.06957, 2017.
[10] M. He, A. P. Sharp, and R. T. L. Chung, “A calibrated deep learning ensemble for abnormality
detection in musculoskeletal radiographs,” Scientific Reports, vol. 11, Art. no. 9097, 2021, doi:
10.1038/s41598-021-88578-w.
[11] H. Chen, C. Gomez, C.-M. Huang, and M. Unberath, “Explainable medical imaging AI needs
human-centered design: Guidelines and evidence from a systematic review,” npj Digital Medicine,
vol. 5, Art. no. 156, 2022, doi: 10.1038/s41746-022-00699-2.
[12] J. Kim, H. Maathuis, and D. Sent, “Human-centered evaluation of explainable AI applications:
A systematic review,” Frontiers in Artificial Intelligence, vol. 7, Art. no. 1456486, 2024, doi:
10.3389/frai.2024.1456486.
[13] Y. Rong et al., “Towards Human-Centered Explainable AI: A Survey of User Studies for Model
Explanations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 4,
pp. 2104–2122, 2024, doi: 10.1109/TPAMI.2023.3331846.
[14] S. Mohseni, N. Zarei, and E. D. Ragan, “A Multidisciplinary Survey and Framework for Design
and Evaluation of Explainable AI Systems,” ACM Transactions on Interactive Intelligent Systems,
vol. 11, no. 3–4, Art. no. 24, 2021, doi: 10.1145/3387166.
[15] J. Mongan, L. Moy, and C. E. Kahn Jr., “Checklist for Artificial Intelligence in Medical Imaging
(CLAIM): A Guide for Authors and Reviewers,” Radiology: Artificial Intelligence, vol. 2, no. 2,
Art. no. e200029, 2020, doi: 10.1148/ryai.2020200029.
[16] K. Lekadir et al., “FUTURE-AI: International consensus guideline for trustworthy and deployable
artificial intelligence in healthcare,” BMJ, vol. 388, Art. no. e081554, 2025, doi: 10.1136/bmj-
2024-081554.
[17] A. Barredo Arrieta et al., “Explainable Artificial Intelligence (XAI): Concepts, taxonomies, op-
portunities and challenges toward responsible AI,” Information Fusion, vol. 58, pp. 82–115, 2020,
doi: 10.1016/j.inffus.2019.12.012.
[18] A. Adadi and M. Berrada, “Peeking Inside the Black-Box: A Survey on Explainable Artificial
Intelligence,” IEEE Access, vol. 6, pp. 52138–52160, 2018, doi: 10.1109/ACCESS.2018.2870052.
[19] S. Amershi et al., “Guidelines for Human-AI Interaction,” in Proceedings of the 2019 CHI Confer-
ence on Human Factors in Computing Systems, 2019, pp. 1–13, doi: 10.1145/3290605.3300233.
[20] A. Holzinger, G. Langs, H. Denk, K. Zatloukal, and H. Müller, “Causability and Explainability of
Artificial Intelligence in Medicine,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge
Discovery, vol. 9, no. 4, Art. no. e1312, 2019, doi: 10.1002/widm.1312.
