Evaluation of large language models for VI-RADS reports: a comparative analysis of zero-shot and few-shot prompting
BMC Medical Imaging, vol.26, no.1, 2026 (SCI-Expanded, Scopus)
- Publication Type: Article / Article
- Volume: 26 Issue: 1
- Publication Date: 2026
- Doi Number: 10.1186/s12880-026-02334-y
- Journal Name: BMC Medical Imaging
- Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, MEDLINE, Directory of Open Access Journals
- Keywords: Artificial intelligence in radiology, Bladder cancer, Few-shot learning, Large language models (LLMs), VI-RADS
- Acibadem Mehmet Ali Aydinlar University Affiliated: Yes
Abstract
Introduction: Accurate preoperative staging is vital in bladder cancer management, particularly for assessing muscle invasion. Multiparametric MRI (mpMRI) combined with the Vesical Imaging–Reporting and Data System (VI-RADS) offers a non-invasive and standardized approach for tumor stratification. However, inter-observer variability among radiologists remains a significant limitation. This study investigates the performance of three large language models (LLMs)—ChatGPT (OpenAI; GPT-5.2), Gemini (Google; Gemini 2.0), and Copilot (Microsoft; Copilot based on GPT-5 architecture)—in classifying bladder lesions according to VI-RADS using zero-shot and few-shot prompting strategies. Materials and Methods: A synthetic dataset of 100 simulated bladder cancer cases was developed from expert-crafted synthetic radiology reports based on structured text descriptors and expert consensus, comprising 20 cases for each VI-RADS category (1–5). Each case included structured imaging descriptors aligned with VI-RADS scoring criteria. The LLMs were evaluated under zero-shot (no examples provided) and few-shot (with illustrative examples) prompting conditions. Performance metrics included accuracy, macro F1 scores, and Cohen’s kappa, with statistical significance assessed using McNemar’s test. Results: Few-shot prompting significantly improved the classification performance of ChatGPT (OpenAI; GPT-5.2) (accuracy: 94%, F1: 0.939) and Gemini (Google; Gemini 2.0) (accuracy: 83%, F1: 0.823) compared to zero-shot. In contrast, Copilot (Microsoft; Copilot based on GPT-5 architecture)‘s accuracy (53%) and F1 score (0.503) declined under few-shot conditions. ChatGPT (OpenAI; GPT-5.2) demonstrated the highest consistency in identifying high-risk lesions (VI-RADS 4–5), followed by Gemini (Google; Gemini 2.0). Conclusion: Few-shot prompting enhances LLM performance in VI-RADS classification, particularly for ChatGPT (OpenAI; GPT-5.2) and Gemini (Google; Gemini 2.0). These findings highlight the potential of AI tools to support radiological decision-making in bladder cancer staging. Further studies using real imaging data and explainable AI are warranted.