Retrieval-augmented generation for BCLC staging classification using open-source large language models

Information from the abstract

Hepatocellular carcinoma represents a major global health burden, with liver cancer ranking among the leading causes of cancer-related mortality worldwide. In Thailand, it is the most prevalent among men and fifth among women. The Barcelona Clinic Liver Cancer (BCLC) classification system aids in treatment planning but faces challenges due to retrieving and interpreting burden from the fragmented information in medical records, resulting in inconsistent BCLC staging documentation. This proof-of-concept study explored the performance of large language models (LLMs) in BCLC staging using both standard prompting, Retrieval-Augmented Generation (RAG), and traditional natural language processing (NLP) baseline approach, focusing on the benefits of open-source models for enhanced data privacy and security. The experiment utilized clinical parameters from the electronic medical records of patients with primary liver cancer treated at the National Cancer Institute in Bangkok, Thailand, during 2024. We categorized experiments into three groups based on model sizes, with the GPT-5-mini and traditional NLP serving as the benchmark comparators. Our experiments found that the GPT-oss-20b model using RAG showed the best performance, achieving 87.69% accuracy with no statistically significant difference from GPT-5-mini. Models with over 20 billion parameters achieved accuracy between 48% and 82% with standard prompting and 57% and 88% with RAG, while smaller models (7-8 billion parameters) had lower accuracy of 9-56% with standard prompting and 36-53% with RAG, most of which failed to surpass the traditional NLP baseline(62.72%). RAG implementation improved performance in the majority of models by 3–39%, although two models (Qwen 8b and Deepseek 32b) showed marginal performance decrements, suggesting that RAG benefits are model-dependent. Therefore, for institutions with sufficient computational resources, open-source LLMs with RAG show promise as a potential alternative to both traditional NLP approaches and proprietary models. However, these findings are preliminary, and prospective multicenter validation is required before clinical implementation.

Why this record is monitored

This record has an Impact Signal of 72/100 based on recency, source, collaboration, and bibliographic signals. It prioritizes monitoring and is not a judgment of research quality.

Related topics: Topic Modeling · Text and Document Classification Technologies · Natural Language Processing Techniques