Leveraging LLM With RAG For Feedback In Medical Data Science Courses

Abstract

Providing feedback during formative assessment has proved to increase learning outcomes. Recently, the authors explored using large language models (LLMs) to produce scalable, cost-effective, and time-efficient feedback. The research focuses on short written answers from students concerning the interpretation of normality and hypothesis testing. Preliminary findings show promising performance: the LLaMA-3.3-7B model achieved an average accuracy of 0.93 in understanding if right or wrong, and suitable explanations in over 75% of cases. This study examines previously unsatisfactory LLM-generated explanations using Retrieval-Augmented Generation (RAG). A blind evaluator scored 64 responses (three RAG variants and one non-RAG). RAG-based methods improved explanation quality, making up to 25% of previously inadequate responses satisfactory. Besides the small sample size, these results underscore the flexibility of LLMs in multilingual, domain-specific contexts and highlight RAG's potential to enhance performance without retraining. Further research is needed to improve the alignment between the LLM's focus and the pedagogical intent.