User-centric Evaluation of GenAI Alignment and Recommendations based on Predictive Learning Analytics
Küçük Resim Yok
Tarih
2025
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
CEUR-WS
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
Predictive models are one of the hallmarks of learning analytics research, relying on learner data to predict academic achievement and dropouts, enabling targeted interventions. Using a user-centric evaluation framework, we assessed the recommendations generated by ChatGPT based on the results of 136 studies that used student data for predictive modeling. The evaluation considered general attributes (accuracy, coherence, justification) as well as education-specific criteria (alignment with learning theories, ethics, learner-centeredness). The results indicate that, while LLM-generated recommendations are generally accurate, coherent and useful, they often lack alignment with diverse learning theories and fail to address inclusivity and higher-order cognitive skills effectively. Therefore, to operationalize LLMs to provide automated feedback to students, these aspects should be explicitly considered in the prompt design. © 2025 Copyright for this paper by its authors.
Açıklama
2nd International Workshop on Generative AI for Learning Analytics, GenAI-LA 2025 -- 3 March 2025 -- Dublin -- 210284
Anahtar Kelimeler
Generative AI, Large Language Models, Predictive Learning Analytics, User-centric Evaluation
Kaynak
CEUR Workshop Proceedings
WoS Q Değeri
Scopus Q Değeri
Q4
Cilt
3994
