Automated Educational Question Generation at Different Bloom�s Skill Levels Using Large Language Models: Strategies and Evaluation

Scaria, N and Dharani Chenna, S and Subramani, D (2024) Automated Educational Question Generation at Different Bloom�s Skill Levels Using Large Language Models: Strategies and Evaluation. In: 25th International Conference on Artificial Intelligence in Education, AIED 2024, 8 July 2024through 12 July 2024, Recife, Brazil., pp. 165-179.

PDF AIED_2024.pdf Restricted to Registered users only Download (1MB) | Request a copy

Abstract

Developing questions that are pedagogically sound, relevant, and promote learning is a challenging and time-consuming task for educators. Modern-day large language models (LLMs) generate high-quality content across multiple domains, potentially helping educators to develop high-quality questions. Automated educational question generation (AEQG) is important in scaling online education catering to a diverse student population. Past attempts at AEQG have shown limited abilities to generate questions at higher cognitive levels. In this study, we examine the ability of five state-of-the-art LLMs of different sizes to generate diverse and high-quality questions of different cognitive levels, as defined by Bloom�s taxonomy. We use advanced prompting techniques with varying complexity for AEQG. We conducted expert and LLM-based evaluations to assess the linguistic and pedagogical relevance and quality of the questions. Our findings suggest that LLMs can generate relevant and high-quality educational questions of different cognitive levels when prompted with adequate information, although there is a significant variance in the performance of the five LLMs considered. We also show that automated evaluation is not on par with human evaluation. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

Item Type:	Conference Proceedings
Publication:	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Series.:	Lecture Notes in Computer Science
Publisher:	Springer Science and Business Media Deutschland GmbH
Additional Information:	The copyright for this article belongs to the publisher.
Keywords:	Automation; Blooms (metal); Computational linguistics; Quality control, Automated educational question generation; Bloom�s taxonomy; Cognitive levels; High quality; Language model; Large language model; Model evaluation; Modelling strategies; Skill levels; Time-consuming tasks, Taxonomies
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	29 Aug 2024 07:00
Last Modified:	29 Aug 2024 07:00
URI:	http://eprints.iisc.ac.in/id/eprint/86059

Actions (login required)

View Item