| Sumario: | Background: In computing education, educators are constantly faced with the challenge of developing new curricula, including learning objectives (LOs), while ensuring that existing courses remain relevant. Large language models (LLMs) were shown to successfully generate a wide spectrum of natural language artefacts in computing education. Objectives: The objective of this study is to evaluate if it is feasible for a state‐of‐the‐art LLM to support curricular design by proposing lists of high‐quality LOs. Methods: We propose a simple LLM‐powered framework for the automatic generation of LOs. Two human evaluators compare the automatically generated LOs to the human‐crafted ones in terms of their alignment with course goals, meeting the SMART criteria, mutual overlap, and appropriateness of ordering. Results: We found that automatically generated LOs are comparable to LOs authored by instructors in many respects, including being measurable and relevant while exhibiting some limitations (e.g., sometimes not being specific or achievable). LOs were also comparable in their alignment with the high‐level course goals. Finally, auto‐generated LOs were often deemed to be better organised (order, non‐overlap) than the human‐authored ones. Conclusions: Our findings suggest that LLM could support educators in designing their courses by providing reasonable suggestions for LOs. Lay Description: What is already known about this topic: Large language models (LLMs) were shown to successfully generate a wide spectrum of natural language artefacts in computing education.GPT‐4 was used to automatically generate module level learning objectives for a practically oriented university course on artificial intelligence.The generated learning objectives were largely sensible and described key sub‐concepts related to the relevant topics.All the learning objectives were properly started with action verbs, and the distribution of action verbs across the learning objectives for conceptual modules and hands‐on projects was as expected.There is no other work utilising LLMs to generate LOs. What this paper adds: The paper goes beyond action verbs and surface language properties and focuses on evaluating the LOs in terms of the SMART criteria, alignment of the LOs with course goals and module topics, as well as the organization of multiple LOs into meaningful lists, that is, if the LOs were ordered appropriately and did not overlap.This is the first study that performs rigorous evaluation of LOs generated by an LLM (the SMART criteria, alignment, ordering, and non‐overlap). Implications for practice and/or policy: LLMs cannot be relied on to generate high‐quality LOs on their own.LLMs generate reasonable draft LOs that can be utilized by an instructor to author higher‐quality LOs more efficiently.
|
|---|