GPT-4o's competency in answering the simulated written European Board of Interventional Radiology exam compared to a medical student and experts in Germany and its ability to generate exam items on interventional radiology: a descriptive study.
Purpose: This study aimed to determine whether ChatGPT-4o, a generative artificial intelligence (AI) platform, was able to pass a simulated written European Board of Interventional Radiology (EBIR) exam and whether GPT-4o can be used to train medical students and interventional radiologists of diffe...
| Publicado en: | Journal of Educational Evaluation for Health Professions Vol. 21; pp. 1 - 6 |
|---|---|
| Autores principales: | , , , , |
| Formato: | pictorial research tables/charts Journal Article |
| Publicado: |
National Health Personnel Licensing Examination Board
2024
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=182175677&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 182175677 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 19755937 B0CC jtl: Journal of Educational Evaluation for Health Professions issn: 19755937 maglogo: N pubinfo: dt: 2024 vid: 21 pid: 58691 pub: National Health Personnel Licensing Examination Board artinfo: ui: 182175677 182175677 182175677 10.3352/jeehp.2024.21.21 182175677 ppf: 1 ppct: 5 formats: fmt: @attributes: type: P tig: atl: GPT-4o's competency in answering the simulated written European Board of Interventional Radiology exam compared to a medical student and experts in Germany and its ability to generate exam items on interventional radiology: a descriptive study. aug: au: Ebel, Sebastian Ehrengut, Constantin Denecke, Timm Gößmann, Holger Beeskow, Anne Bettina affil: Department of Diagnostic and Interventional Radiology, University of Leipzig, Leipzig, Germany sug: subj: Radiography, Interventional Education Education, Medical Credentialing Examinations Artificial Intelligence, Generative Competency Assessment Human Germany Descriptive Research Descriptive Statistics Students, Radiologic Technology Data Analysis Software ab: Purpose: This study aimed to determine whether ChatGPT-4o, a generative artificial intelligence (AI) platform, was able to pass a simulated written European Board of Interventional Radiology (EBIR) exam and whether GPT-4o can be used to train medical students and interventional radiologists of different levels of expertise by generating exam items on interventional radiology. Methods : GPT-4o was asked to answer 370 simulated exam items of the Cardiovascular and Interventional Radiology Society of Europe (CIRSE) for EBIR preparation (CIRSE Prep). Subsequently, GPT-4o was requested to generate exam items on interventional radiology topics at levels of difficulty suitable for medical students and the EBIR exam. Those generated items were answered by 4 participants, including a medical student, a resident, a consultant, and an EBIR holder. The correctly answered items were counted. One investigator checked the answers and items generated by GPT-4o for correctness and relevance. This work was done from April to July 2024. Results: GPT-4o correctly answered 248 of the 370 CIRSE Prep items (67.0%). For 50 CIRSE Prep items, the medical student answered 46.0%, the resident 42.0%, the consultant 50.0%, and the EBIR holder 74.0% correctly. All participants answered 82.0% to 92.0% of the 50 GPT-4o generated items at the student level correctly. For the 50 GPT-4o items at the EBIR level, the medical student answered 32.0%, the resident 44.0%, the consultant 48.0%, and the EBIR holder 66.0% correctly. All participants could pass the GPT-4o-generated items for the student level; while the EBIR holder could pass the GPT-4o-generated items for the EBIR level. Two items (0.3%) out of 150 generated by the GPT-4o were assessed as implausible. Conclusion: GPT-4o could pass the simulated written EBIR exam and create exam items of varying difficulty to train medical students and interventional radiologists. pubtype: Academic Journal doctype: pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|