ChatGPT as an Assessment Tool for MBBS Physiology Exams: A Comparative Study on MCQ and SAQ Answering Efficiency
DOI:
https://doi.org/10.31436/imjm.v25i01.3212Keywords:
ChatGPT, medical physiology, block examination, multiple-choice questions (MCQ), short-answer questions (SAQ)Abstract
INTRODUCTION: ChatGPT, a language model, is well-known for its capacity to generate human-like responses, but its use in medical education, particularly in assessment contexts, is underexplored. The aim of this study was to evaluate the efficiency of ChatGPT as an assessment tool in medical physiology examinations by comparing its performance in answering MCQs and SAQs. The findings of this study may impact the use of AI in medical education in a constantly digitised academic environment. MATERIALS AND METHODS: The study evaluated the performance of ChatGPT in answering 30 multiple-choice questions (MCQs) and 12 short-answer questions (SAQs) from each of the four physiology blocks. The questions were chosen from previous block exams to ascertain consistency. Two independent evaluators assessed the correctness and relevance of responses from ChatGPT using the answer key. The mean marks obtained by first-year medical students for 120 MCQs and 48 SAQs were compared with those of ChatGPT. RESULTS: ChatGPT performed better than first-year medical students in MCQs in all block exams and the difference in marks was statistically significant in blocks 1, 2, and 3. In SAQs, ChatGPT also performed better than the students in most questions. Students scored better in SAQ 11 in block 2, SAQ 12 in block 3 and SAQ 1, 2, 5 in block 3. CONCLUSION: ChatGPT is an effective AI tool for answering medical physiology questions. However, its performance varies across some MCQs and SAQs, indicating potential limitations in reasoning, contextual interpretation, and application-based problem-solving.
Downloads
Downloads
Published
How to Cite
Issue
Section
License
All material submitted for publication is assumed to be submitted exclusively to the IIUM Medical Journal Malaysia (IMJM) unless the contrary is stated. Manuscript decisions are based on a double-blinded peer review process. The Editor retains the right to determine the style and if necessary, edit and shorten any material accepted for publication.
IMJM retain copyright to all the articles published in the journal. All final ‘proof’ submissions must be accompanied by a completed Copyright Assignment Form, duly signed by all authors. The author(s) or copyright owner(s) irrevocably grant(s) to any third party, in advance and in perpetuity, the right to use, reproduce or disseminate the research article in its entirety or in part, in any format or medium, provided that no substantive errors are introduced in the process, proper attribution of authorship and correct citation details are given, and that the bibliographic details are not changed. If the article is reproduced or disseminated in part, this must be clearly and unequivocally indicated.
