Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students?
Introduction: Very Short Answer Questions (VSAQs) minimise cueing and simulate actual clinical practice more accurately than Single Best Answer Questions, as multiple-choice options might not be realistic. Phramongkutklao College of Medicine has developed a Self-Marked VSAQ (SM-VSAQ) for formative a...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
National University of Singapore
2025-04-01
|
| Series: | The Asia Pacific Scholar |
| Subjects: | |
| Online Access: | https://medicine.nus.edu.sg/taps/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850053462593634304 |
|---|---|
| author | Sethapong Lertsakulbunlue Anupong Kantiwong |
| author_facet | Sethapong Lertsakulbunlue Anupong Kantiwong |
| author_sort | Sethapong Lertsakulbunlue |
| collection | DOAJ |
| description | Introduction: Very Short Answer Questions (VSAQs) minimise cueing and simulate actual clinical practice more accurately than Single Best Answer Questions, as multiple-choice options might not be realistic. Phramongkutklao College of Medicine has developed a Self-Marked VSAQ (SM-VSAQ) for formative assessments. This study determines the validity and reliability of the SM-VSAQs.
Methods: Ninety-four third-year pre-clinical students took two occasions of 10-question SM-VSAQ exams regarding cardiovascular drugs. Each question consisted of two steps: (1) clinical vignettes with questions and (2) expected answers with scores, self-marking, and feedback comprehension. Scores ranged from 0.00 to 1.00 in 0.25 increments, though not every increment was applied to all questions. A distribution of the rating agreement between students' and teacher’s ratings was presented to determine criterion-related validity and inter-rater reliability.
Results: Criterion-related validity revealed 90.64% and 93.19% of the ratings demonstrated exact agreement between students' and teachers' ratings, with an inter-rater reliability of 0.972 and 0.977 for the first and second occasions, respectively (p=0.001). The exact agreement was relatively lower on the first occasion for questions with more diverse expected answers (85.11%, r=0.867, p=0.001) and drugs requiring their specific full names for a perfect mark (74.47%, r=0.849, p=0.001). While questions with specific guides do not require complex answers, they received a higher exact agreement.
Conclusion: The SM-VSAQ format effectively combines guided answers with the VSAQ model. The agreement with teacher-rated is excellent. Marking discrepancies rooted in misconceptions underscores the importance of teacher feedback in improving self-grading in formative assessments. Regular self-assessment practice is recommended to enhance grading accuracy. |
| format | Article |
| id | doaj-art-2c211dbb0a02401e9612ef6b2fcf26cc |
| institution | DOAJ |
| issn | 2424-9335 2424-9270 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | National University of Singapore |
| record_format | Article |
| series | The Asia Pacific Scholar |
| spelling | doaj-art-2c211dbb0a02401e9612ef6b2fcf26cc2025-08-20T02:52:31ZengNational University of SingaporeThe Asia Pacific Scholar2424-93352424-92702025-04-01102828510.29060/TAPS.2025-10-2/SC3345Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students?Sethapong Lertsakulbunlue0Anupong Kantiwong1Department of Pharmacology, Phramongkutklao College of Medicine, ThailandDepartment of Pharmacology, Phramongkutklao College of Medicine, ThailandIntroduction: Very Short Answer Questions (VSAQs) minimise cueing and simulate actual clinical practice more accurately than Single Best Answer Questions, as multiple-choice options might not be realistic. Phramongkutklao College of Medicine has developed a Self-Marked VSAQ (SM-VSAQ) for formative assessments. This study determines the validity and reliability of the SM-VSAQs. Methods: Ninety-four third-year pre-clinical students took two occasions of 10-question SM-VSAQ exams regarding cardiovascular drugs. Each question consisted of two steps: (1) clinical vignettes with questions and (2) expected answers with scores, self-marking, and feedback comprehension. Scores ranged from 0.00 to 1.00 in 0.25 increments, though not every increment was applied to all questions. A distribution of the rating agreement between students' and teacher’s ratings was presented to determine criterion-related validity and inter-rater reliability. Results: Criterion-related validity revealed 90.64% and 93.19% of the ratings demonstrated exact agreement between students' and teachers' ratings, with an inter-rater reliability of 0.972 and 0.977 for the first and second occasions, respectively (p=0.001). The exact agreement was relatively lower on the first occasion for questions with more diverse expected answers (85.11%, r=0.867, p=0.001) and drugs requiring their specific full names for a perfect mark (74.47%, r=0.849, p=0.001). While questions with specific guides do not require complex answers, they received a higher exact agreement. Conclusion: The SM-VSAQ format effectively combines guided answers with the VSAQ model. The agreement with teacher-rated is excellent. Marking discrepancies rooted in misconceptions underscores the importance of teacher feedback in improving self-grading in formative assessments. Regular self-assessment practice is recommended to enhance grading accuracy.https://medicine.nus.edu.sg/taps/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students/very short answer questionself-assessmentmedical educationundergraduatepharmacology |
| spellingShingle | Sethapong Lertsakulbunlue Anupong Kantiwong Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? The Asia Pacific Scholar very short answer question self-assessment medical education undergraduate pharmacology |
| title | Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? |
| title_full | Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? |
| title_fullStr | Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? |
| title_full_unstemmed | Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? |
| title_short | Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students? |
| title_sort | is self mark dependable in very short answer question formats among pre clinical medical students |
| topic | very short answer question self-assessment medical education undergraduate pharmacology |
| url | https://medicine.nus.edu.sg/taps/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students/ |
| work_keys_str_mv | AT sethaponglertsakulbunlue isselfmarkdependableinveryshortanswerquestionformatsamongpreclinicalmedicalstudents AT anupongkantiwong isselfmarkdependableinveryshortanswerquestionformatsamongpreclinicalmedicalstudents |