Measuring Competency: Improving the validity of your procedural performance assessments

Objective:. The objective of the study was to compare the use of ordinal scales and interval scales for capturing surgical competency information for general surgeons performing 3 complex trauma procedures. Background:. Surgical performance assessment is typically captured using nonparametric data (...

Full description

Saved in:
Bibliographic Details
Main Authors: Pamela B. Andreatta, PhD, Christopher H. Renninger, MD, Mark W. Bowyer, MD, Jennifer M. Gurney, MD
Format: Article
Language:English
Published: Wolters Kluwer Health 2023-12-01
Series:Annals of Surgery Open
Online Access:http://journals.lww.com/10.1097/AS9.0000000000000346
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Objective:. The objective of the study was to compare the use of ordinal scales and interval scales for capturing surgical competency information for general surgeons performing 3 complex trauma procedures. Background:. Surgical performance assessment is typically captured using nonparametric data (eg, checklists) that do not support inferential analyses. Interval scales support parametric analyses that are essential for determining competency. We compared assessment outcomes for surgeons performing 3 complex trauma procedures using ordinal and interval scales. Methods:. All participants were board-certified or eligible general surgeons. Each participant was assessed by an experienced trauma surgeon while performing 3 trauma procedures on cadavers. All assessors completed a rigorous assessment certification process. We calculated descriptive statistics to examine the differences between interval (parametric) and ordinal (nonparametric) outcomes. Results:. Ordinal scales overestimated competence in up to 100% of the participants and did not identify specific performance gaps. Interval scales provided more granularity and identified specific capability gaps. Conclusions:. Imprecise instrumentation conveys a false sense of competence and deprives surgeons of opportunities to close capability gaps. Measuring discrete procedural components with interval scales provides a more precise measurement of surgical competency.
ISSN:2691-3593