Measuring Competency: Improving the validity of your procedural performance assessments
Objective:. The objective of the study was to compare the use of ordinal scales and interval scales for capturing surgical competency information for general surgeons performing 3 complex trauma procedures. Background:. Surgical performance assessment is typically captured using nonparametric data (...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wolters Kluwer Health
2023-12-01
|
| Series: | Annals of Surgery Open |
| Online Access: | http://journals.lww.com/10.1097/AS9.0000000000000346 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832589735687618560 |
|---|---|
| author | Pamela B. Andreatta, PhD Christopher H. Renninger, MD Mark W. Bowyer, MD Jennifer M. Gurney, MD |
| author_facet | Pamela B. Andreatta, PhD Christopher H. Renninger, MD Mark W. Bowyer, MD Jennifer M. Gurney, MD |
| author_sort | Pamela B. Andreatta, PhD |
| collection | DOAJ |
| description | Objective:. The objective of the study was to compare the use of ordinal scales and interval scales for capturing surgical competency information for general surgeons performing 3 complex trauma procedures.
Background:. Surgical performance assessment is typically captured using nonparametric data (eg, checklists) that do not support inferential analyses. Interval scales support parametric analyses that are essential for determining competency. We compared assessment outcomes for surgeons performing 3 complex trauma procedures using ordinal and interval scales.
Methods:. All participants were board-certified or eligible general surgeons. Each participant was assessed by an experienced trauma surgeon while performing 3 trauma procedures on cadavers. All assessors completed a rigorous assessment certification process. We calculated descriptive statistics to examine the differences between interval (parametric) and ordinal (nonparametric) outcomes.
Results:. Ordinal scales overestimated competence in up to 100% of the participants and did not identify specific performance gaps. Interval scales provided more granularity and identified specific capability gaps.
Conclusions:. Imprecise instrumentation conveys a false sense of competence and deprives surgeons of opportunities to close capability gaps. Measuring discrete procedural components with interval scales provides a more precise measurement of surgical competency. |
| format | Article |
| id | doaj-art-e6f551c4604c4a499c1b3d5902a04d04 |
| institution | Kabale University |
| issn | 2691-3593 |
| language | English |
| publishDate | 2023-12-01 |
| publisher | Wolters Kluwer Health |
| record_format | Article |
| series | Annals of Surgery Open |
| spelling | doaj-art-e6f551c4604c4a499c1b3d5902a04d042025-01-24T09:18:12ZengWolters Kluwer HealthAnnals of Surgery Open2691-35932023-12-0144e34610.1097/AS9.0000000000000346202312000-00007Measuring Competency: Improving the validity of your procedural performance assessmentsPamela B. Andreatta, PhD0Christopher H. Renninger, MD1Mark W. Bowyer, MD2Jennifer M. Gurney, MD3From the * Department of Surgery, Uniformed Services University of the Health Science and the Walter Reed National Military Medical Center, Bethesda, MDFrom the * Department of Surgery, Uniformed Services University of the Health Science and the Walter Reed National Military Medical Center, Bethesda, MDFrom the * Department of Surgery, Uniformed Services University of the Health Science and the Walter Reed National Military Medical Center, Bethesda, MD† Department of Trauma, San Antonio Military Medical CenterObjective:. The objective of the study was to compare the use of ordinal scales and interval scales for capturing surgical competency information for general surgeons performing 3 complex trauma procedures. Background:. Surgical performance assessment is typically captured using nonparametric data (eg, checklists) that do not support inferential analyses. Interval scales support parametric analyses that are essential for determining competency. We compared assessment outcomes for surgeons performing 3 complex trauma procedures using ordinal and interval scales. Methods:. All participants were board-certified or eligible general surgeons. Each participant was assessed by an experienced trauma surgeon while performing 3 trauma procedures on cadavers. All assessors completed a rigorous assessment certification process. We calculated descriptive statistics to examine the differences between interval (parametric) and ordinal (nonparametric) outcomes. Results:. Ordinal scales overestimated competence in up to 100% of the participants and did not identify specific performance gaps. Interval scales provided more granularity and identified specific capability gaps. Conclusions:. Imprecise instrumentation conveys a false sense of competence and deprives surgeons of opportunities to close capability gaps. Measuring discrete procedural components with interval scales provides a more precise measurement of surgical competency.http://journals.lww.com/10.1097/AS9.0000000000000346 |
| spellingShingle | Pamela B. Andreatta, PhD Christopher H. Renninger, MD Mark W. Bowyer, MD Jennifer M. Gurney, MD Measuring Competency: Improving the validity of your procedural performance assessments Annals of Surgery Open |
| title | Measuring Competency: Improving the validity of your procedural performance assessments |
| title_full | Measuring Competency: Improving the validity of your procedural performance assessments |
| title_fullStr | Measuring Competency: Improving the validity of your procedural performance assessments |
| title_full_unstemmed | Measuring Competency: Improving the validity of your procedural performance assessments |
| title_short | Measuring Competency: Improving the validity of your procedural performance assessments |
| title_sort | measuring competency improving the validity of your procedural performance assessments |
| url | http://journals.lww.com/10.1097/AS9.0000000000000346 |
| work_keys_str_mv | AT pamelabandreattaphd measuringcompetencyimprovingthevalidityofyourproceduralperformanceassessments AT christopherhrenningermd measuringcompetencyimprovingthevalidityofyourproceduralperformanceassessments AT markwbowyermd measuringcompetencyimprovingthevalidityofyourproceduralperformanceassessments AT jennifermgurneymd measuringcompetencyimprovingthevalidityofyourproceduralperformanceassessments |