A Statistical Framework to Detect and Quantify Operator-Learning Curves in Medical Device Safety Evaluation
Henry C Ssemaganda,1 Sharon E Davis,2 Usha S Govindarajulu,3 Jejo D Koola,4 Jialin Mao,5 Dax Marek Westerman,6 Amy M Perkins,6 Theodore Speroff,7 Craig R Ramsay,8 Art Sedrakyan,5 Lucila Ohno-Machado,9 Michael E Matheny,10,11,* Frederic S Resnic1,12,* 1Division of Cardiovascul...
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Dove Medical Press
2025-07-01
|
| Series: | Medical Devices: Evidence and Research |
| Subjects: | |
| Online Access: | https://www.dovepress.com/a-statistical-framework-to-detect-and-quantify-operator-learning-curve-peer-reviewed-fulltext-article-MDER |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Henry C Ssemaganda,1 Sharon E Davis,2 Usha S Govindarajulu,3 Jejo D Koola,4 Jialin Mao,5 Dax Marek Westerman,6 Amy M Perkins,6 Theodore Speroff,7 Craig R Ramsay,8 Art Sedrakyan,5 Lucila Ohno-Machado,9 Michael E Matheny,10,11,* Frederic S Resnic1,12,* 1Division of Cardiovascular Medicine and Comparative Effectiveness Research Institute, Lahey Hospital and Medical Center, Burlington, MA, USA; 2Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, TN, USA; 3Center for Biostatistics, Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY, USA; 4Department of Medicine, University of California San Diego, La Jolla, CA, USA; 5Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA; 6Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA; 7Departments of Medicine and Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA; 8Health Services Research Unit, University of Aberdeen, Aberdeen, UK; 9Department of Biomedical Informatics and Data Science, Yale School of Medicine, Haven, CT, USA; 10Departments of Biomedical Informatics, Biostatistics, and Medicine, Vanderbilt University School of Medicine, Nashville, TN, USA; 11Geriatrics Research Education and Clinical Care Service, Tennessee Valley Healthcare System VA, Nashville, TN, USA; 12Department of Medicine, University of Massachusetts Chan School of Medicine, Worcester, MA, USA*These authors contributed equally to this workCorrespondence: Henry C Ssemaganda, Division of Cardiovascular Medicine and Comparative Effectiveness Research Institute, Lahey Hospital & Medical Center, 41 Mall Road, Burlington, MA, 01803, USA, Email semahen@gmail.comImportance: Safety issues leading to patient harm and significant costs have been identified in several post-market medical devices. Recently, powerful learning effects (LE) have been documented in numerous medical devices. Correctly attributing safety signals to learning or device effects allows for appropriate corrective actions and recommendations to improve patient safety.Objective: To develop and assess the statistical performance of an analytic framework to detect the presence of LE and quantify the learning curve (LC).Design and Setting: We generated synthetic datasets based on observed clinical distributions and complex feature correlations among patients hospitalized at US Department of Veterans Affairs facilities. Each dataset represents a hypothetical early experience in the use of high-risk medical devices, with a device of interest and a reference device. The study blinded the analysis team to the data-generation process.Methods: We developed predictive models using generalized additive models and estimated LC parameters using the Levenberg-Marqualdt algorithm. We evaluated the performance using sensitivity, specificity, and likelihood ratio (LR) in detecting the presence of LE and, if present, the goodness-of-fit of the estimated LC based on the root-mean squared error.Results: Among the 2483 simulated datasets, the median (IQR) number of cases was 218,000 (116,000– 353,000). LE were detected in 2065 of the 2291 datasets for which learning was specified (sensitivity: 90%; specificity: 88%; LR: 7). We adequately estimated the LC in 1632 (81%) of the 2013 datasets in which LE was detected and estimated LC.Discussion: This study demonstrated the framework to be robust in disentangling LE from device safety signals and in estimating LC.Conclusion: In medical device safety evaluation, the operator-learning effects associated with the safety of medical devices can be effectively modeled and characterized. This study warrants subsequent framework validation by using real-world clinical datasets.Keywords: post-market surveillance, generalized additive models, Levenberg-Marqualdt algorithm, learning curve |
|---|---|
| ISSN: | 1179-1470 |