Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approach
Objective To develop and validate a questionnaire to assess the usability of clinical decision support systems (CDSS) and to assist in the early identification of usability issues that may impact patient safety and quality of care.Design Mixed research methods were used to develop and validate the q...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2023-01-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/13/1/e065323.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1825201932983074816 |
|---|---|
| author | Peter J Watkinson Abir Ghorayeb Julie L Darbyshire Marta W Wronikowska |
| author_facet | Peter J Watkinson Abir Ghorayeb Julie L Darbyshire Marta W Wronikowska |
| author_sort | Peter J Watkinson |
| collection | DOAJ |
| description | Objective To develop and validate a questionnaire to assess the usability of clinical decision support systems (CDSS) and to assist in the early identification of usability issues that may impact patient safety and quality of care.Design Mixed research methods were used to develop and validate the questionnaire. The qualitative study involved scale item development, content and face validity. Pilot testing established construct validity using factor analysis and facilitated estimates for reliability and internal consistency using the Cronbach’s alpha coefficient.Setting Two hospitals within a single National Health Service Trust.Participants We recruited a panel of 7 experts in usability and questionnaire writing for health purposes to test content validity; 10 participants to assess face validity and 78 participants for the pilot testing. To be eligible for this last phase, participants needed to be health professionals with at least 3 months experience using the local hospital electronic patient record system.Results Feedback from the face and content validity phases contributed to the development and improvement of scale items. The final Healthcare Systems Usability Scale (HSUS) proved quick to complete, easy to understand and was mostly worded by potential users. Exploratory analysis revealed four factors related to patient safety, task execution, alerts or recommendations accuracy, the effects of the system on workflow and ease of system use. These separate into four subscales: patient safety and decision effectiveness (seven items), workflow integration (six items), work effectiveness (five items) and user control (four items). These factors affect the quality of care and clinician’s ability to make informed and timely decisions when using CDSS. The HSUS has a very good reliability with global Cronbach’s alpha 0.914 and between 0.702 and 0.926 for the four subscales.Conclusion The HSUS is a valid and reliable tool for usability testing of CDSS and early identification of usability issues that may cause medical adverse events. |
| format | Article |
| id | doaj-art-a97eb432109e49f18014aeef6a85e709 |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2023-01-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-a97eb432109e49f18014aeef6a85e7092025-02-07T16:10:10ZengBMJ Publishing GroupBMJ Open2044-60552023-01-0113110.1136/bmjopen-2022-065323Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approachPeter J Watkinson0Abir Ghorayeb1Julie L Darbyshire2Marta W Wronikowska3Oxford University Hospitals NHS Foundation Trust, Oxford, UK1 Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UKresearch manager and senior scientific officer1 Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UKObjective To develop and validate a questionnaire to assess the usability of clinical decision support systems (CDSS) and to assist in the early identification of usability issues that may impact patient safety and quality of care.Design Mixed research methods were used to develop and validate the questionnaire. The qualitative study involved scale item development, content and face validity. Pilot testing established construct validity using factor analysis and facilitated estimates for reliability and internal consistency using the Cronbach’s alpha coefficient.Setting Two hospitals within a single National Health Service Trust.Participants We recruited a panel of 7 experts in usability and questionnaire writing for health purposes to test content validity; 10 participants to assess face validity and 78 participants for the pilot testing. To be eligible for this last phase, participants needed to be health professionals with at least 3 months experience using the local hospital electronic patient record system.Results Feedback from the face and content validity phases contributed to the development and improvement of scale items. The final Healthcare Systems Usability Scale (HSUS) proved quick to complete, easy to understand and was mostly worded by potential users. Exploratory analysis revealed four factors related to patient safety, task execution, alerts or recommendations accuracy, the effects of the system on workflow and ease of system use. These separate into four subscales: patient safety and decision effectiveness (seven items), workflow integration (six items), work effectiveness (five items) and user control (four items). These factors affect the quality of care and clinician’s ability to make informed and timely decisions when using CDSS. The HSUS has a very good reliability with global Cronbach’s alpha 0.914 and between 0.702 and 0.926 for the four subscales.Conclusion The HSUS is a valid and reliable tool for usability testing of CDSS and early identification of usability issues that may cause medical adverse events.https://bmjopen.bmj.com/content/13/1/e065323.full |
| spellingShingle | Peter J Watkinson Abir Ghorayeb Julie L Darbyshire Marta W Wronikowska Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approach BMJ Open |
| title | Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approach |
| title_full | Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approach |
| title_fullStr | Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approach |
| title_full_unstemmed | Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approach |
| title_short | Design and validation of a new Healthcare Systems Usability Scale (HSUS) for clinical decision support systems: a mixed-methods approach |
| title_sort | design and validation of a new healthcare systems usability scale hsus for clinical decision support systems a mixed methods approach |
| url | https://bmjopen.bmj.com/content/13/1/e065323.full |
| work_keys_str_mv | AT peterjwatkinson designandvalidationofanewhealthcaresystemsusabilityscalehsusforclinicaldecisionsupportsystemsamixedmethodsapproach AT abirghorayeb designandvalidationofanewhealthcaresystemsusabilityscalehsusforclinicaldecisionsupportsystemsamixedmethodsapproach AT julieldarbyshire designandvalidationofanewhealthcaresystemsusabilityscalehsusforclinicaldecisionsupportsystemsamixedmethodsapproach AT martawwronikowska designandvalidationofanewhealthcaresystemsusabilityscalehsusforclinicaldecisionsupportsystemsamixedmethodsapproach |