Direct oral penicillin challenge in secondary care with low-risk patients: the SPACE mixed-methods study with cost-effectiveness analysis
Some text in this abstract has been adapted from Krishna MT, Bhogal R, Ng BY, Kildonaviciute K, Jani YH, Williams I, et al. A multicentre observational study to investigate feasibility of a direct oral penicillin challenge in de-labelling ‘low risk’ patients with penicillin allergy by non-allergy he...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
NIHR Journals Library
2025-04-01
|
| Series: | Health and Social Care Delivery Research |
| Subjects: | |
| Online Access: | https://doi.org/10.3310/MTYW6557 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849725039419588608 |
|---|---|
| author | Mamidipudi Thirumala Krishna Yogini H Jani Iestyn Williams Ruben Mujica-Mota Rebecca Bestwick Michele Siciliano Robert Michael West Rashmeet Bhogal Bee Yean Ng Kornelija Kildonaviciute Rachel Pollard Nicola Jones Louise Dunsmure Mairead McErlean Neil Powell Chidanand C Hullur Ariyur Balaji Jonathan Sandoe Amena Warner Ron Daniels Caroline Thomas Siraj A Misbah Louise Savic |
| author_facet | Mamidipudi Thirumala Krishna Yogini H Jani Iestyn Williams Ruben Mujica-Mota Rebecca Bestwick Michele Siciliano Robert Michael West Rashmeet Bhogal Bee Yean Ng Kornelija Kildonaviciute Rachel Pollard Nicola Jones Louise Dunsmure Mairead McErlean Neil Powell Chidanand C Hullur Ariyur Balaji Jonathan Sandoe Amena Warner Ron Daniels Caroline Thomas Siraj A Misbah Louise Savic |
| author_sort | Mamidipudi Thirumala Krishna |
| collection | DOAJ |
| description | Some text in this abstract has been adapted from Krishna MT, Bhogal R, Ng BY, Kildonaviciute K, Jani YH, Williams I, et al. A multicentre observational study to investigate feasibility of a direct oral penicillin challenge in de-labelling ‘low risk’ patients with penicillin allergy by non-allergy healthcare professionals (SPACE study): Implications for healthcare systems. J Infect 2024;88:106116. https://doi.org/10.1016/j.jinf.2024.01.015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) license. See: https://creativecommons.org/licenses/by-nc-nd/4.0/. The text below includes minor additions and formatting changes to the original text. Background One in five inpatients carries a penicillin allergy label. However, 90–95% of labels are incorrect. Penicillin allergy labels lead to increased risk for serious hospital infections and longer hospital stay and are associated with higher estimated healthcare costs. Penicillin allergy testing is onerous and requires a specialist. Routine inpatient testing is not available. Recent evidence suggests that a direct oral penicillin challenge delivered by non-allergy specialists is safe in ‘low risk’ patients, who are highly unlikely to be allergic based on history. Aims To explore behaviour, attitudes and acceptability of patients, healthcare professionals and managers regarding a direct oral penicillin challenge in ‘low risk’ patients. To inform development of an implementation framework and determine potential cost-effectiveness. Methods This study (1 May 2021–30 April 2023) involved delivery of direct oral penicillin challenge by non-allergy specialists across three clinical settings (medical/infectious diseases wards, presurgical and haematology-oncology units) at three hospitals. The study had three workstreams: Workstream 1: Screening for potential suitability. Patients were stratified into ‘low risk’ and ‘high risk’. ‘Low-risk’ patients underwent direct oral penicillin challenge. Workstream 2: One-to-one semistructured interviews with patients (N = 43) and focus group (N = 28) discussions with stakeholders. Workstream 3: Care pathway mapping, decision-analytic modelling and value of information analysis were carried out to determine potential cost-effectiveness of direct oral penicillin challenge. Results One thousand and fifty-four of 2257 screened patients were eligible, 270 of 643 approached patients consented (42%). Two hundred and fifty-nine patients were risk-stratified (155 ‘low risk’; 104 ‘high risk’). Of the 155 ‘low risk’ patients, 126 underwent direct oral penicillin challenge, 122 (97%) were de-labelled with no serious allergic reactions and 43 patients were interviewed. Low-risk patients accepted their allergy labels, had limited knowledge of the adverse impact and most were keen to have their labels reviewed. Healthcare professionals demonstrated a risk-averse approach, although would engage in the intervention with training, resource availability and a governance framework in place. The total costs of the direct oral penicillin challenge pathway were higher than the costs of direct oral penicillin challenge alone (£940 vs. £98–288 per patient). There were minimal expected savings in antibiotic and hospital costs in the short term and potentially large healthcare cost savings over 5 years. Limitations Relatively small sample size for direct oral penicillin challenge, poor conversion rate, particularly in acute settings, patients with limited English language proficiency could not be included and the study was not sufficiently powered and controlled to conduct a cost-effectiveness evaluation. Conclusions This first multicentre United Kingdom study showed that non-allergy specialist-led direct oral penicillin challenge is feasible in secondary care. A high proportion of direct oral penicillin challenges were successful, with positive feedback from patients. Majority of screened patients did not progress through the study pathway. Going forward, a multipronged approach is needed to enhance equitability of direct oral penicillin challenge in routine practice. Follow-up mechanisms to consider the intervention during a clinically stable state and a governance framework for those lacking capacity to consent are needed. The cost of delivering a direct oral penicillin challenge pathway in its entirety is significantly higher than the costs of performing direct oral penicillin challenge per se. Future work A randomised controlled trial with long-term follow-up is needed to determine the cost-effectiveness of direct oral penicillin challenge. Study registration This study is registered as ISRCTN55524365. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: NIHR129069) and is published in full in Health and Social Care Delivery Research; Vol. 13, No. 9. See the NIHR Funding and Awards website for further award information.
Plain language summary Many patients who experience minor side effects when taking penicillin are wrongly labelled as ‘allergic’, leading to huge numbers of people with incorrect penicillin allergy labels in hospitals. Patients with an allergy label receive different antibiotics which might be toxic and less effective. Allergy tests are not routinely available due to very few specialists in the National Health Service. We developed a simple way to remove incorrect labels in patients at low risk of genuine allergy. This involves taking a careful history from the patient to decide if they are at low risk of having a genuine allergy to penicillin. Then, if suitable, they are offered a test dose or a challenge test (penicillin capsule to be swallowed without doing other allergy tests) in a hospital after taking prior informed consent. In this study, the test was performed by trained healthcare professionals with no specialist background in allergy. We included patients admitted in hospital wards and outpatients from surgery and cancer units in three hospitals. We did 126 challenge tests and showed that almost all (97%) did not have an allergy. One patient developed stomach upset and three had mild rash. Very few patients in the wards were eligible to participate for medical reasons, and a greater percentage of outpatients were suitable. We spoke to patients and healthcare professionals about this pathway. Professionals understood its importance but thought appropriate training, safety and resources were required. Patients were keen to find out if they were really allergic and felt safe undergoing the challenge test in hospital. The main reasons for not taking part were personal circumstances. We showed that the penicillin challenge test can be delivered by non-allergy healthcare professionals, but a smaller percentage of patients were eligible to undergo this test than that shown in previous studies. We developed recommendations for wider roll-out of this pathway and gained good understanding of how it would work in the ‘real world’. The cost saving appears small in the short term and more research is needed to understand the long-term benefits.
Scientific summary Some text in this scientific summary has been republished with permission from Krishna MT, Bhogal R, Ng BY, Kildonaviciute K, Jani YH, Williams I, et al. A multicentre observational study to investigate feasibility of a direct oral penicillin challenge in de-labelling ‘low risk’ patients with penicillin allergy by non-allergy healthcare professionals (SPACE study): Implications for healthcare systems. J Infect 2024;88:106116. https://doi.org/10.1016/j.jinf.2024.01.015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) license. See: https://creativecommons.org/licenses/by-nc-nd/4.0/. The text below includes minor additions and formatting changes to the original text. Background Penicillin allergy labels (PALs) occur in 6% of the population in England, and 15–20% of inpatients have a PAL in their records. Previous studies have shown that 90–95% of PALs are inaccurate. The current standard of care involves allergy specialist review, skin tests and a challenge procedure. There is an unmet need for allergy services and penicillin allergy testing is not routinely available. Penicillin allergy labels lead to use of alternative antibiotics which might be less effective, might enhance risk of antimicrobial resistance (AMR) and hospital-acquired infections, contribute to longer hospital stay, higher re-admission rates and mortality. Mis-labelling patients has been linked to suboptimal skills and knowledge gaps in allergy history taking and poor documentation in clinical records. Patients can be stratified into ‘low risk’ and ‘high risk’ based on a structured clinical history and scrutiny of clinical and prescription records. ‘Low risk’ patients are highly unlikely to have an immune-mediated reaction to penicillin. A direct oral penicillin challenge (DPC; without undertaking skin tests) has been shown to be safe and feasible and successfully delivered by non-allergy healthcare professionals (HCPs), such as clinical pharmacists and physicians. There is limited evidence from the UK, but there is a larger body of evidence from the USA and Australia regarding the success of DPC in mitigating the adverse impact of PALs. Safe implementation depends on an appropriate clinical setting and uptake amongst patients and relevant stakeholders. Objectives Primary To explore behaviour, attitudes and acceptability of patients, HCPs and managers regarding use of DPC in ‘low risk’ patients. To develop treatment pathways and a clinical governance framework for this service model. Secondary To assess the proportion of ‘low risk’ patients with a PAL who would be eligible for a DPC. To assess the proportion of ‘low risk’ patients who would be willing and will complete a DPC. To explore practical aspects of implementing this de-labelling programme in secondary care by investigating factors, such as organisational context, treatment pathway, protocol implementation, time taken and resources. To evaluate the potential cost-effectiveness of this service model. Methods This study spanned 24 months and involved three clinical settings [acute medical units (AMUs)/infectious diseases (ID) units, presurgical units and haematology-oncology units] in three hospitals including University Hospitals Birmingham NHS Foundation Trust, Leeds Teaching Hospitals and Oxford University Hospitals NHS Foundation Trust. The study included three workstreams (WS): Workstream 1 Patients with a PAL were identified and screened from respective Trust clinical records using a structured proforma employing predetermined study criteria. Clinically unstable patients, pregnant or breastfeeding patients, those with current COVID-19 infection, patients lacking capacity to give informed consent or taking part in another research intervention study were not approached. Those potentially suitable were approached for expression of interest. Informed consent was obtained from interested participants and a risk stratification process was conducted based on pre-determined study criteria. Participants were classified as ‘low risk’ and ‘high risk’. Risk stratification was conducted by non-allergy specialist HCPs (Research Pharmacists at Birmingham and Oxford and Research Nurses at Leeds) and a non-allergy specialist study consultant provided clinical supervision for the entire process. ‘Low risk’ patients were those who were highly unlikely to have a true allergy and ‘high risk’ patients gave a history suggestive of an immune-mediated reaction and/or had concomitant comorbidity, such as severe asthma or cardiac disorder. ‘Low risk’ patients were offered a DPC. This involved administration of a single dose of 500 mg amoxicillin and 1-hour observation to monitor for an immediate hypersensitivity reaction (HSR) followed by 250 mg twice daily for 3 days to assess for a delayed HSR. All patients were reviewed on day 5 to check for their clinical tolerance. One hundred and twenty-six DPCs were conducted [119 ‘opportunistic’ and 7 ‘therapeutic’ (to treat current bacterial infection)]. Data analysis Descriptive data were generated for all clinical settings across the three sites. To aid in the detection of potential associations, continuous variables were compared between the hospital/clinical setting groups using Kruskal–Wallis tests. Similarly, categorical variables were compared across hospital/clinical settings groups using Pearson chi-squared tests. Detailed analysis was provided to compare outcomes, such as de-labelled ‘Yes/No’, with the use of logistic regression. First, the outcome was regressed upon each potential ‘risk factor’, such as gender, producing an unadjusted odds ratio (OR; by exponentiation of the fitted parameter), and then a multivariable logistic regression used to yield adjusted ORs. Workstream 2 Patients ‘Low risk’ patients were invited to participate in the qualitative arm of the study at the time of recruitment to WS1. Purposive sampling was undertaken to achieve a diverse sample. One-to-one semistructured interviews were conducted with patients {total N = 43 [mean age 61 (± 14 standard deviation) years]} by telephone using a pre-specified interview schedule. The interview questions were informed by risk perception theories and developed iteratively with our patient and public partners to ensure face validity. Healthcare professionals and other stakeholders Three focus group discussions were conducted with 28 participants in total across the 3 centres. We purposively sampled staff to include representation from relevant stakeholders [including general practitioners (GPs), junior doctors, consultants, nurses, pharmacists, managers, commissioners] at each site. Focus groups were held in person (two sites) and online (one site). The discussions were audio-recorded and written notes were taken. Two members of the research team facilitated the discussions using a pre-specified topic guide to prompt healthy discussion, informed by relevant domains of the ‘Theoretical Domains Framework’. Audio recordings of interviews and focus groups were transcribed verbatim and coded. Thematic analysis was undertaken using both inductive and deductive approaches and was informed by ‘Theoretical Domains Framework’. Workstream 3 An economic evaluation of DPC relative to current standard of care, that is no PAL testing, was carried out in ‘low risk’ study participants. We estimated the costs of the DPC pathway at each study site and its expected impact on the costs of antibiotic regimens in the period immediately after DPC, for AMU/IDU and presurgical patients, or up to one course of chemotherapy after DPC in haematology-oncology patients. We combined prospectively collected individual patient data on the time that clinical and non-clinical staff spent in screening and risk stratification and retrospectively, collected data on staff time inputs into all DPC steps, to value time inputs according to the midpoint of the salary scales for the title and grade of staff and derived the costs of the full DPC pathway. We used Monte Carlo simulation methods to produce estimates of sampling uncertainty of costs in the form of 95% credible intervals (95% CrI). We also estimated the value of conducting further research to capture the economic impact of DPC in terms of hospital re-admissions, length of hospital stays of initial and subsequent admissions, and GP visits over a 4.5-year follow-up period, the maximum over which evidence was found in the literature. This was a value of information analysis with the expected value of perfect information (EVPI) as outcome, conducted separately for each new service pathway implemented across the three study sites. In sensitivity analysis, we explored the impact of increasing the proportion of eligible patients who are approached for consent to undergo testing, accounting for the costs of training and varying the staff skill mix to allow for the possibility that less experienced staff may deliver the new testing pathway. In addition, we held discussions with local managers at each study site to identify key questions emerging from our findings and areas of uncertainty that would need to be addressed by further studies. Patient and public involvement and engagement The investigators worked in collaboration with representatives from Allergy UK and the UK Sepsis Trust and two groups of members of the public in Oxford and Birmingham. Their input was sought for the study protocol, study amendments, participant recruitment strategies, patient facing documents, data interpretation and recommendations for further research. Results Workstream 1 A total of 2257 patients (834 males and 1423 female) were screened across the 3 clinical settings and the 3 participating sites. One thousand two hundred and three (53.3%) were deemed ineligible based on study criteria at screening. A total of 1054 (46.6%) patients were considered eligible, 643 were approached and 412 not approached due to practical factors. Of 643 patients who were approached, 373 declined to participate and 270 (116 males and 154 female) consented. The overall conversion rate from screening to informed consent stage across all clinical settings was 12%. It was very low (3.3%) in the acute settings but greater in the elective settings at 17.7%. Progression in study pathway significantly was greater in Oxford (OR −2.06; p = 0.001), less in Leeds (OR −0.37; p < 0.001) and greater in elective settings including haematology-oncology (OR −2.20; p < 0.002) and presurgical (OR −3.55; p < 0.001). Male patients (OR −1.36; p = 0.02), those at Oxford (OR −1.73; p = 0.002), haematology-oncology (OR −3.30; p < 0.001) and presurgical (OR −5.51; p < 0.001) were significantly more likely and those ≥ 80 years (OR −0.23; p = 0.001) significantly less likely to consent. There were 102, 77 and 91 consented patients from Birmingham [age median 60.00 interquartile range (51.50–69.75) years], Oxford [58.00 (46.00–68.25) years] and Leeds [61.00 (48.00–73.00) years], respectively. Of 270 consented patients, 259 (81.2%) were subsequently risk stratified, with 155 (60%) stratified as ‘low risk’ and 104 (40%) as ‘high risk’. Of 155 ‘low risk’ patients, 126 underwent DPC and 122 (97%) of these patients were successfully de-labelled. There were no serious immediate or delayed HSRs among any patient who underwent DPC. Workstream 2 Most of the ‘low risk’ patients we interviewed accepted the validity of their PAL. They were receptive to having this reviewed, for multiple reasons including potential individual benefits as well as the contribution to wider society. Prior to involvement in the study, they reported having little or no understanding of the adverse impact of PAL on their health, although some noted that they had to receive multiple antibiotic courses as a consequence. Some HCPs perceived potential risks to individuals undergoing DPC as well as to their own professional practice and accountability and were typically more risk averse than patients. A key theme for HCPs was around the level of training required to undertake risk stratification and DPC. The range of background knowledge around allergy testing varied substantially between sites. In general, pharmacists were more familiar with taking an allergy history and prescribing medication, based on interaction with allergy services in their Trusts. The research nurses came from a different clinical background and had very little prior knowledge around allergy. This differential training may have been reflected in the greater confidence of pharmacists to risk stratify and de-label patients. Infrastructure, clinical skill, dedicated space and equipment, a clear governance framework and appropriate timing of the intervention within a care pathway, were considered essential for routine use of DPC. Workstream 3 Total costs of the full DPC pathway averaged across the three study sites, from screening to de-labelling, were £940 per de-labelled patient. In contrast, the costs of performing DPC alone, ignoring the costs required to identify and consent patients to undergo testing for their PAL, varied between £98 and £288 per patient undergoing DPC across study sites. The costs of delivering DPC amounted to 50 times the size of the potential cost savings from switching to penicillin from second-line antibiotic medications and reduced length of hospital stay during the index admission. Subgroup analysis revealed that the full cost of DPC per de-labelled patient varied between £502 (95% CrI: 269 to 1438) in haematology-oncology at Leeds to £1829 (1115 to 4943) in AMU/IDU in Birmingham and £2329 (947 to 19,504) in AMU/IDU in Leeds. The costs of DPC were sensitive to the rate of eligible patients who were approached at Leeds but not at Oxford or Birmingham. The EVPI was highest in Oxford, where the value of further research per de-labelled patient varied in the range of £20–25 across clinical settings, whereas in Leeds it varied between £100 per de-labelled patient in haematology-oncology, £6 in the presurgical group and £0 in AMU/IDU while in Birmingham it was £0 in all three patient groups. Discussion with local managers in Leeds revealed that increasing the proportion of presurgical patients who receive DPC in time to benefit from optimal prophylactic management with penicillin may require investing in an additional HCP role to help identify eligible patients at their initial outpatient assessment appointment. At Birmingham, further research on the role that DPC may play in relieving the pressure on specialist allergy testing services was highlighted. At Oxford, identifying service configurations in which the service may pay for itself from a provider perspective was perceived as key to the economic case for adoption. Conclusions This is the first multicentre UK study to demonstrate safety and feasibility of delivering DPCs in ‘low risk’ patients by non-allergy specialist HCPs in a safe clinical environment in secondary care. Implications for service There was a very high uptake of DPC amongst those ‘low risk’ patients who consented to take part, and the majority of these were successfully de-labelled. There were no serious type-I or type-IV HSRs. A very large proportion of patients screened were deemed ineligible due to clinical instability, comorbidity, or being uncontactable. ‘Opportunistic de-labelling’ in outpatient settings may offer greater opportunities than inpatient de-labelling. Patients with a PAL were keen to have their record reviewed and be considered for de-labelling employing DPC. ‘Low risk’ patients declined DPC mainly due to personal circumstances. There is a need to raise awareness of the adverse impact of PAL amongst patients. While HCPs recognised complexities of the DPC, data also support the feasibility of delivery by non-allergy specialists with appropriate resources, training, facilities, local policies and governance framework and escalation or referral pathways to allergy specialists. The full costs of delivering DPC are significantly higher than the costs of performing the oral challenge test itself alone, which requires careful consideration by providers as to which delivery model is most efficient given their local case mix, service configuration and information technology service infrastructure. The study highlights the importance of a flexible or staged approach to adoption of the DPC intervention, so that it is properly embedded into organisational cultures and systems. Future research Value of information analysis suggests that a randomised controlled trial with long-term follow-up for up to 5 years is needed to determine the cost-effectiveness of DPC by capturing relevant data relating to GP consultations for infections, hospital admission rates and duration, AMR, hospital-acquired infections and mortality. Future studies with opportunistic de-labelling may offer better value for money by designing the intervention as a non-allergy specialist pharmacist-led DPC along the lines of the model implemented at the Oxford study site. Study registration This study is registered as ISRCTN55524365. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: NIHR129069) and is published in full in Health and Social Care Delivery Research; Vol. 13, No. 9. See the NIHR Funding and Awards website for further award information. |
| format | Article |
| id | doaj-art-22c2bdc4ae664467b72291234ac3f52f |
| institution | DOAJ |
| issn | 2755-0079 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | NIHR Journals Library |
| record_format | Article |
| series | Health and Social Care Delivery Research |
| spelling | doaj-art-22c2bdc4ae664467b72291234ac3f52f2025-08-20T03:10:35ZengNIHR Journals LibraryHealth and Social Care Delivery Research2755-00792025-04-01130910.3310/MTYW6557NIHR129069Direct oral penicillin challenge in secondary care with low-risk patients: the SPACE mixed-methods study with cost-effectiveness analysisMamidipudi Thirumala Krishna0Yogini H Jani1Iestyn Williams2Ruben Mujica-Mota3Rebecca Bestwick4Michele Siciliano5Robert Michael West6Rashmeet Bhogal7Bee Yean Ng8Kornelija Kildonaviciute9Rachel Pollard10Nicola Jones11Louise Dunsmure12Mairead McErlean13Neil Powell14Chidanand C Hullur15Ariyur Balaji16Jonathan Sandoe17Amena Warner18Ron Daniels19Caroline Thomas20Siraj A Misbah21Louise Savic22Institute of Immunology and Immunotherapy, University of Birmingham and Department of Allergy and Immunology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UKCentre for Medicines Optimisation Research and Education, University College London Hospitals NHS Foundation Trust and UCL School of Pharmacy, London, UKHealth Services Management Centre, University of Birmingham, Birmingham, UKSchool of Medicine, University of Leeds, Leeds, UKSchool of Medicine, University of Leeds, Leeds, UKSchool of Medicine, University of Leeds, Leeds, UKSchool of Medicine, University of Leeds, Leeds, UKDepartment of Pharmacy, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UKDepartment of Pharmacy, Oxford University Hospitals NHS Foundation Trust, Oxford, UKDepartment of Pharmacy, Oxford University Hospitals NHS Foundation Trust, Oxford, UKDepartment of Anaesthesia, Oxford University Hospitals NHS Foundation Trust, Oxford, UKDepartment of Infectious Diseases, Oxford University Hospitals NHS Foundation Trust, Oxford, UKDepartment of Pharmacy, Oxford University Hospitals NHS Foundation Trust, Oxford, UKCentre for Medicines Optimisation Research and Education, University College London Hospitals NHS Foundation Trust and UCL School of Pharmacy, London, UKDepartment of Pharmacy, Royal Cornwall Hospitals NHS Trust, Truro, UKDepartment of Anaesthesia, Birmingham Heartlands Hospital, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UKAcute Medicine Unit, Birmingham Heartlands Hospital, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UKSchool of Medicine, University of Leeds, Leeds, UKAllergy UK, Crayford, UKThe UK Sepsis Trust, Walsall, UKDepartment of Anaesthesia, St James’ University Hospital, Leeds Teaching Hospitals NHS Trust, Leeds, UKDepartment of Clinical Immunology, Oxford University Hospitals NHS Foundation Trust, Oxford, UKDepartment of Anaesthesia, St James’ University Hospital, Leeds Teaching Hospitals NHS Trust, Leeds, UKSome text in this abstract has been adapted from Krishna MT, Bhogal R, Ng BY, Kildonaviciute K, Jani YH, Williams I, et al. A multicentre observational study to investigate feasibility of a direct oral penicillin challenge in de-labelling ‘low risk’ patients with penicillin allergy by non-allergy healthcare professionals (SPACE study): Implications for healthcare systems. J Infect 2024;88:106116. https://doi.org/10.1016/j.jinf.2024.01.015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) license. See: https://creativecommons.org/licenses/by-nc-nd/4.0/. The text below includes minor additions and formatting changes to the original text. Background One in five inpatients carries a penicillin allergy label. However, 90–95% of labels are incorrect. Penicillin allergy labels lead to increased risk for serious hospital infections and longer hospital stay and are associated with higher estimated healthcare costs. Penicillin allergy testing is onerous and requires a specialist. Routine inpatient testing is not available. Recent evidence suggests that a direct oral penicillin challenge delivered by non-allergy specialists is safe in ‘low risk’ patients, who are highly unlikely to be allergic based on history. Aims To explore behaviour, attitudes and acceptability of patients, healthcare professionals and managers regarding a direct oral penicillin challenge in ‘low risk’ patients. To inform development of an implementation framework and determine potential cost-effectiveness. Methods This study (1 May 2021–30 April 2023) involved delivery of direct oral penicillin challenge by non-allergy specialists across three clinical settings (medical/infectious diseases wards, presurgical and haematology-oncology units) at three hospitals. The study had three workstreams: Workstream 1: Screening for potential suitability. Patients were stratified into ‘low risk’ and ‘high risk’. ‘Low-risk’ patients underwent direct oral penicillin challenge. Workstream 2: One-to-one semistructured interviews with patients (N = 43) and focus group (N = 28) discussions with stakeholders. Workstream 3: Care pathway mapping, decision-analytic modelling and value of information analysis were carried out to determine potential cost-effectiveness of direct oral penicillin challenge. Results One thousand and fifty-four of 2257 screened patients were eligible, 270 of 643 approached patients consented (42%). Two hundred and fifty-nine patients were risk-stratified (155 ‘low risk’; 104 ‘high risk’). Of the 155 ‘low risk’ patients, 126 underwent direct oral penicillin challenge, 122 (97%) were de-labelled with no serious allergic reactions and 43 patients were interviewed. Low-risk patients accepted their allergy labels, had limited knowledge of the adverse impact and most were keen to have their labels reviewed. Healthcare professionals demonstrated a risk-averse approach, although would engage in the intervention with training, resource availability and a governance framework in place. The total costs of the direct oral penicillin challenge pathway were higher than the costs of direct oral penicillin challenge alone (£940 vs. £98–288 per patient). There were minimal expected savings in antibiotic and hospital costs in the short term and potentially large healthcare cost savings over 5 years. Limitations Relatively small sample size for direct oral penicillin challenge, poor conversion rate, particularly in acute settings, patients with limited English language proficiency could not be included and the study was not sufficiently powered and controlled to conduct a cost-effectiveness evaluation. Conclusions This first multicentre United Kingdom study showed that non-allergy specialist-led direct oral penicillin challenge is feasible in secondary care. A high proportion of direct oral penicillin challenges were successful, with positive feedback from patients. Majority of screened patients did not progress through the study pathway. Going forward, a multipronged approach is needed to enhance equitability of direct oral penicillin challenge in routine practice. Follow-up mechanisms to consider the intervention during a clinically stable state and a governance framework for those lacking capacity to consent are needed. The cost of delivering a direct oral penicillin challenge pathway in its entirety is significantly higher than the costs of performing direct oral penicillin challenge per se. Future work A randomised controlled trial with long-term follow-up is needed to determine the cost-effectiveness of direct oral penicillin challenge. Study registration This study is registered as ISRCTN55524365. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: NIHR129069) and is published in full in Health and Social Care Delivery Research; Vol. 13, No. 9. See the NIHR Funding and Awards website for further award information. Plain language summary Many patients who experience minor side effects when taking penicillin are wrongly labelled as ‘allergic’, leading to huge numbers of people with incorrect penicillin allergy labels in hospitals. Patients with an allergy label receive different antibiotics which might be toxic and less effective. Allergy tests are not routinely available due to very few specialists in the National Health Service. We developed a simple way to remove incorrect labels in patients at low risk of genuine allergy. This involves taking a careful history from the patient to decide if they are at low risk of having a genuine allergy to penicillin. Then, if suitable, they are offered a test dose or a challenge test (penicillin capsule to be swallowed without doing other allergy tests) in a hospital after taking prior informed consent. In this study, the test was performed by trained healthcare professionals with no specialist background in allergy. We included patients admitted in hospital wards and outpatients from surgery and cancer units in three hospitals. We did 126 challenge tests and showed that almost all (97%) did not have an allergy. One patient developed stomach upset and three had mild rash. Very few patients in the wards were eligible to participate for medical reasons, and a greater percentage of outpatients were suitable. We spoke to patients and healthcare professionals about this pathway. Professionals understood its importance but thought appropriate training, safety and resources were required. Patients were keen to find out if they were really allergic and felt safe undergoing the challenge test in hospital. The main reasons for not taking part were personal circumstances. We showed that the penicillin challenge test can be delivered by non-allergy healthcare professionals, but a smaller percentage of patients were eligible to undergo this test than that shown in previous studies. We developed recommendations for wider roll-out of this pathway and gained good understanding of how it would work in the ‘real world’. The cost saving appears small in the short term and more research is needed to understand the long-term benefits. Scientific summary Some text in this scientific summary has been republished with permission from Krishna MT, Bhogal R, Ng BY, Kildonaviciute K, Jani YH, Williams I, et al. A multicentre observational study to investigate feasibility of a direct oral penicillin challenge in de-labelling ‘low risk’ patients with penicillin allergy by non-allergy healthcare professionals (SPACE study): Implications for healthcare systems. J Infect 2024;88:106116. https://doi.org/10.1016/j.jinf.2024.01.015. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) license. See: https://creativecommons.org/licenses/by-nc-nd/4.0/. The text below includes minor additions and formatting changes to the original text. Background Penicillin allergy labels (PALs) occur in 6% of the population in England, and 15–20% of inpatients have a PAL in their records. Previous studies have shown that 90–95% of PALs are inaccurate. The current standard of care involves allergy specialist review, skin tests and a challenge procedure. There is an unmet need for allergy services and penicillin allergy testing is not routinely available. Penicillin allergy labels lead to use of alternative antibiotics which might be less effective, might enhance risk of antimicrobial resistance (AMR) and hospital-acquired infections, contribute to longer hospital stay, higher re-admission rates and mortality. Mis-labelling patients has been linked to suboptimal skills and knowledge gaps in allergy history taking and poor documentation in clinical records. Patients can be stratified into ‘low risk’ and ‘high risk’ based on a structured clinical history and scrutiny of clinical and prescription records. ‘Low risk’ patients are highly unlikely to have an immune-mediated reaction to penicillin. A direct oral penicillin challenge (DPC; without undertaking skin tests) has been shown to be safe and feasible and successfully delivered by non-allergy healthcare professionals (HCPs), such as clinical pharmacists and physicians. There is limited evidence from the UK, but there is a larger body of evidence from the USA and Australia regarding the success of DPC in mitigating the adverse impact of PALs. Safe implementation depends on an appropriate clinical setting and uptake amongst patients and relevant stakeholders. Objectives Primary To explore behaviour, attitudes and acceptability of patients, HCPs and managers regarding use of DPC in ‘low risk’ patients. To develop treatment pathways and a clinical governance framework for this service model. Secondary To assess the proportion of ‘low risk’ patients with a PAL who would be eligible for a DPC. To assess the proportion of ‘low risk’ patients who would be willing and will complete a DPC. To explore practical aspects of implementing this de-labelling programme in secondary care by investigating factors, such as organisational context, treatment pathway, protocol implementation, time taken and resources. To evaluate the potential cost-effectiveness of this service model. Methods This study spanned 24 months and involved three clinical settings [acute medical units (AMUs)/infectious diseases (ID) units, presurgical units and haematology-oncology units] in three hospitals including University Hospitals Birmingham NHS Foundation Trust, Leeds Teaching Hospitals and Oxford University Hospitals NHS Foundation Trust. The study included three workstreams (WS): Workstream 1 Patients with a PAL were identified and screened from respective Trust clinical records using a structured proforma employing predetermined study criteria. Clinically unstable patients, pregnant or breastfeeding patients, those with current COVID-19 infection, patients lacking capacity to give informed consent or taking part in another research intervention study were not approached. Those potentially suitable were approached for expression of interest. Informed consent was obtained from interested participants and a risk stratification process was conducted based on pre-determined study criteria. Participants were classified as ‘low risk’ and ‘high risk’. Risk stratification was conducted by non-allergy specialist HCPs (Research Pharmacists at Birmingham and Oxford and Research Nurses at Leeds) and a non-allergy specialist study consultant provided clinical supervision for the entire process. ‘Low risk’ patients were those who were highly unlikely to have a true allergy and ‘high risk’ patients gave a history suggestive of an immune-mediated reaction and/or had concomitant comorbidity, such as severe asthma or cardiac disorder. ‘Low risk’ patients were offered a DPC. This involved administration of a single dose of 500 mg amoxicillin and 1-hour observation to monitor for an immediate hypersensitivity reaction (HSR) followed by 250 mg twice daily for 3 days to assess for a delayed HSR. All patients were reviewed on day 5 to check for their clinical tolerance. One hundred and twenty-six DPCs were conducted [119 ‘opportunistic’ and 7 ‘therapeutic’ (to treat current bacterial infection)]. Data analysis Descriptive data were generated for all clinical settings across the three sites. To aid in the detection of potential associations, continuous variables were compared between the hospital/clinical setting groups using Kruskal–Wallis tests. Similarly, categorical variables were compared across hospital/clinical settings groups using Pearson chi-squared tests. Detailed analysis was provided to compare outcomes, such as de-labelled ‘Yes/No’, with the use of logistic regression. First, the outcome was regressed upon each potential ‘risk factor’, such as gender, producing an unadjusted odds ratio (OR; by exponentiation of the fitted parameter), and then a multivariable logistic regression used to yield adjusted ORs. Workstream 2 Patients ‘Low risk’ patients were invited to participate in the qualitative arm of the study at the time of recruitment to WS1. Purposive sampling was undertaken to achieve a diverse sample. One-to-one semistructured interviews were conducted with patients {total N = 43 [mean age 61 (± 14 standard deviation) years]} by telephone using a pre-specified interview schedule. The interview questions were informed by risk perception theories and developed iteratively with our patient and public partners to ensure face validity. Healthcare professionals and other stakeholders Three focus group discussions were conducted with 28 participants in total across the 3 centres. We purposively sampled staff to include representation from relevant stakeholders [including general practitioners (GPs), junior doctors, consultants, nurses, pharmacists, managers, commissioners] at each site. Focus groups were held in person (two sites) and online (one site). The discussions were audio-recorded and written notes were taken. Two members of the research team facilitated the discussions using a pre-specified topic guide to prompt healthy discussion, informed by relevant domains of the ‘Theoretical Domains Framework’. Audio recordings of interviews and focus groups were transcribed verbatim and coded. Thematic analysis was undertaken using both inductive and deductive approaches and was informed by ‘Theoretical Domains Framework’. Workstream 3 An economic evaluation of DPC relative to current standard of care, that is no PAL testing, was carried out in ‘low risk’ study participants. We estimated the costs of the DPC pathway at each study site and its expected impact on the costs of antibiotic regimens in the period immediately after DPC, for AMU/IDU and presurgical patients, or up to one course of chemotherapy after DPC in haematology-oncology patients. We combined prospectively collected individual patient data on the time that clinical and non-clinical staff spent in screening and risk stratification and retrospectively, collected data on staff time inputs into all DPC steps, to value time inputs according to the midpoint of the salary scales for the title and grade of staff and derived the costs of the full DPC pathway. We used Monte Carlo simulation methods to produce estimates of sampling uncertainty of costs in the form of 95% credible intervals (95% CrI). We also estimated the value of conducting further research to capture the economic impact of DPC in terms of hospital re-admissions, length of hospital stays of initial and subsequent admissions, and GP visits over a 4.5-year follow-up period, the maximum over which evidence was found in the literature. This was a value of information analysis with the expected value of perfect information (EVPI) as outcome, conducted separately for each new service pathway implemented across the three study sites. In sensitivity analysis, we explored the impact of increasing the proportion of eligible patients who are approached for consent to undergo testing, accounting for the costs of training and varying the staff skill mix to allow for the possibility that less experienced staff may deliver the new testing pathway. In addition, we held discussions with local managers at each study site to identify key questions emerging from our findings and areas of uncertainty that would need to be addressed by further studies. Patient and public involvement and engagement The investigators worked in collaboration with representatives from Allergy UK and the UK Sepsis Trust and two groups of members of the public in Oxford and Birmingham. Their input was sought for the study protocol, study amendments, participant recruitment strategies, patient facing documents, data interpretation and recommendations for further research. Results Workstream 1 A total of 2257 patients (834 males and 1423 female) were screened across the 3 clinical settings and the 3 participating sites. One thousand two hundred and three (53.3%) were deemed ineligible based on study criteria at screening. A total of 1054 (46.6%) patients were considered eligible, 643 were approached and 412 not approached due to practical factors. Of 643 patients who were approached, 373 declined to participate and 270 (116 males and 154 female) consented. The overall conversion rate from screening to informed consent stage across all clinical settings was 12%. It was very low (3.3%) in the acute settings but greater in the elective settings at 17.7%. Progression in study pathway significantly was greater in Oxford (OR −2.06; p = 0.001), less in Leeds (OR −0.37; p < 0.001) and greater in elective settings including haematology-oncology (OR −2.20; p < 0.002) and presurgical (OR −3.55; p < 0.001). Male patients (OR −1.36; p = 0.02), those at Oxford (OR −1.73; p = 0.002), haematology-oncology (OR −3.30; p < 0.001) and presurgical (OR −5.51; p < 0.001) were significantly more likely and those ≥ 80 years (OR −0.23; p = 0.001) significantly less likely to consent. There were 102, 77 and 91 consented patients from Birmingham [age median 60.00 interquartile range (51.50–69.75) years], Oxford [58.00 (46.00–68.25) years] and Leeds [61.00 (48.00–73.00) years], respectively. Of 270 consented patients, 259 (81.2%) were subsequently risk stratified, with 155 (60%) stratified as ‘low risk’ and 104 (40%) as ‘high risk’. Of 155 ‘low risk’ patients, 126 underwent DPC and 122 (97%) of these patients were successfully de-labelled. There were no serious immediate or delayed HSRs among any patient who underwent DPC. Workstream 2 Most of the ‘low risk’ patients we interviewed accepted the validity of their PAL. They were receptive to having this reviewed, for multiple reasons including potential individual benefits as well as the contribution to wider society. Prior to involvement in the study, they reported having little or no understanding of the adverse impact of PAL on their health, although some noted that they had to receive multiple antibiotic courses as a consequence. Some HCPs perceived potential risks to individuals undergoing DPC as well as to their own professional practice and accountability and were typically more risk averse than patients. A key theme for HCPs was around the level of training required to undertake risk stratification and DPC. The range of background knowledge around allergy testing varied substantially between sites. In general, pharmacists were more familiar with taking an allergy history and prescribing medication, based on interaction with allergy services in their Trusts. The research nurses came from a different clinical background and had very little prior knowledge around allergy. This differential training may have been reflected in the greater confidence of pharmacists to risk stratify and de-label patients. Infrastructure, clinical skill, dedicated space and equipment, a clear governance framework and appropriate timing of the intervention within a care pathway, were considered essential for routine use of DPC. Workstream 3 Total costs of the full DPC pathway averaged across the three study sites, from screening to de-labelling, were £940 per de-labelled patient. In contrast, the costs of performing DPC alone, ignoring the costs required to identify and consent patients to undergo testing for their PAL, varied between £98 and £288 per patient undergoing DPC across study sites. The costs of delivering DPC amounted to 50 times the size of the potential cost savings from switching to penicillin from second-line antibiotic medications and reduced length of hospital stay during the index admission. Subgroup analysis revealed that the full cost of DPC per de-labelled patient varied between £502 (95% CrI: 269 to 1438) in haematology-oncology at Leeds to £1829 (1115 to 4943) in AMU/IDU in Birmingham and £2329 (947 to 19,504) in AMU/IDU in Leeds. The costs of DPC were sensitive to the rate of eligible patients who were approached at Leeds but not at Oxford or Birmingham. The EVPI was highest in Oxford, where the value of further research per de-labelled patient varied in the range of £20–25 across clinical settings, whereas in Leeds it varied between £100 per de-labelled patient in haematology-oncology, £6 in the presurgical group and £0 in AMU/IDU while in Birmingham it was £0 in all three patient groups. Discussion with local managers in Leeds revealed that increasing the proportion of presurgical patients who receive DPC in time to benefit from optimal prophylactic management with penicillin may require investing in an additional HCP role to help identify eligible patients at their initial outpatient assessment appointment. At Birmingham, further research on the role that DPC may play in relieving the pressure on specialist allergy testing services was highlighted. At Oxford, identifying service configurations in which the service may pay for itself from a provider perspective was perceived as key to the economic case for adoption. Conclusions This is the first multicentre UK study to demonstrate safety and feasibility of delivering DPCs in ‘low risk’ patients by non-allergy specialist HCPs in a safe clinical environment in secondary care. Implications for service There was a very high uptake of DPC amongst those ‘low risk’ patients who consented to take part, and the majority of these were successfully de-labelled. There were no serious type-I or type-IV HSRs. A very large proportion of patients screened were deemed ineligible due to clinical instability, comorbidity, or being uncontactable. ‘Opportunistic de-labelling’ in outpatient settings may offer greater opportunities than inpatient de-labelling. Patients with a PAL were keen to have their record reviewed and be considered for de-labelling employing DPC. ‘Low risk’ patients declined DPC mainly due to personal circumstances. There is a need to raise awareness of the adverse impact of PAL amongst patients. While HCPs recognised complexities of the DPC, data also support the feasibility of delivery by non-allergy specialists with appropriate resources, training, facilities, local policies and governance framework and escalation or referral pathways to allergy specialists. The full costs of delivering DPC are significantly higher than the costs of performing the oral challenge test itself alone, which requires careful consideration by providers as to which delivery model is most efficient given their local case mix, service configuration and information technology service infrastructure. The study highlights the importance of a flexible or staged approach to adoption of the DPC intervention, so that it is properly embedded into organisational cultures and systems. Future research Value of information analysis suggests that a randomised controlled trial with long-term follow-up for up to 5 years is needed to determine the cost-effectiveness of DPC by capturing relevant data relating to GP consultations for infections, hospital admission rates and duration, AMR, hospital-acquired infections and mortality. Future studies with opportunistic de-labelling may offer better value for money by designing the intervention as a non-allergy specialist pharmacist-led DPC along the lines of the model implemented at the Oxford study site. Study registration This study is registered as ISRCTN55524365. Funding This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: NIHR129069) and is published in full in Health and Social Care Delivery Research; Vol. 13, No. 9. See the NIHR Funding and Awards website for further award information.https://doi.org/10.3310/MTYW6557penicillin allergyinaccurate penicillin allergy labelspenicillin allergy de-labellingdirect oral penicillin challengeantimicrobial resistance and low riskmixed-methods study |
| spellingShingle | Mamidipudi Thirumala Krishna Yogini H Jani Iestyn Williams Ruben Mujica-Mota Rebecca Bestwick Michele Siciliano Robert Michael West Rashmeet Bhogal Bee Yean Ng Kornelija Kildonaviciute Rachel Pollard Nicola Jones Louise Dunsmure Mairead McErlean Neil Powell Chidanand C Hullur Ariyur Balaji Jonathan Sandoe Amena Warner Ron Daniels Caroline Thomas Siraj A Misbah Louise Savic Direct oral penicillin challenge in secondary care with low-risk patients: the SPACE mixed-methods study with cost-effectiveness analysis Health and Social Care Delivery Research penicillin allergy inaccurate penicillin allergy labels penicillin allergy de-labelling direct oral penicillin challenge antimicrobial resistance and low risk mixed-methods study |
| title | Direct oral penicillin challenge in secondary care with low-risk patients: the SPACE mixed-methods study with cost-effectiveness analysis |
| title_full | Direct oral penicillin challenge in secondary care with low-risk patients: the SPACE mixed-methods study with cost-effectiveness analysis |
| title_fullStr | Direct oral penicillin challenge in secondary care with low-risk patients: the SPACE mixed-methods study with cost-effectiveness analysis |
| title_full_unstemmed | Direct oral penicillin challenge in secondary care with low-risk patients: the SPACE mixed-methods study with cost-effectiveness analysis |
| title_short | Direct oral penicillin challenge in secondary care with low-risk patients: the SPACE mixed-methods study with cost-effectiveness analysis |
| title_sort | direct oral penicillin challenge in secondary care with low risk patients the space mixed methods study with cost effectiveness analysis |
| topic | penicillin allergy inaccurate penicillin allergy labels penicillin allergy de-labelling direct oral penicillin challenge antimicrobial resistance and low risk mixed-methods study |
| url | https://doi.org/10.3310/MTYW6557 |
| work_keys_str_mv | AT mamidipudithirumalakrishna directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT yoginihjani directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT iestynwilliams directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT rubenmujicamota directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT rebeccabestwick directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT michelesiciliano directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT robertmichaelwest directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT rashmeetbhogal directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT beeyeanng directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT kornelijakildonaviciute directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT rachelpollard directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT nicolajones directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT louisedunsmure directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT maireadmcerlean directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT neilpowell directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT chidanandchullur directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT ariyurbalaji directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT jonathansandoe directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT amenawarner directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT rondaniels directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT carolinethomas directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT sirajamisbah directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis AT louisesavic directoralpenicillinchallengeinsecondarycarewithlowriskpatientsthespacemixedmethodsstudywithcosteffectivenessanalysis |