Introduction: Opioid overdose is a preventable cause of death in the United States and timely administration of naloxone, a reversal agent, can be a life-saving intervention. Brief curricular intervention has been shown to improve naloxone prescription rates, and demonstrate improved knowledge among internal medicine residents.

Methods: Our study examined the difference in knowledge before and after a 45-minute trivia-based, competitive educational program and assessment on opioid overdose and naloxone distribution (OEND) practices for eight, specialty-specific didactic sessions. The study’s primary outcome was the frequency of resident-issued naloxone prescriptions, measured for two weeks prior to the first training, for two month-long periods concurrent with the trainings and for two weeks following resident trainings. We hypothesized that by providing innovative OEND instruction, naloxone prescription rates and resident knowledge of OEND practices would improve.

Results: In six of the seven survey pre- and post- intervention survey questions, residents self-reported statistically significant improvement in knowledge and practice confidence as measured by change in mean response score. Comparison of rate of naloxone prescriptions prior to, during and following curricular implementation did not demonstrate significant change as measured by Chi-square testing.

Conclusions: While innovative resident OEND training did correspond with improved resident self-reported knowledge, it did not correspond with a significant change in resident behavior as measured by naloxone prescription rates

• Naloxone

• Risk-reduction

• Resident education

• Opioid overdose

McElhinny ML, Christensen CR, Hockley MM, Bhattarai B, Quan D, Geren K (2022) Effect of Innovative Opioid Risk-Reduction Curriculum for Resident Physicians on the Frequency of Resident Naloxone Prescription. J Subst Abuse Alcohol 9(1): 1097

OUD: Opioid Use Disorder; OEND: Overdose Education and Naloxone Distribution; ED: Emergency Department; ACGME: Accreditation Council for Graduate Medical Education; HER: Electronic Health Record

The Centers for Disease Control (CDC) estimates that 42,240 deaths in 2016 were a result of opioid overdose, with only 20% of patients with opioid use disorder receiving treatment [1]. Between April 2020 and April 2021, the CDC estimates a total of 75,673 deaths secondary to opioid overdose, representing a 35% increase in opioid overdose deaths as compared the prior year [2]. Efforts to address the opioid epidemic at a large, urban, safety-net hospital in Phoenix have been slow to take hold, including a pilot naloxone distribution program, the first emergency department (ED) distribution program in the state of Arizona. Our ED naloxone distribution program evaluation suggested poor provider recognition of opioid use disorder (OUD). Challenges include provider hesitation and confusion about counseling patients on OUD risk-reduction as well as shortcomings in providing of naloxone prescriptions [3].

Growing literature suggests that physicians are willing to provide their patients with actionable information to reduce use of opioids and increase distribution of naloxone, although they face several logistical challenges [4-6]. Resident physicians of multiple disciplines were considered ideal targets for additional opioid overdose education and naloxone distribution (OEND) education to drive practice change within our institution. In a 2016 study, internal medicine residents were willing to prescribe naloxone (90%) although few (15%) had prescribed it [7]. The most common barriers to providing a prescription included lack of knowledge on how to identify patients at-risk for opioid use disorder and how to prescribe naloxone [7]. Naloxone education has been shown in interprofessional didactic sessions among medical residents to improve knowledge, attitude and selfefficacy in pre- and post- training evaluations [8]. While improved knowledge and efficacy should drive practice improvement and clinical outcomes, there is no evidence to suggest these trainings lead to lasting practice change.

The purpose of this study was to develop an innovative resident curriculum to address gaps in knowledge and selfefficacy in delivering opioid OEND practices to patients. This novel educational intervention was designed to engage residents in long-lasting practice change and improve the frequency of naloxone prescriptions provided to patients. We used a prospective intervention study design to measure the change in the number of naloxone prescriptions issued by resident physicians before, during and after the training intervention. The primary outcome of this study was a comparison of prescription frequencies before, during, and after the opioid OEND curriculum was presented to resident physicians. Our secondary outcome was a comparison of mean change in self-reported resident knowledge and self-efficacy in providing opioid OEND counseling to patients.

This investigation was conducted in a public, urban, safety-net healthcare system that serves a large, under-insured and diverse community. Program directors from each of the nine physician residency programs within the hospital system were solicited for didactic time with the residents to present a specialty-specific, competitive, trivia-based educational session on opioid OEND. The resident physicians of the eight specialty-specific programs that completed the curriculum also completed pre- and postintervention assessments. The frequency of resident-issued naloxone prescriptions was measured for two weeks prior to the first training, for two month-long periods concurrent with the trainings and for two weeks following the trainings. An electronic health record (EHR) query was used to search and collate all naloxone prescriptions written by resident physicians of participating residencies during each of these periods. Residents were surveyed prior to and following the training regarding their self-assessed knowledge and self-efficacy of OEND practices. Resident physicians were excluded from the study if their program leadership declined presentation of the curriculum, if they were not in attendance during training offerings or if they did not consent to inclusion in the study. Study protocol was reviewed and approved by the Valleywise Institutional Review Board prior to curricular implementation.

The curriculum presented to the resident physicians consisted of two parts-an educational presentation and a test of knowledge. First the participants were introduced to topics in OEND through a competitive, team-based trivia-style presentation that lasted approximately 15-30 minutes. Trivia presentations included three standard categories including identifying at-risk populations, counseling patients and naloxone prescription. Specialty-specific categories, including pediatric, obstetric or surgery topics, were included for respective resident physician groups. The remaining time was allocated to an interactive test of knowledge and knowledge consolidation with use of online Kahoot!® (Kahoot! Oslo, Norway) platform for a total of 30-45 minutes of didactic time.

To allow for equal access to overdose and naloxone training, all eligible resident physicians were offered the intervention training. Immediately before and after the training intervention the resident physicians completed a survey about opioid OEND knowledge and self-efficacy, using a 5-point Likert scale. A reminder email about the availability of naloxone prescriptions was sent to all resident physicians at the outset of the observation period to guard against recency bias.

Each resident served as their matched control for preand post- survey comparison. After matching pre- and postresponses, identifiers were removed to protect resident physician privacy. Improvement in scores after the intervention were compared using Wilcoxon signed rank test. Association between the frequency of naloxone prescription across the three intervention periods were compared using chi-square testing.

The number of naloxone prescriptions written by resident physicians were identified and collated by research associates within the electronic medical record and then blinded to the study team prior to comparison between intervention periods. The frequency of naloxone prescriptions by residents were compared across three periods: prior to, during and following the training intervention. The training intervention for each residency specialty subgroups was scheduled at random. Resident prescription observation periods were chosen to best characterize the most short-lived behavior change in naloxone prescribing after curricular implementation. Ebbinghaus’ ‘forgetting curve’ suggests that in approximately one month, only 20% of material imparted at a single point in time will be retained. Therefore, intervals on the similar timescale were targeted in measuring prescription rates following our single curricular intervention [9].

Resident physicians work throughout our hospital system. It was not possible to extract the specific number of patient encounters for resident physicians who participated in the intervention. As a result, the number of Emergency Department (ED) encounters was used as a metric for overall patient volume, and therefor encounters, throughout the hospital system.

Study participants were representative of eight residency physician groups including emergency medicine (n=20), internal medicine (n=9), obstetrics/gynecology (n=9), pediatric (n=77), podiatry (n=8), pediatric dentistry (n=7), and surgery (n=14) from an estimated total of 175 residents within our health system. Of the 156 resident physician respondents, 144 indicated a specialty. Although resident physician prescriptions were not collated by specialty, pediatric resident physicians where overwhelming represented in our sample. Resident physicians estimated how often they worked with patients who struggle with opioid use. Of the 137 responses, 18 (13.1%) stated multiple (3+) times a day, 11 (8.0%) stated once or twice a day, 13 (9.5%) stated multiple (3+) times weekly, 15 (10.9%) stated once or twice a week, and 70 (51.1%) stated once or twice monthly or less frequently with 10 (7.3%) indicating multiple frequencies. Results summarized in Table 1 compare pre- and post- training rating of knowledge and confidence in providing opioid OEND practices. Residents self-reported improvement in knowledge and confidence between pre- and post- survey was measured by change in mean response score. Six of the seven survey questions listed demonstrated statistically significant change between pre- and post- survey by chi-squared testing with corresponding p-values < 0.001.

Table 1: Comparison summary of pre- and post-survey response scores.

In Table 2, the frequency of naloxone prescriptions is shown 15 days prior to curricular implementation, during the curricular implementation (4 trainings between April 1-30 and 4 trainings between May 9-June 14) and 15 days following curricular implementation.

Table 2: Frequency of naloxone prescriptions across study periods, standardized against the respective number of ED encounters

During these time periods the frequency of naloxone prescriptions did not improve and a lower frequency of naloxone prescriptions was observed from May15-June 14 while the curricula was still being implemented. Program evaluation of a previous project in our ED indicated poor provider recognition of opioid use disorder (OUD) and both provider hesitation and confusion about counseling patients on risk reduction in OUD such as naloxone prescription [3]. These shortcomings provided a spring board for these novel opioid OEND curricula. Illuminating which educational practices can translate training into practice change is important and valuable for improving patient care. Previously published resident curricula focusing on opioid OEND by Monteriro et al. [10] Ruff et al. [11] and Wakeman et al. [12] chose heavily didactic-style designs, some including panel discussions or small group discussion. In addition, Alford et al. [13] designed OEND curricula featuring the Objective Structured Clinical Exam (OSCE). Alford’s discussion of his curricular design highlighted the power of collaborative, applied learning similar to that found with our competitive, trivia-based approach [13]. Trainings that involved standardize patients were often described, but also reported to have been included at significant cost and coordination. Internal medicine resident curriculum has been demonstrated to improve provider self-assessed skills, and Taylor et al. [14] demonstrated patient-centered improvement with an increase in resident naloxone prescription frequencies after two, one- hour didactic sessions addressing opioid OEND. Cooper et al. [15] reviewed opioid use disorder curriculums aimed at both medical students and residents and found a range of content from harm reduction to chronic pain management but our innovative curriculum was the first interventional study to feature interdisciplinary trainings.

Our curriculum also applied gamification, or application of game playing elements, to improve resident education engagement and retainment of knowledge and skills, as a new and innovative approach which has previously demonstrated in ultrasound literature [16].

While this competitive, trivia-based curriculum does not offer evidence to suggest it can motivate practice change, it demonstrated significant improvement in resident knowledge and confidence. Use of interactive quiz software improves the learner’s application and retainment of OUD risk-reduction practices and represents a low-cost, easily reproducible, and time-savvy way to deliver opioid OEND education. Improving logistics of naloxone prescription and distribution processes may serve long-lasting change and should be studied in closer detail. While curricular development is an important part of behavior change, additional optimization of practice conditions may be impactful.

While it was beneficial to spread the OEND curriculum to resident physicians of varying specialties, it did make for numerous limitations. First, the resident physician groups were not equally represented. Pediatric residents comprised 53% of the resident physicians who identified their specialty. Given naloxone prescribing patterns may be different for children than adults, the practice patterns of pediatric resident physicians are over represented in this study. The abundance of pediatric residents may also explain why 62% of the respondents indicated they only see patients who struggle with opioid use disorder “3 or less times per week.” Alternatively, this may also indicate that resident physicians underestimate the number of patients they see who struggle of opiate use disorder. Second, resident physicians rotate at multiple clinical sites and at varying intervals affecting data collection of naloxone prescription numbers. Frequency of emergency department encounters was used as an index of overall health system volume varying by month. While this is suboptimal in providing sub-specialty detail on prescription rates, it was the available measure approximating volume for residents who rotate among various clinical settings and specialties during their training.

Our sample size did not provide adequate number of participants for rigorous sub-group analysis by specialty. With sub-group analysis we might have expected certain specialties (such as emergency medicine, surgery and internal medicine) to have higher naloxone prescriptions frequencies.

Our results demonstrate resident education using a novel training approach did improve self-reported knowledge about OUD and opioid OEND. It did not significantly increase resident physician naloxone prescribing frequency. Opioid OEND training in graduate medical education (GME) could be studied more robustly with a larger sample size that is more representative of the general resident physician population and over a longer period of time. This study may serve as justification for further investigation of resource allotment to a systems-based approach including electronic medical record design, advocacy for improved regulations for distribution of naloxone or other efforts, as education alone did not improve naloxone prescription frequencies.

This study was generously funded by a joint grant from the Emergency Medicine Foundation (EMF) and the National Institute of Drug Abuse (NIDA) as well as the Maricopa Medical Foundation (MMF).

1. Taylor JL, Rapoport AB, Rowley CF, Mukamal KJ, Stead W. An opioid overdose curriculum for medical residents: Impact on naloxone prescribing, knowledge, and attitudes. Subst Abus. 2018; 39:371-376.

2. McCance-Katz E. SAMHSA/HHS: An Update on the Opioid Crisis. 2018.

3. Alyssa M Peckham, Maria E Niculete, Howard Steinberg, Douglas L Boggs. A Survey of Prescribers’ Attitudes, Knowledge, Comfort, and Fear of Consequences Related to an Opioid Overdose Education and Naloxone Distribution Program. J Public Health Manag Pract. 2018; 24:310-317.

4. Cooper RL, Ramesh A, Juarez PD, Edgerton R, Paul M, Tabatabai M, et al. Systematic Review of Opioid Use Disorder Treatment Training for Medical Students and Residents. J Health Care Poor Underserved. 2020; 31:26-42.

5. Alford DP, Carney BL, Brett B, Parish SJ, Jackson AH. Improving Residents’ Safe Opioid Prescribing for Chronic Pain Using an Objective Structured Clinical Examination. J Grad Med Educ. 2016; 8: 390-397.

6. CDC NC for HS. Drug Overdose Deaths in the U.S. Top 100,000 Annually 2021. (accessed January 12, 2022).

7. Ruff AL, Alford DP, Butler R, Isaacson JH. Training internal medicine residents to manage chronic pain and prescription opioid misuse. Subst Abus. 2017; 38: 200-204.

8. Andrew S Liteplo, Kristin Carmody, Matt J Fields, Rachel B Liu, Resa E Lewiss. SonoGames: Effect of an Innovative Competitive Game on the Education, Perception, and Use of Point-of-Care Ultrasound. J Ultrasound Med. 2018; 37: 2491-2496.

9. J Deanna Wilson, Natalie Spicyn, Pamela Matson, Anika Alvanzo, Leonard Feldman. Internal medicine resident knowledge, attitudes, and barriers to naloxone prescription in hospital and clinic settings. Subst Abus. 2016; 37: 480-487.

10.McElhinny M, Chea K, Carter-Powell A, Mishler A, Bhattarai B, Geren K. Adult emergency department naloxone education and prescription program: Video and pamphlet education comparison. J Subst Abuse Treat. 2021; 127.

11.Jermaine D Jones, Perrine Roux, Sharon Stancliff, William Matthews, Sandra D Comer. Brief overdose education can significantly increase accurate recognition of opioid overdose among heroin users. Int J Drug Policy. 2014; 25: 166-170.

12.Wakeman SE, Pham-Kanter G, Baggett MV, Campbell EG. Medicine Resident Preparedness to Diagnose and Treat Substance Use Disorders: Impact of an Enhanced Curriculum. Subst Abus. 2015; 36: 427-433.

13.Monteiro K, Dumenco L, Collins S, Bratberg J, MacDonnell C, Jacobson A, et al. An interprofessional education workshop to develop health professional student opioid misuse knowledge, attitudes, and skills. J Am Pharm Assoc (2003). 2017; 57: S113-117.

14.Ebbinghaus H. Memory; a contribution to experimental psychology. New York city, Teachers college, Columbia university; 1913.

15.Hargraves D, White CC, Mauger MR, Puthota A, Pallerla H, Wigle P, et al.Evaluation of an interprofessional naloxone didactic and skills session with medical residents and physician assistant learners. Pharm Pract (Granada). 2019; 17: 1591. 

16.Eswaran V, Allen KC, Bottari DC, Splawski JA, Bains S, Aks SE, et al. Take-Home Naloxone Program Implementation: Lessons Learned From Seven Chicago-Area Hospitals. Annals of Emergency Medicine. 2020; 76: 318-327.