Background: Pediatric critical care nurses, residents, and physicians receive a variety of resuscitation training, but emergencies are high stakes, low frequency events. Skill and psychomotor decay are threats to effective performance. Simulation training improves teamwork, communication, and efficiency with pediatric resuscitation guidelines, but currently no simulation curriculum exists to provide crisis resource management training to an interprofessional team in the pediatric intensive care unit (PICU).

Methods: An interprofessional team established priorities for learning using a Delphi needs assessment. A training curriculum was designed to include biannual, training sessions in which interprofessional teams worked through 3-5 scenarios. Case difficulty was progressively increased and matched to the level of the team leader. Course efficacy was measured through pre- and post-course surveys of confidence in clinical skill, teamwork, and situational awareness. Teamwork and leadership were assessed by outside observers using a modified Ottawa scale. Clinical learning checklists were completed to assess learning objectives. A 60-day post training survey was sent to participants to assess relevance of training session on clinical practice.

Results: The simulation curriculum demonstrated an improvement in PICU nurses’ and pediatric resident’s skill, efficiency, and confidence in resuscitation events.

Conclusion: A simulation curriculum can improve pediatric critical care teamwork, skill, and efficiency in pre, post and 60 day follow up survey. High fidelity simulation with rapid cycle deliberate practice (RCDP) was shown to improve perceived and actual PICU nursing skills and pediatric resident resuscitation leadership during successive scenarios.

Simulation, Pediatric residents, Crisis resource management, Pediatric critical care nursing, Resuscitation, Rapid cycle deliberate practice

Miller MK, Tennent K, Singleton M, Brenner A, Fussell M (2023) Pediatric Resuscitation and Communication Innovative Core Education (PICU PRACTICE): Development of an Interprofessional Pediatric Crisis Resource Management Curriculum. Ann Pediatr Child Health 2023; 11(3): 1311.

PICU: Pediatric Intensive Care Unit; RCDP: Rapid cycle deliberate practice; CRM: Crisis Resource Management; PPEC: PICU PRACTICE Executive Committee PSTC: Patient Safety Training Center; RN: Registered Nurse CPR: Cardiopulmonary Resuscitation

Adherence to clinical practice guidelines during resuscitation and high acuity clinical events has contributed to improved survival to discharge in pediatric patients [1,2]. Proper training of pediatric physicians and nurses in resuscitation skills is essential to improving the survival of pediatric patients during high-acuity clinical events [1-4]. Yet, the relative infrequency with which such events occur poses a risk for knowledge decay and clinical skill loss [5]. High fidelity simulation and crisis resource management (CRM) training are effective methods for teaching the skills necessary to manage high-acuity patient situations, and are widely used in the emergency department, surgical, obstetrics, and intensive care settings [6-14]. Repeated simulation training has been shown to increase overall competence and protect against deterioration of clinical skills and knowledge over time [15-17].

Interprofessional simulation training has been shown to significantly improve resuscitation skills, teamwork, and confidence among resident physicians [18-20]. The PICU PRACTICE (Pediatric Resuscitation and Communication Training Innovative Core Education) curriculum was developed to address an existing need for resuscitation skills training among pediatric resident physicians and pediatric intensive care unit (PICU) nurses in a low volume, high-acuity PICU in a rural tertiary care center. The study institution experienced a reduction of PICU beds in 2018, which resulted in decreased admissions and reduced exposure of the team to clinical emergencies and resuscitation. An interdisciplinary team composed of physicians, advanced practice providers, clinical nurse leaders, and registered nurses (RNs), later referred to as the PICU PRACTICE Executive Committee (PPEC), identified a need for an educational program designed to address emergency response skill development and maintenance among pediatric resident physicians, pediatric critical care nurses, and other members of the pediatric resuscitation team. It was essential that this curriculum address the learning needs of all learners involved to ensure high quality care delivery during infrequent pediatric clinical emergencies at this institution. Further, interprofessional education is an identified priority in graduate medical education [21]. While interprofessional, high-fidelity simulation training programs exist to improve resuscitation skills and teamwork among adult healthcare providers [22-25], literature is limited on programs designed specifically to address such skills in interprofessional healthcare teams in the PICU.

The current study seeks to describe the development of the PICU PRACTICE curriculum and to address the following research objectives: 

1. To identify learning priorities for a pediatric resuscitation and communication simulation training program as defined by pediatric residents and pediatric ICU and intermediate care nurses.

2. To determine whether the PICU PRACTICE curriculum increases self-reported confidence among pediatric residents and pediatric ICU nurses in managing high risk clinical events.

3. To determine whether the PICU PRACTICE curriculum improves teamwork among interprofessional teams during high-risk clinical events.

4. To determine whether the PICU PRACTICE curriculum improves leadership among pediatric residents leading interprofessional resuscitation teams.

5. To determine whether the PICU PRACTICE curriculum improves pediatric residents and pediatric ICU nurses’ skill in managing high-risk clinical situations as measured by clinical learning checklists and time to key interventions

Training Curriculum Development

The PICU PRACTICE curriculum was designed and implemented by the PPEC. A Delphi needs assessment was conducted with pediatric residents and pediatric ICU nurses. The results were used by PPEC to design 22 different case-based scenarios, with 14 recurrent performance bundles outlining tasks to be addressed during management of each scenario. The skill level required for team leaders to progress through each case was targeted toward that of a second- or third-year pediatric resident. Scenarios were rotated to be offered once every 2 years so that each pediatric resident would train on a new case mix during each session during a 2-year cycle.

Participant and facilitator feedback was encouraged and following each session, the PPEC used Plan-Do-Study-Act (PDSA) quality improvement cycles to improve the curriculum prior to the next training. Modifications made to the curriculum during the research period of March 2020 to October 2022 were primarily small updates to improve the delivery of scenarios, or addition of equipment and supplies to the simulation room. Most notably, the scenario specific checklists used to evaluate learner adherence to performance bundles were altered throughout the research period. Initially, clinical learning checklists were measured entirely by time-to-event measurement. Due to discrepancies in third party raters, the decision was made to transition the measurement of certain elements to “yes/no” outcomes to ensure consistent data collection. A second notable change was the addition of nurses from the institution’s critical care transport team (air and neonatal) when staff availability allowed. This addition, which has been well received in other interprofessional simulation programs [33], made the teams more consistent with those attending code blue events at the study institution.

Course Delivery

PICU PRACTICE simulation sessions were held bi-annually in the Patient Safety Training Center (PSTC) at the study institution. Cases were rotated on a 2-year cycle, so that resident physicians would attend PICU PRACTICE four times over the course of their second and third year of residency. At each session, resident physicians act as team leaders for progressively more complex cases.

Participants for each session were divided into teams reflecting the mix of professionals typically in attendance for acute clinical events at this institution. Whenever possible, teams were comprised of 2 pediatric residents, 2 pediatric ICU nurses, 1-2 pediatric intermediate care nurses, 1 respiratory therapist, and 1-2 additional nurses from the critical care transport team or institution adult code blue response team. Facilitators for each team included one pediatric critical care attending physician and one advance practice provider or RN. There was some variability of team composition based on staff availability, which ultimately mimicked the reality of staffing when acute patient events occur.

A pre-course email was sent to each participant one week prior to participation that outlined the pillars of a successful pediatric critical care team, highlighting: 1. the shared mental model, 2. communication, and 3. practice. On the day of simulation, facilitators for each team led a pre-brief lecture which reviewed the content provided in the pre-course email, oriented participants to the supplies and equipment to be used and set objectives and expectations for the team.

Each session was designed to be four hours in length, include three to five scenarios, and run back-to-back without significant time for breaks. The session is designed to mimic a busy day in the PICU, where admissions and acute events occur rapidly and in rapid succession. Individual scenarios were run uninterrupted to a predetermined point, at approximately 15 minutes, before being stopped. Rapid cycle deliberate practice was then used to repeat key aspects of the case. Debriefs were held prior to RCDP, at the end of each scenario, and at the conclusion of the session.

Course Measures Curriculum effectiveness was evaluated using: 1. Pre-and post- training surveys, 2. Independent reviewer ratings of leadership and teamwork 3. Clinical learning checklists.

Pre and Post Training Surveys: Pre, post, and 60-day post-training surveys were created by the PPEC team and administered to all participants using Research Electronic Data Capture (REDCap) [34]. The survey (Appendix A) used a 5-point Likert scale to assess self-reported confidence in the following domains: 1. Clinical skills, 2. Situational awareness and 3. Teamwork. The pre-training survey was administered during the pre-brief lecture and collected learner demographics. Posttraining surveys were administered during the final de-brief. The 60-day post training survey was sent via email to participants to assess long-term retention of skills and the effect of simulation on clinical practice.

Independent Reviewer Ratings: Independent reviewers were recruited to assess team leader performance and overall teamwork during PICU PRACTICE sessions. Reviewers were healthcare professionals with advanced degrees employed by the study institution who were unaffiliated with the PICU. Reviewers were positioned in the simulation control room behind a two-way mirror to minimize observation bias. The Ottawa Crisis Resource Management (CRM) Global Rating Scale is considered the gold standard for measurement of physician performance during simulated emergencies [35], but no tool currently exists to evaluate performance of interprofessional teams. An adapted Ottawa-scale was developed to evaluate overall functioning of the team and team leader using a 4-point Likert scale (Appendix B). The skills in the tool highlight essential aspects of crisis management. Assessment areas include leadership, problem-solving, situational awareness, resource utilization, and communication skills.

Clinical Learning Checklists and Time to Event Measurement: Clinical learning checklists for each scenario were developed using national practice guidelines or expert consensus and highlighted key interventions for learners to complete [27-32]. Checklists for the most frequently measured skills are included in Appendix C. Both time-to-event and task completion were measured By observers positioned either behind observation glass or just outside the simulation room.

Statistical Analyses

Survey response means were analyzed using unpaired t tests and analysis of variance (ANOVA). A p value of <0.05 was considered significant. For all study measures, unanswered items were excluded from analysis. The data were also analyzed using nonparametric methods and validity testing with randomized subset comparisons. Only the results from parametric analyses are presented because they were comparable to the nonparametric results. All analyses was completed using the statistical software package Stata 17.0 (Stata Corporation, College Station, TX).

Results

Needs Assessment

A Delphi needs assessment was completed in six cohorts between 2017 and 2020. In total, 110 pediatric residents and pediatric ICU nurses responded. Mean scores greater than 3.5 were prioritized for learning and top priorities included shock, resuscitation, respiratory failure, and trauma (Figure 1).

Figure 1: Priorities for learning identified by Delphi needs assessment

Participant Demographics

A total of 218 nurse and physician participants attended PICU PRACTICE between March 2020 and October 2022. Nurse participants (n=159) included RN’s from PICU, intermediate care, pediatric critical care transport team, neonatal intensive care unit, and adult code blue response team. Physician participants included second year (n=29) and third year (n=30) pediatric residents.

Pre and Post Training Surveys

In comparison of pre training to post training surveys, statistically significant improvements in mean confidence ratings were noted in 13 out of 15 parameters. Improvements were seen in clinical skills, teamwork, and situational awareness domains, represented in Figure 2, Figure 3, and Figure 4 respectively.

Figure 2: Clinical skills domain: comparison of pre-course and immediate post course mean confidence ratings.

Figure 3: Teamwork domain: comparison of pre-course and immediate postcourse mean confidence ratin.

Figure 4: Situational awareness domain: comparison of pre-course and immediate post-course mean confidence ratings

Reported confidence improvements were not statistically significant in the situational awareness parameter “asking for help during a code blue situation” or the clinical skill parameter of “fluid bolus administration.” Both parameters had high pre training mean confidence ratings, which remained high.

Skill Retention

Fifty-nine participants returned 60-day post training surveys. Analysis of pre-course, post-course, and 60-day post course surveys showed significantly improved confidence ratings in 10 of 15 parameters from pre to 60-day post course (n=59). Learner confidence in management of intracranial pressure (ICP) increased from 3.56 pre-course to 4.08 at 60-days post course (p=0.0001), confidence in rhythm interpretation increased from 3.69 to 4.03 from pre to 60-days post course (p=0.002), and confidence in algorithm identification increased from 3.82 to 4.05 (p=0.034). Comparison of selected key parameters representing the trend are displayed in Figure 5.

Figure 5: Mean confidence ratings in selected domains of pre-course, immediate post-course, and 60-day post-course surveys.

In all areas that showed increased confidence from pre to 60-days post-course, none showed statistically significant decline from immediately post course to 60 days post course, indicating retention improvements made during the course.

Teamwork and Leadership Ratings

Teamwork and leadership ratings were scored using a modified Ottawa scale for 14 teams between October 2021 and March 2022. While statistical significance was limited by low sample size, initial trends show increasing average scores in teamwork and leadership scores from the first to last scenario in each session (Figure 6).

Figure 6: Mean teamwork and leadership ratings by scenario order.

Average teamwork scores increased from 16.30 (out of 24 possible points) for the first scenario of a session to 18.73 for the last (n=13). Average leadership scores increased from 15.69 (n=13) for the first scenario to 18.29 for the third scenario (n=14). There was a significant increase in average leadership ratings for scenarios run by third year residents in comparison to those run by second year residents (18.80 vs 16.65, p=0.02).

Clinical Learning Checklists

Comparison of clinical learning checklist performance from March 2022 to performance in October 2021 demonstrated a 16.9% increase in checklist elements completed for scenarios requiring scenarios requiring cardiopulmonary resuscitation (CPR). There was also a 19% increase in checklist item completion for scenarios requiring sepsis management, a 16.6% increase for scenarios requiring intubation, and a 71% increase for scenarios requiring increased ICP management.

Simulation is a widely implemented tool used by medical professionals to improve the delivery of healthcare. Most simulations described currently in literature measure one aspect of training, are conducted in the silo of a single discipline, or only measure the performance of one profession. The successful resuscitation of pediatric patients, however, is achieved through the interactions of an interprofessional team. The PPEC asked the pertinent question, “Why do we not practice as we play?” PICU PRACTICE removed the interprofessional barrier and designed a unique curriculum with objectives prioritized by an interprofessional community, with participants learning as an interprofessional team, and with outcomes measured as an interprofessional unit. The unique participant makeup not only reflected a pediatric resuscitation team at a smaller institution with high nursing turnover and transient resident leadership, it allowed that team to improve together through the practice of care delivery in high acuity, low frequency events.

The modified Delphi approach allowed participants to communicate their learning priorities. The PPEC then designed scenarios centered on participant identified priorities which included the improvement of care in patient resuscitation, traumatic injury, shock, and respiratory failure. Learners not only reported improvement in the ability to perform resuscitation and manage traumatic injury, but measured outcomes also demonstrated learners’ improved adherence to pediatric guidelines in cardiopulmonary resuscitation, traumatic brain injury, septic shock, and intubation. The participant reported improvement in pediatric resuscitation skills following training then continued clinically as learners reported an improvement of those skills when providing care to critically ill children in the PICU during the 60 days following training.

The introduction of third-party rating allowed objective documentation of improvement in resident performance. The 6 core competencies of the Accreditation Council for Graduate Medical Education are patient care, medical knowledge, professionalism, interpersonal communication skills, practicebased learning and improvement, and systems-based practice. The PICU PRACTICE curriculum not only provided a structured learning environment that met all 6 competencies, the Ottawa leadership ratings also demonstrated the ability of the resident to progress from a leader who is unable to prioritize or request help to one that is clearly able to use resources to maximal effectiveness and communicate in a clear and concise manner.

Interprofessional training and joint accreditation continues to increase in graduate medical education. However, there is a paucity of literature on validated tools to measure improvements in the interprofessional team. PICU PRACTICE demonstrated that the adaptation of the Ottawa to the team environment provides a useful tool to not only measure improved teamwork but also demonstrated that the curriculum was successful in improving teamwork. As the interprofessional team moved from scenario 1 to scenario 3, teams that were unable to work together to complete tasks in a timely manner during case 1 subsequently learned and demonstrated the ability to remain calm and in control for the entire crisis while monitoring and reassessing the situation without cues.

This course is the first of its kind to demonstrate an interprofessional simulation designed by an interprofessional team that is specific to the PICU setting, evaluates outcomes for both registered nurse and physician learners, and objectively measures how well they function as a team. PICU PRACTICE is a well described process to design, implement, and measure outcomes of a pediatric CRM curriculum that may be used by other institutions, adapted to the unique makeup of those institutional interprofessional teams, and used to evaluate team performance.

PICU PRACTICE highlights multiple future opportunities in the research of interprofessional education and simulation. The teamwork rating scale has proven useful in the measurement of improved teamwork. However, further validation of this tool is required. Future research must also be explored as to what value measuring teamwork can add to clinical practice. In addition, evaluations of how simulation translates into clinical practice must be developed in order to further understand the incorporation of learned skills into clinical practice.

We have identified some limitations in our pediatric crisis resource management curriculum. The ability to generalize the existing curriculum to other centers will be challenging given the extensive time and resources needed to implement this learning program. We have been fortunate to have a state-of-the-art simulation center and utilized high fidelity simulation throughout this curriculum. In addition, numerous highly skilled simulation technicians have been instrumental with the implementation. The institution has recognized and supported this education program and funded interdisciplinary team training time biannually.

The PICU PRACTICE pediatric crisis resource management is a novel, interprofessional tool to train and evaluate teams in pediatric resuscitation management skills. Future directions of research include evaluating the translation of PICU PRACTICE curriculum to performance in high-risk, clinical situations and the development of validated crisis resource management tools to evaluate performance of interprofessional teams. While training has improved skill efficiency and knowledge, the long-term effects and retention of this training would require further study. Lastly, refinement of the measurement tool for clinical checklist elements is necessary to increase the integrity of the data.

This work was supported by Dartmouth Health Children’s and Dartmouth Health Patient Training Simulation Center (PTSC). The authors would like to thank the following individuals for their contributions to this research: Kelly Corbett, Matthew Braga, Christopher Jones, Katie Darak, and Briggs Carhart.

1. Girotra S, Spertus JA, Li Y, Berg RA, Nadkarni VM, Chan PS. Survival trends in pediatric in-hospital cardiac arrests an analysis from get with the guidelines-resuscitation. Circulation: Cardiovascular Quality and Outcomes. 2013; 6: 42-49.

2. Wolfe HA, Morgan RW, Zhang B, Topjian AA, Fink EL, Berg RA, et al. Deviations from AHA guidelines during pediatric cardiopulmonary resuscitation are associated with decreased event survival. Resuscitation. 2020; 149: 89-99.

3. Jonathan Guilfoyle F, Milner R, Kissoon N. Resuscitation interventions in a tertiary level pediatric emergency department: Implications for maintenance of skills. CJEM. 2011; 13: 90-95.

4. Donoghue AJ, Nadkarni VM, Elliott M, Durbin D. Effect of hospital characteristics on outcomes from pediatric cardiopulmonary resuscitation: A report from the national registry of cardiopulmonary resuscitation. Pediatrics. 2006; 118: 995-1001.

5. Sandquist M, Tegtmeyer K. No more pediatric code blues on the floor: Evolution of pediatric rapid response teams and situational awareness plans. Transl Pediatr. 2018; 7: 291-298.

6. Blackwood J, Duff JP, Nettel-Aguirre A, Djogovic D, Joynt C. Does teaching crisis resource management skills improve resuscitation performance in pediatric residents? Pediatr Crit Care Med. 2014; 15.

7. Solana NE, Xu D, Williams M. Building capacity in critical care nurses to improve individual and team performance in cardiopulmonary arrest events: an educational initiative and simulation training. Canadian J Cardiol. 2019; 35: S207.

8. Lehner M, Heimberg E, Hoffmann F, Heinzel O, Kirschner HJ, Heinrich M. Evaluation of a Pilot Project to Introduce Simulation-Based Team Training to Pediatric Surgery Trauma Room Care. Int J Pediatr. 2017; 2017: 1-6.

9. Hudgins K. Clinical simulation learning in critical care. Crit Care Nurs Q. 2017; 40: 108-110.

10. Couto TB, Kerrey BT, Taylor RG, Fitzgerald M, Geis GL. Teamwork skills in actual, in situ, and in-center pediatric emergencies: Performance levels across settings and perceptions of comparative educational impact. Simul Healthc. 2015; 10: 76-84.

11. Boyle WA, Murray DJ, Beyatte MB, Knittel JG, Kerby PW, Woodhouse J, et al. Simulation-Based Assessment of Critical Care “Front-Line” Providers. Crit Care Med. 2018; 46: e516-e522.

12. Toy S, Miller CR, Guris RJD, Duarte SS, Koessel S, Schiavi A. Evaluation of 3 cognitive load measures during repeated simulation exercises for novice anesthesiology residents. Simul Healthc. 2020; 15: 388-396.

13. McLaughlin CM, Wieck MM, Barin EN, Rake A, Burke RV, Roesly HB, et al. Impact of simulation-based training on perceived provider confidence in acute multidisciplinary pediatric trauma resuscitation. Pediatr Surg Int. 2018; 34: 1353-1362.

14. Cheng A, Lockey A, Bhanji F, Lin Y, Hunt EA, Lang E. The use of highfidelity manikins for advanced life support training-A systematic review and meta-analysis. Resuscitation. 2015; 93: 142-149.

15. Singleton MN, Allen KF, Li Z, McNerney K, Naber UH, Braga MS. Rolling-refresher simulation improves performance and retention of paediatric intensive care unit nurse code cart management. BMJ Simul Technol Enhanc Learn. 2018; 4: 77-82.

16. Braun LR, Sawyer T, Smith K, Hsu A, Behrens M, Chan D, et al. Retention of pediatric resuscitation performance after a simulationbased mastery learning session: A multicenter randomized trial. Pediatr Crit Care Med. 2015; 16: 131-138.

17. Auerbach M, Roney L, Aysseh A, Gawel M, Koziel J, Barre K, et al. In situ pediatric trauma simulation: assessing the impact and feasibility of an interdisciplinary pediatric in situ trauma care quality improvement simulation program. Pediatr Emerg Care. 2014; 30: 884-891.

18. Steinemann S, Berg B, Skinner A, DiTulio A, Anzelon K, Terada K, et al. In Situ, Multidisciplinary, Simulation-Based Teamwork Training Improves Early Trauma Care. J Surg Educ. 2011; 68: 472-477.

19. Weis JJ, Croft CL, Bhoja R, Lee JH, Scott DJ. Multidisciplinary Simulation Activity Effectively Prepares Residents for Participation in Patient Safety Activities. J Surg Educ. 2019; 76: e232-e237.

20. Bullard MJ, Leuck JA, Howley LD. Unifying interdisciplinary education: Designing and implementing an intern simulation educational curriculum to increase confidence in critical care from PGY1 to PGY2. BMC Research Notes. 2017; 10.

21. Acgme. ACGME Common Program Requirements (Residency).

22. Ziesmann MT, Widder S, Park J. S.T.A.R.T.T. J Trauma Acute Care Surg. 2013; 75: 753-758.

23. Long AM, Lefebvre CM, Masneri DA, Mowery NT, Chang MC, Johnson JE, et al. The Golden Opportunity: Multidisciplinary Simulation Training Improves Trauma Team Efficiency. J Surg Educ. 2019; 76: 1116-1121.

24. McLaughlin C, Barry W, Barin E, Kysh L, Auerbach MA, Upperman JS, et al. Multidisciplinary Simulation-Based Team Training for Trauma Resuscitation: A Scoping Review. J Surg Educ. 2019; 76: 1669-1680.

25. Gillman LM, Brindley P, Paton-Gay JD, Angels PT, Park J, Vergis A, et al. Simulated Trauma and Resuscitation Team Training course— evolution of a multidisciplinary trauma crisis resource management simulation course. Am J Surg. 2016; 212: 188-193.e3.