Robert Morgan, DO¹; Bradley Sanville, MD¹; Shashank Bathula⁴; Shaban Demirel, PhD³; R. Serene Perkins, MD^{2, 3}; Gordon E. Johnson, MD¹

(1) Legacy Emanuel Medical Center, Department of Internal Medicine, Legacy Health, Portland, OR, USA

(2) Legacy Institute for Surgical Education and Innovation, Legacy Research Institute, Legacy Health, Portland, OR, USA

(3) Department of Clinical and Outcomes Research, Legacy Research Institute, Legacy Health, Portland, OR, USA

(4) Western University of Health Sciences, Lebanon, OR, USA

---

Download article PDF – POCUS Journal 2019; 4(2):33-37.

---

Article updated June 12, 2020. Previous version was missing Figure 2. Please see notice of erratum for more detail.

---

Abstract

Introduction: Focused Cardiac Ultrasound (FoCUS) is a relatively new technology that requires training and mentoring. The use of a FoCUS simulator is a novel training method that may prompt greater adoption of this technology by physicians at different levels of training and experience. The objective of this study was to determine if simulation training using an advanced echo simulator (Real Ultrasound^®) is a feasible means of delivering training in FoCUS. Methods: Twenty-five residents and attending physicians participated in this study. After performing a pretest, training on the Real Ultrasound^® was administered. Improvement was assessed immediately after simulator training. Additionally, some participants were retested six months after training to determine whether learned skills were retained. Results: Of the 25 participants recruited, all completed the pretest phase, and 17 completed the training and immediate posttest assessment. At pretest, the median angular deviation of acquired images from anatomically correct was 37°, which improved to 30° after training (p<0.002). Technical skill was largely maintained at six months of follow-up, with a median angle error of 27 and 31°, respectively (p=0.093) in 8 participants who completed the post and six-month retention assessments. The median pretest image interpretation score improved from 55% to 70% (p=0.028); median post and six month scores in the 8 participants were 72 and 68%, respectively (p=0.735). Conclusions: Simulation training in FoCUS significantly improves skills in image acquisition. These skills appear to be retained over time. This study adds support for the use of advanced echocardiographic simulators to enhance formal FoCUS training in a real-world setting.

Introduction

Point of care ultrasound (PoCUS) techniques, such as focused cardiac ultrasound (FoCUS), are useful adjuncts to the clinical examination of patients and are being rapidly adopted [1]. Currently, training in PoCUS is provided either during graduate medical education, at postgraduate educational conferences, or informally via online self-study. Much of the training being offered today is not hands-on, so skills consolidation requires that image acquisition and interpretation be learned under the direct supervision of a highly-trained instructor on live patient models. Although there is a growing need for formal training, it is both time and resource intensive [2].

The Real Ultrasound® cardiac ECHO simulator was developed to address these issues, as it can be utilized by physicians at any level of training and experience [3]. One advantage of simulators is that they allow clinicians to train on their own time and in a self-directed fashion. In this mode of training, clinicians benefit from real-time feedback regarding the anatomic accuracy with which they acquire standard views, which can be especially important early in the process of skills development [4-6].

We aimed to assess the utility of FoCUS simulation as a training aid for physicians at the graduate and postgraduate levels. In theory, by providing a freely accessible asynchronous mode of training via simulation, traditional time and cost limitations of PoCUS training can be largely overcome, while ensuring that training remains rigorous, reproducible, and effective. In this study, we assessed skills for cardiac image acquisition and interpretation, as well as improvement (short-term and long-term retention of skills) using a newly developed ultrasound simulator (Real Ultrasound^®).

Methods

Study population

Invitations to participate in this study were sent to internal medicine residents, as well as internal medicine and emergency medicine attending physicians at Legacy Emanuel Medical Center (Portland, OR, USA). Twenty-five individuals (internal medicine residents, hospitalists, and emergency medicine physicians) gave their written consent to participate after being informed of the risks and benefits of study participation. This study was evaluated and approved by the Legacy Health Institutional Review Board.

Study design

The FoCUS simulator used in this study was developed at the University of Washington (UW) and has been validated in multiple studies for training and competency assessment [3,7,8]. Participants first completed a pre-training assessment (pretest), then a training curriculum of instructional modules, followed by six assigned interactive practice cases (with the option to complete all 10 available cases if the participant desired), and finally a post-training assessment test (posttest). The curriculum included instructional modules on focused cardiac ultrasound image acquisition and image interpretation in six standard views (parasternal long axis, parasternal short axis at the mitral valve, at the aortic valve, apical 4-chamber, apical long axis, and subcostal) of normal and commonly encountered abnormal pathology. To assess long-term skills and knowledge retention, a retention test was administered 6 months after training. The simulator hardware included tracking of the location and orientation of the mock transducer using a magnetic field system on mannequin’s chest [3]. The tracking enabled assessment of participants’ technical skill in terms of the angular deviation between the plane of a learner-acquired image and the plane of the anatomically correct image for a specified standard view (Figure 1). The correct view plane is defined geometrically from 3D reconstruction of each case’s heart chambers and associated structures and verified by expert sonographers involved in training cardiology fellows at the UW 3. In addition the distance between the transducer locations for acquired and correct views was measured.  In addition to image acquisition skills, participants were challenged to interpret the pathology (image interpretation skills). Cognitive skill in image assessment was our third skill metric; it was measured using multiple-choice examination of didactic knowledge. Skill was assessed using these three metrics during the pre, post and retention phases to gauge progress.

Simulation exercises

The simulator was available to participants 24 hours a day, seven days a week at the Legacy Emanuel Medical Center campus. Participants were tested on two skills: image acquisition and image interpretation. The transducer’s position and orientation relative to the mannequin were used to generate and display a two dimensional image from a three dimensional data set in real-time on a monitor (Figure 1). All cases on the simulator display original transthoracic echocardiographic image data from normal subjects or from patients with pathologic findings such as reduced EF, pericardial effusions, valvular pathology, LVH etc.

During the training portion of the study, once the participants obtained what they felt was the optimal view, they were instructed to turn on a three-dimensional visual guidance indicator, which provided real-time feedback, to assist with manipulating the transducer into position to optimize the view. However, this guidance was not available during the testing portions of the study.

Six months after initial training and assessment (pretest, training, and posttest), participants were invited to repeat testing of their imaging skill and image interpretation to assess knowledge retention. Participants were also asked to complete an online survey to allow us to collect demographic and training/experience level data.

Statistical analysis

All statistical comparisons were performed using the Wilcoxon signed-ranks test with Bonferroni correction for multiple comparisons. Values are reported as the median. All statistical analyses were performed using the SPSS software package.

Results

Participants

Twenty-five individuals participated in this study and completed the pretest assessment. Of these, 17 of 25 (68%) completed some training modules and performed the posttest assessment. Only 8 of 25 (32%) participants completed the 6-month follow-up testing of their imaging skill and image interpretation knowledge in the retention phase of the study. Fifteen of 25 (60%) participants completed the online demographic survey, which captured data regarding age, sex, level of training, medical specialty, and amount of prior PoCUS training (Table 1).  The participants were not receiving any other formal training during the study, other than self-directed learning  

Table 1. Demographic characteristics of study participants.

Study Participants
	Total enrolled	25
	Completed survey	15
Age — no. (%)	21-30	4 (26.7)
	31-40	5 (33.3)
	41-50	4 (26.7)
	51+	2 (13.3)
Gender — no. (%)	Male	10 (66.7)
	Female	5 (33.3)
Level of training — no. (%)	Medical Student	1 (6.7)
	Resident	6 (40.0)
	Attending	8 (53.3)
Specialty — no. (%)	Medical Student	1 (6.7)
	Internal Medicine	13 (86.7)
	Emergency Medicine	1 (6.7)
Prior US training — no. (%)	0-1 days	5 (33.3)
	1-2 days	4 (26.7)
	2+ days	6 (40.0)

Assessment of immediate skills acquisition and knowledge

For image acquisition, the angle error decreased between pretest and posttest in the 17 participants who completed training and both tests (median 37° pretest to 30° posttest/ Z=-3.385, p<0.002). For knowledge assessment, participants answered a median of 55% of multiple choice questions correctly on the pretest and improved to an average of 76% on the posttest (Z=-2.457, p=0.028). There was no difference in probe placement error between the pre and post phases of the study, with median placement errors being 3.2 cm and 3.1 cm (Z=-0.308, p=0.758 compared to pretest) respectively.

Skills retention and knowledge after six months

In the 8 participants who completed the pretest, training and posttest session, angle error in image acquisition did not change significantly from median 27◦ pre to 31◦ post (Z=-1.680, p=0.186). Individual participants’ angle error over time is shown in Figure 2. Didactic knowledge remained consistent, with participants answering 72% (median) of questions correctly on the pretest and 68% after 6 months (Z=-0.339, p=0.735). Probe placement error was also similar in the pre and retention phases of the study, with median placement errors being 2.9 and 3.1 cm, respectively (Z=-1.680, p=0.186.)

Discussion

Training on a FoCUS simulator can result in significant improvement in cognitive and motor skills that is largely sustained over time in physicians at the graduate and postgraduate levels. This study provides further support for the use of FoCUS simulation, in agreement with previously published studies [3]. There was a wide range of baseline skill levels among the study participants, but this variability narrowed following training on the simulator (Figure 2). Simulator training appeared to have the biggest impact on participants whose baseline scores were poorer, but improvement was more modest for those participants whose scores were good at the initial assessment.

The clinical utility and expanding presence of ultrasound in both graduate and postgraduate medical training generates increasing demands for continuing education [1]. Bedside ultrasound is increasingly becoming a useful adjunct to didactic lectures and hands-on training sessions with time and resource-intensive patient models [9]. In this study, we provided free, around-the-clock access to the simulator, which allowed participants to undergo self-directed, asynchronous learning with objectively measurable improvement in knowledge and skills across a wide range of participants at both graduate and postgraduate levels. As graduate medical education increasingly incorporates ultrasound into curricula, high-fidelity simulation can provide both training, and objective assessment of skills and knowledge, needed to ensure competency [10]. For postgraduates who trained before the PoCUS era, simulators also provide a means to become current in ultrasound techniques as well as increase proficiency in this important diagnostic modality.

This study’s conclusions are limited by the size of the study population, particularly for those who returned for a third test at 6 months. The investigation was conducted with volunteers who participated during their limited nonclinical time, so enrollment and completion rates were relatively low. However, completion rates in previously published studies were also low, with Skinner et al [7] further voicing the need for protected training time for PoCUS and FoCUS. One study comparing clinical training alone vs. combined with simulator training for residents in obstetrics and gynecology had a much higher rate of follow-up, perhaps because the study was more formal than ours: it was a randomized trial conducted in three hospitals, and recruitment was continued until the number required for statistical analysis had completed the performance test [11]. Second, more studies are needed to assess the transferability of ultrasound skills and knowledge learned on simulators to real-world point of care ultrasound scenarios with patients. There is a large body of evidence to date, however, reporting that procedural skills learned on the simulator (T1) transfer to improved downstream patient care practices (T2) and improved patient and public health (T3) [12].

Conclusions

Training in FoCUS using a simulator with immediate feedback resulted in significant improvements in image acquisition and image interpretation skills. These skills appear to be sustained over time. Although participants displayed a wide range in their baseline skills, this range narrowed following training with the simulator. This study adds further support for the use of advanced echocardiographic simulators to enhance formal, reproducible, FoCUS training.

Acknowledgements

The authors would like to thank Dr. Florence Sheehan and her collaborators at the University of Washington for loaning us the Real Ultrasound^® simulator used in this study and for their assistance with data extraction and aggregation from the simulator.

References

1. Schnobrich DJ, Gladding S, Olson AP, Duran-Nelson A. Point-of-Care Ultrasound in Internal Medicine: A National Survey of Educational Leadership. J Grad Med Educ. 2013;5(3):498-502.

2. Conlin F, Roy Connelly N, Raghunathan K, Friderici J, Schwabauer A. Focused Transthoracic Cardiac Ultrasound: A Survey of Training Practices. J Cardiothorac Vasc Anesth. 2016;30(1):102-106.

3. Sheehan FH, Otto CM, Freeman RV. Echo simulator with novel training and competency testing tools. Stud Health Technol Inform. 2013;184:397-403.

4. Cawthorn TR, Nickel C, O’Reilly M, et al. Development and evaluation of methodologies for teaching focused cardiac ultrasound skills to medical students. J Am Soc Echocardiogr. 2014;27(3):302-309.

5. Ogilvie E, Vlachou A, Edsell M, et al. Simulation-based teaching versus point-of-care teaching for identification of basic transoesophageal echocardiography views: a prospective randomised study. Anaesthesia. 2015;70(3):330-335.

6. Zierler RE, Leotta DF, Sansom K, Aliseda A, Anderson MD, Sheehan FH. Development of a Duplex Ultrasound Simulator and Preliminary Validation of Velocity Measurements in Carotid Artery Models. Vasc Endovascular Surg. 2016;50(5):309-316.

7. Skinner AA, Freeman RV, Sheehan FH. Quantitative Feedback Facilitates Acquisition of Skills in Focused Cardiac Ultrasound. Simul Healthc. 2016;11(2):134-138.

8. Winter R, Lindqvist P, Sheehan FH. Test of simulator-based assessment of psychomotor skill in transthoracic echo. Clin Physiol Funct Imaging. 2018; Mar 2: Epub ahead of print.

9. Kelm DJ, Ratelle JT, Azeem N, et al. Longitudinal Ultrasound Curriculum Improves Long-Term Retention Among Internal Medicine Residents. J Grad Med Educ. 2015;7(3):454-457.

10. Amini R, Stolz LA, Javedani PP, et al. Point-of-care echocardiography in simulation-based education and assessment. Adv Med Educ Pract. 2016;7:325-328.

11.  Tolsgaard MG, Ringsted C, Dreisler E et al. Sustained effect of simulation-based ultrasound training on clinical performance: a randomized trial. Ultrasound Obstet Gynecol 2015;46:312-318.

12.  McGaghie WC, Draycott TJ, Dunn WF, Lopez CM, Stefanidis D. Evaluating the impact of simulation on translational patient outcomes. Simul Healthc 2011;6 (Suppl):S42-S47