Rimi Sambi and Heather Sawula¹, Brent Wolfrom², and Joseph Newbigging³

(1) PGY-3 Queen’s University, Department of Family Medicine, CCFP-EM, equal contributing authors
(2) Assistant Professor, Queen’s University, Department of Family Medicine
(3) Assistant Professor, Queens University, Division of Emergency Medicine

Download article PDF – POCUS Journal 2017; 2(2):15-17

Abstract

As point of care ultrasound (PoCUS) becomes increasingly popular and a standard of care in many clinical settings, the interest for integration in medical undergraduate curriculum is also growing [1]. This project aims to assess whether formal bedside Focused Abdominal Scan for Trauma (FAST) exam training of medical students increases their knowledge and comfort with the use of bedside ultrasound in a family medicine setting at Queen’s University. Third year medical students (n=18) were recruited to participate in a training session involving a 1-hour online video and 2-hour hands-on session. Knowledge based surveys were completed before and after the training. A survey was completed 4 months after the teaching session evaluating knowledge retention, comfort, and application of skills. Student knowledge of PoCUS and FAST increased and was maintained (pre-training 56%±20%, post-training 82%±10%, p<0.001). Self-evaluation of comfort performing a FAST examination (5-point Likert scale) similarly increased post-training session (pre-training 1.4±0.8, post-training 3.8±0.9, p<0.005), but decreased 4 months later (3±1.2, p<0.005). Students in this study were unanimously interested in ultrasound training and the methods used effectively increased theoretical knowledge and comfort with use. Students did not retain their comfort levels with FAST exam 4 months after the training session, nor did they have the opportunity to utilize the skills learned. Further evidence is required to identify the applicability of these results to undergraduate curriculum development.

Introduction

Point of care ultrasound (PoCUS) has become a new standard of care and clinical tool used commonly by physicians in many clinical settings including the Emergency Department, hospital wards, and outpatient clinics. Preliminary pilot projects from the USA, UK, and Canada demonstrate feasibility and benefit to integrating teaching into undergraduate medical education (UGME) curriculum and surveys of Canadian medical school vice-deans and medical students show strong support and interest in formalized teaching of bedside ultrasound [1,2,3,4,5,6,7]. Comprehensive studies demonstrate bedside ultrasound is a skill that medical students are able to learn and enhances their anatomy knowledge, clinical accuracy, and physical exam skills. Formalized bedside ultrasound training in Canada is a developing field and there is opportunity to create a program at Queen’s University.

We elected to provide ultrasound teaching focused on medical students beginning community family medicine rotations as they will be exposed to elements of acute care in these practice settings. Family physicians work in a wide variety of settings where PoCUS is applied and a critical skill, especially if there is decreased access to formalized ultrasound in the acute care setting.

Given the applicability of the FAST examination in a variety of clinical settings, we chose it to act as a model module for teaching bedside ultrasound. This project aims to assess whether the introduction of structured FAST training improves medical students’ knowledge and comfort level with ultrasound use.

Methods

Our study population consisted of (n=18) third-year medical students who were entering the clerkship phase of their curriculum in fall 2016 and were participating in their family medicine core rotations at a variety of community sites associated with Queen’s University. Due to recruitment challenges, recruitment was opened to all third-year medical students to increase sample size. Following recruitment, the students signed letters of consent in compliance with Queen’s University Health Sciences and Affiliated Hospitals Research Ethics Board.

Students completed an electronic survey at three points throughout the study – before any training, immediately after hands-on training session, and 4 months into clerkship. Each survey similarly assessed knowledge of the FAST exam, self-rated comfort level with use of the ultrasound machine, and number of PoCUS exams performed and/or observed. The surveys were designed with standardized 5-point Likert evaluation scales to gather information on opinions and comfort levels. The second and third surveys had additional components evaluating the teaching session and barriers to using bedside ultrasound.

Students watched a 45 min video on the FAST exam at home prior to the 2-hour hands on training session. This session had a ratio of 1:3 instructor to learner ratio with six scanning stations with standardized patients and certified instructors. The FAST exam to act as a model module for teaching bedside ultrasound due to its applicability in a variety of clinical settings. The FAST exam is defined as an assessment of the hepatorenal recess, the splenorenal recess, 4-chamber view of the heart and pericardium, and the retrovesicular space.

An objective structured clinical examination (OSCE) was held the day after the training session to assess technical skills and students’ comfort with the use of the ultrasound machine to perform a FAST exam. Certified ultrasound providers evaluated this using a standardized evaluation tool that rated students on a 5-point scale (1=inferior, 2=novice, 3=competent, 4=advanced, 5=superior) across multiple aspects of ultrasound performance (preparation, image acquisition, image optimization, clinical integration, entrustment decision).

Data were recorded in Excel and statistical analysis was performed using Excel’s data analysis tool. Descriptive data, where applicable, was expressed as mean±SD. Significant differences were determined by repeated measures ANOVA for comparison of knowledge acquisition and self-reported scores of comfort. A sub-group analysis was performed to see if there were any differences between students at community sites versus urban sites.

Results

The majority (61%) of students had no prior formal ultrasound training. Those that had done prior training had performed 5 or fewer FAST examinations prior to the training session. Of the 18 study participants, 8 were beginning clerkship in community sites and comprised our subgroup population.

Students initially scored a mean score of 56%±20% on the content-based knowledge survey. This significantly increased to a score of 82%±10% (p<0.001) after the teaching session. When this survey was repeated 4 months into clerkship there was no significant difference compared to the post-test results (71%±28%, p = 0.17).

Student self-evaluation of their comfort performing a FAST examination increased from an average rating of 1.4±0.8 on a 5-point Likert scale to 3.8±0.9 after the training session. This was statistically significant (p<0.001). Four months after the training session the students’ self-evaluation was had decreased from 3.8±0.5 to 3±1.2, which was also statistically significant (p<0.005).  Results were similar in the sub group analysis.

According to instructor evaluations of student OSCE performances to superior, the median overall rating was a score of 2, or “novice performance”. Students scored similarly across the various components of the OSCE assessment.

Figure 1. FAST knowledge survey scores. (n=18)

Figure 2. Student responses to “I am comfortable performing a FAST exam independently with supervision” (n= 16)

Discussion

These results demonstrate that the online and hands-on training were effective in increasing student knowledge about bedside ultrasound and the FAST examination. Similarly, students felt significantly more comfortable performing a bedside examination after a single training session. However there is a discrepancy between the students’ comfort levels and instructor evaluation of performance of the FAST examination. This illustrates both that a single training session was insufficient to make students competent in the FAST exam, as well as the importance of objectively measuring skill acquisition. While increased student comfort performing the examination is beneficial, it is important to be wary of false confidence. This was similarly demonstrated in a paper from McGill, where the students self-rated scores on bedside ultrasound performance grossly overestimated the scores assigned by instructors [7].

There was a significant decrease in students’ comfort with ultrasound use 4 months after the training session. This is unsurprising as none of the students performed a bedside ultrasound during the study period after the training session. While there was no decrease in students’ knowledge over the study period, there was a decrease in comfort which suggests that knowledge is likely only a single factor that determines student comfort level. The study by Steinmetz et al from McGill University suggests that multiple sessions throughout the year help students maintain their comfort level with ultrasound use, along with the theoretical knowledge for ultrasound use [7].

Barriers to ultrasound use during the study period, as cited by the students, included preceptor discomfort with its use, lack of opportunity due to patient presentation or machine availability, student personal comfort level, and expectation that it is a skill restricted to residents and staff physicians. Although the students did not perform bedside ultrasound, those that witnessed it in clinical use unanimously reported a better understanding of what they were seeing.

This pilot project highlights some of the barriers to use specific to the medical student perspective, which include availability of machines and preceptor familiarity. Previous studies looking at ultrasound training in family medicine residency programs cited lack of trained faculty and limited access to ultrasound equipment as common barriers [8].

According to a working group termed ‘Ultrasound in Medical Education, California’ (UMECali), successful integration of ultrasound education in undergraduate medical training requires consideration of several key themes for success [10]. A bottom-up approach with medical student involvement and integration of ultrasound in the preclinical years are key items for program success. This pilot project encompassed some of these key themes that could be built upon moving forwards. Similarly, the concept of longitudinal training is critical and can be achieved in several ways. An online curriculum, such as the video distributed prior to the teaching session, can help to maximize the time spent with hands on learning. Continuing these types of components across time could be another strategy to help retain knowledge when access to ultrasound machines is limited.

Some of the limitations to this study relate to the scale and timeframe of the project. Due to resource limitations we were only able to accommodate 18 students which resulted in a small overall sample size. We were also unable to administer another OSCE to evaluate skill retention due to time and logistical constraints. As evidenced by the study from McGill, student self-rated assessment varied from instructor evaluations and cannot be used as a surrogate marker for skill retention [7]. While knowledge retention is beneficial, we are unable to comment on skill retention, which is an important aspect of bedside ultrasound training. As well, since all of our participants were volunteers, there is volunteer bias, which limits the external validity of this study.

Conclusion

This pilot project demonstrates that students are very interested in ultrasound training and the methods used to teach were effective in increasing knowledge and comfort with ultrasound use. While no students in the group performed the skill in the 4-month period after the teaching session, students reported a better understanding when they witnessed bedside ultrasound performed.

Some of the barriers identified through this project such as preceptor knowledge regarding the use of PoCUS, and attitudes towards medical students performing this skill may change with training of clinicians across all levels and a bottom-up approach with education at the undergraduate medical education level.

Future studies would benefit from larger sample size, repeated teaching sessions and online curriculum to maintain exposure to PoCUS. A longer duration of follow up and re-testing skill acquisition would be an important component of curriculum assessment. Further evidence is required to identify the optimal way to integrate ultrasound into undergraduate curriculum development.

References

1. Steinmetz P, Dobrescu O, Oleskevich S, and Lewis J. Bedside ultrasound education in Canadian medical schools: A national survey. Can Med Educ J 2016; 7(1):e78–e86
2. Arger PH, Schultz SM, Sehgal CM et al. Teaching medical students diagnostic sonography. J Ultrasound Med 2005; 24(10):1365-1369.
3. Dinh V, Dukes W, Prigge J, Avila M. Ultrasound Integration in Undergraduate Medical Education: Comparison of Ultrasound Proficiency Between Trained and Untrained Medical Students. J Ultrasound Med 2015; 34(10):1819-1824.
4. Favot M, Courage C, Mantouffel J, Amponsah D. Ultrasound Training in the Emergency Medicine Clerkship. WestJEM. 2015; 16(6):938-942.
5. Gogalniceanu P, Sheena Y, Kashef E, et al. Is basic emergency ultrasound training feasible as part of standard undergraduate medical education?. J Surgical Ed 2010; 67(3):152-156.
6. Shapiro RS, Ko PP, & Jacobson S. A pilot project to study the use of ultrasonography for teaching physical examination to medical students. Comp Bio Med 2002; 32(6):403-409.
7. Steinmetz P, Oleskevich S, & Lewis J. Acquisition and Long‐term Retention of Bedside Ultrasound Skills in First‐Year Medical Students. J Ultrasound Med 2016; 35(9):1967-1975.
8. Day J, Dabis J, Riesenberg L et al. Integrating Sonography Training Into Undergraduate Medical Education: A Study of the Previous Exposure of One Institution’s Incoming Residents. J Ultrasound Med 2015;34(7):1273-1257.
9. Hall JW, Holman H, Bornemann P et al. Point of Care Ultrasound in Family Medicine Residency Programs: A CERA Study. Family medicine 2015; 47(9):706-711.
10. Chiem AT, Soucy Z, Dinh VA et al. Integration of Ultrasound in Undergraduate Medical Education at the California Medical Schools. J Ultrasound Med 2016; 35(2):221-233