Research: Emergency medicine residents’ acquisition of point-of-care ultrasound knowledge and their satisfaction with the flipped classroom andragogy

by Khalid Bashir MD1; Aftab Azad, MD; Kaleelullah Saleem Farook, MD; Shahzad Anjum, MD; Sameer Pathan, MD; Zain Bhutta, MD; Stephen Hodges Thomas,MD

Emergency Department, Hamad General Hospital, Doha Qatar

 

Download article PDF – POCUS Journal 2018; 3(1):2-5.

 

Abstract

Background: One of the traditional approaches for knowledge transfer in medical education is through face-to-face (F2F) teaching. We aimed to evaluate the acquisition of knowledge about point-of-care ultrasound (POCUS) and learner’s satisfaction with the flipped classroom (FC) teaching approach. Methods: This was a prospective, mixed-method, crossover study and included 29 emergency medicine (EM) residents in current training program. Over a period of three months, each resident was exposed to F2F and FC teaching models in a crossover manner. There was a multiple-choice questions (MCQ) test before and after each educational intervention (F2F & FC). Two months after each educational intervention a final MCQ test was administered to assess the retention of knowledge between the two approaches. After each educational approach feedback was sought from a selected group of residents concerning the acceptability of the two educational approaches through a semi structured interview. Results: A total of 29 EM residents participated in this study. The numbers of residents by year of post-graduation training were seven (24.14%) PGY-1, eight (27.59%) PGY-2, six (20.69%) PGY-3, and eight (27.59%) PGY-4. The baseline mean score was 15.82 using MCQs test mean scores. For the face-to-face teaching model, the difference between pre and post-intervention scores was 2.7 (95% CI 2.1 to 3.3, p=0.001); whereas, for the flipped classroom teaching model, the difference was 3.93 (95% CI 3.2 to 4.5, p= 0.001). At two months post-intervention, for face-to-face teaching model, the MCQ assessment showed an increase of 1.7 (95% CI 1.1 to 2.2, p= 0.001) mean scores when compared to the pre-intervention mean scores; whereas, for the flipped classroom model this difference was significantly higher, recorded as 4.48 (95% CI 3.7 to 5.1, p= 0.001). Finally, the difference between mean scores for F2F and FC teaching models was 2.75 (95% CI 1.87 to 3.64, p=0.001) at two months post-intervention.  Overall, the participants expressed a preference for the FC teaching methodology. Conclusion: Both F2F and FC teaching methods resulted in significant and sustained improvements in POCUS knowledge base. The FC teaching method accomplished higher test scores than the F2F teaching method both at the end of the teaching and after two months of completing the educational program.

 

 

Introduction 

Point-of-care ultrasound (POCUS) is an extremely useful skill set for emergency physicians in rapid identification of pathologies at bedside [1]. Emergency medicine (EM) has a long history of teaching and training residents in bed side ultrasound teaching. POCUS has become an extended part of medical examination for the emergency physician. The Accreditation Council for Medical Education (ACGME) milestones project recognizes POCUS as one of the core skills included in emergency procedures list [1].

The effective instructional teaching strategies include active, passive and mixed learning. Active learning encourages the learners to understand the educational material, partake in the educational session and work together with the colleagues and the faculty [2-4]. The traditional POCUS teaching of EM residents is delivered through didactics followed by demonstration of POCUS skill and subsequent completion of a logbook while working in clinical areas under the supervision of faculty. The didactic teaching is usually considered as a one-way transmission of information from the lecturer to the students. The students then assimilate and apply this knowledge outside the lecture-room environment. Lecturing approach in delivering knowledge seems to be a popular approach among the millennial learners, followed by learning through collaborative learning and learning by doing [5-7].

Lately, the flipped classroom (FC) has become a popular method of delivering knowledge. Two essential components of FC are access to an online educational material and a face to face component [8]. It could be argued that the FC could be the reverse of the traditional face-to-face (F2F) classroom method of teaching. However, in FC the learners develop additional understanding about the topic through watching videos, podcasts or interactive lecture slides online before the didactic teaching. The F2F component of FC is utilized to understand the topic further and clarifying any misconceptions or ambiguities with the help of the faculty [8]. FC is increasingly being used in medical education with successful outcomes in improving knowledge [9]. In addition to medicine, other allied specialties such as pharmacy and clinical epidemiology have also utilized FC method of teaching [10, 11]. However, the usefulness of FC has been questioned by a few studies in assessing students’ satisfaction in learning neuroanatomy [12], and in nursing students groups [13]. In addition, there is a lack of evidence regarding teaching POCUS through FC in the assessment of common EM indications such as Abdominal Aortic Aneurysm (AAA), cardiac POCUS in arrest situation, and Extended Focused Assessment with Sonography for Trauma (E-FAST).

The aim of this study was to assess knowledge acquisition of POCUS by EM residents by FC teaching in comparison to F2F model. The secondary aim was to evaluate the learner satisfaction with FC and F2F model in learning POCUS.

METHODS

Subjects and Settings

Our EM residency training is a four-year program, accredited by ACGME- International (ACGEM-I), and had 39 residents in training at the time of this study. It is a teaching hospital, and POCUS teaching faculty are available on the shop floor on 24/7 basis. The first POCUS teaching session and MCQ test was attended by 29 residents. Although remaining POCUS teaching sessions were attended by more residents, only those 29 who attended the first session were included in the analysis.

Design and data collection

Between January and March 2017, a total of five teaching conferences were delivered on different aspects of point-of-care ultrasound.  Three of those sessions were delivered using F2F didactic teaching approach, while two sessions were delivered as FC approach. The F2F approach consisted of our traditional PowerPoint (Microsoft Corporation, Chicago, IL, USA) format through a projector and some discussion towards the end. The FC slides were prepared including embedded links to short videos and self-assessment questions to stimulate learning.  The online educational material and learning objectives was sent three days before the didactic part of FC session. The didactic part of FC session lasted 45 minutes and was dedicated to answering any queries and clarifying the concepts.

The same MCQ test was administered before and after each educational approach to reduce bias. The first test (Pretest) was administered before any educational intervention and covered various aspects of POCUS knowledge in the above-mentioned subject areas. Result range was between 0 and 25 since there were 25 items on this test (and on all tests given in this study). Next, all residents underwent standard face-to-face (F2F) teaching on distinct subunits within POCUS knowledge areas. In a few weeks, when this F2F teaching was finished, the residents took a test assessing the POCUS knowledge subset that had been covered in the F2F lectures. This test administered immediately after the F2F teaching was coded as the PostF2F test. The same groups of residents were exposed to the second educational intervention i.e. (FC). All of the residents underwent FC teaching on distinct POCUS knowledge areas that were different from those covered by F2F approach. In a few weeks, when FC was finished, the residents took a test assessing the POCUS knowledge subset that had been covered in the FC. This test, administered immediately after the FC, was coded as the PostFC test. Approximately 2 months after the educational interventions (F2F lectures and FC lectures) the residents underwent final MCQ tests assessing the material that had been presented in the F2F and FC approaches. (PostF2F2months & PostFC2months).

After each educational approach feedback was sought from selected group of six residents concerning the appropriateness of the two educational approaches through a brief ten minutes semi structured interview.  Different residents participated in the two interviews. The free text comments were written down by one study author.

Statistical analysis

The data were collected in Microsoft Excel sheet for Mac 2013 (Microsoft Corporation, Redmond, WA.) The Shapiro-Wilk test was used to assess the distributions of test scores.  The data was normally distributed and presented as mean (Standard deviation).  Analyses were conducted with STATA 14 MP, with significance defined at p 0.05 and confidence intervals (CIs) reported at the 95% level. Thematic analysis was used to describe the comments written during semistructured structured interviews.

RESULTS

A total of 29 residents participated in each educational intervention (F2F & FC). There were nine females (31.03%) and twenty males (68.97%) participants.  The number of residents by year of post-graduation training were seven (24.14%) PGY-1, eight (27.59%) PGY-2, six (20.69%) PGY-3, and eight (27.59%) PGY-4. The PostF2F score improvement from PreF2F was 2.7 points with a 95% confidence interval (CI) of 2.1 to 3.3 points (P value 0.001), whereas PostFC score improvement from PreFC was 3.93 points with 95% confidence interval (CI) 3.2 to 4.5 (P value 0.001). Similarly PostF2F2months from PreF2F  showed an improvement of 1.7 points with 95% confidence interval (CI) 1.1 to 2.2(P value of 0.001).  PostFC2months score improvement from preFC was 4.48 points with 95% confidence interval (CI) 3.7 to 5.1(P value of 0.001). The PostFC2months and the postF2F2months showed a mean score of 1 with 95% confidence interval (CI) 0.71 to 1.2(P value of 0.001).  The PostFC score improvement from postF2F was 1.20 points with 95% confidence interval (CI)  0.37 to 2.04(P value of 0.001). Finally, comparing the PostFC2months score improvement from PostF2F2months  was 2.75 points with 95% confidence interval (CI) 1.87 to 3.64(P value of 0.001) (Table 1).

 

Table 1.  Summary and comparison of the test scores at a different time periods

Serial no Variable MCQ mean score difference 95% Confidence  Interval P value
1 Comparing PostF2F V PreF2F   2.72 2.14 to 3.29 0.001
2 Comparing PostFC V PreFC 3.93 3.29 to 4.56 0.001
3 Comparing PostF2F2months V PreF2F 1.72 1.16 to 2.29 0.001
4 Comparing PostFC2months V PreFC 4.48 3.77 to 5.19 0.001
8 Comparing PostFC2months VPostF2F2months V 2.75  1.87 to 3.64 0.001

PreF2F – MCQ Test conducted before the F2F; PreFC- MCQ Test conducted before the FC; Posttest F2F – MCQ Test conducted immediately after F2F; Posttest FC – MCQ Test conducted immediately after FC; PostF2F2months – MCQ Test conducted after two months of F2F; PostF2F2months – MCQ Test conducted after two months of FC.

On average, the residents spent 15 – 60 minutes at home for the online part of FC sessions. Comments were invited from the residents after each educational intervention (Table 2). Residents remarked that the lecture was “interesting in the beginning” but later became “boring” due to the length of the lecture hence preferred “shorter lectures”.   Residents commented that they were more attentive, and learned more during FC session and expressed a preference to have teaching through the FC methodology.  The residents felt that FC teaching was “very stimulating”, “engaging throughout”, they were able to inquire further about the topic, remained alert during the educational session, and “retained most of the educational material”. The also suggested a preference for FC teaching approach for future medical education

 

Table 2. Residents comments after the educational interventions

Comments Face to Face Teaching (F2F) Flipped classroom(FC)
1 Interesting in the beginning Very stimulating
2 Became boring halfway through Engaging through out
3 Too much information Was able to ask lot of questions
4 Felt sleepy Remain awake throughout
5 Less retention Retained most of the information
6 Would prefer shorter lecture Would prefer all teaching through FC

 

Discussion

This study was designed to assess acquisition of POCUS knowledge through two educational approaches. Both educational approaches (F2F and FC) resulted in improving the knowledge of POCUS among the residents. The improvement in the acquisition of knowledge continued at two months post intervention assessment. FC educational approach resulted in statistically enhanced retention of knowledge both immediately and at two months.  Generally the residents felt encouraged to learn through FC methodology as compared to the traditional F2F classroom teaching.

There are several possible reasons for the effectiveness of FC as compared to traditional F2F in acquisition of knowledge related to POCUS. Firstly, FC appears to offer bespoke study approach in which residents have liberty and flexibility to study before the shared teaching session [14].   Secondly, the attention span in a lecture based education attends to be low, usually 10-20 minutes from the beginning of the lecture [15]. Thirdly, the residents had the liberty of accessing other relevant websites to develop deeper understanding of the topics before attending the classroom which is usually not possible in the didactics.  Fourthly, the residents in the flipped classroom actually spend more time in acquiring the knowledge as compared to F2F teaching. Our findings of enhanced knowledge acquisition and retention through FC methodology are consistent with previous studies [16, 17].

In our study of the FC approach, the residents expressed greater engagement, more satisfaction, decreased boredom and active involvement during the didactic component of the FC methodology. The residents felt that in FC approach, they were able to retain more information and this was reflected in the results of the MCQ test scores, immediately and at 2 months. Our study results are similar to the results of higher satisfaction reported in the radiology teaching for medical students undergoing the FC teaching approach [18]. An increasing number of educational institutions outside medicine have been utilizing FC approach of teaching with successful outcomes [19, 20].

We have identified several limitations to this study. The sample size was small. Only twenty nine residents from one teaching hospital participated.  Our FC approach may not be generalizable as each FC approach is dependent on the needs of the learners, faculty preferences and locally available resources. However the present study was conducted in the largest teaching hospital in the country. There was no randomization in the selection of study population. The FC method required additional work for the residents at home which ranged between 15-60 minutes.  We have only assessed the acquisition of theoretical knowledge and not its translation into clinical skills. However, it could be argued that neither FC nor the traditional F2F teaching approaches may be best suitable for attaining the POCUS skills. MCQ test were used for the assessment and retention of knowledge. For the evaluation of critical and higher level thinking skills as described by Blooms taxonomy, the assessment should have included short answers, and a coursework which would involve additional work for the residents and the faculty [21].

In spite of the limitations in our study, FC appears to be a popular method in acquisition and retention of knowledge related to different aspects of POCUS. We recommend future studies through utilizing FC approach to evaluate the acquisition of knowledge and skills related to POCUS. The future studies should include outcome based on Kirkpatrick’s classification of higher level thinking measures- perception, attitude and alteration in knowledge and skill sets.

Conclusion

Both F2F and FC teaching methods resulted in significant and sustained improvements in POCUS knowledge base. The FC teaching method was associated with even more improvement than the F2F teaching method, as assessed both at the conclusion of the teaching and after two months of completing the educational program. While there are potential confounders (e.g. the order of teaching methods was not randomized), the results strongly support movement to a FC teaching for at least some material; the FC approach is certainly no worse, and appears significantly better, than traditional F2F teaching. Residents appear to have a preference for FC method of instruction (Table 2).

 

References

  1. Lewiss RE, Hoffmann B, Beaulieu Y, et al. Point-of-care ultrasound education: the increasing role of simulation and multimedia resources. J Ultrasound Med. 2014;33(1):27-32.
  2. Mehta NB, Hull AL, Young JB, et al. Just imagine: new paradigms for medical education. Acad Med. 2013;88(10):1418-23.
  3. Freeman S, Eddy SL, McDonough M, et al. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A. 2014;111(23):8410-5.
  4. Prober CG, Heath  C. Lecture Halls without Lectures — A Proposal for Medical Education. N Engl J Med. 2012;366(18):1657-9.
  5. Twenge JM. Generational changes and their impact in the classroom: teaching Generation Me. Med Educ. 2009;43(5):398-405.
  6. Evans KH, Ozdalga E, Ahuja N. The Medical Education of Generation Y. Acad Psychiatry. 2016;40(2):382-5.
  7. Pettit RK, McCoy L, Kinney M. What millennial medical students say about flipped learning. Advances in medical education and practice. 2017;8:487-97.
  8. Hawks SJ. The flipped classroom: now or never? AANA J. 2014;82(4):264-9.
  9. Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in medical education. Med Educ. 2017;51(6):585-97.
  10. Evans KH, Thompson AC, O’Brien C, et al. An Innovative Blended Preclinical Curriculum in Clinical Epidemiology and Biostatistics: Impact on Student Satisfaction and Performance. Acad Med. 2016;91(5):696-700.
  11. McLaughlin JE, Griffin LM, Esserman DA, et al. Pharmacy student engagement, performance, and perception in a flipped satellite classroom. Am J Pharm Educ. 2013;77(9):196.
  12. Whillier S, Lystad RP. No differences in grades or level of satisfaction in a flipped classroom for neuroanatomy. The Journal of chiropractic education. 2015;29(2):127-33.
  13. Missildine K, Fountain R, Summers L, et al. Flipping the classroom to improve student performance and satisfaction. J Nurs Educ. 2013;52(10):597-9.
  14. Lin Y, Zhu Y, Chen C, et al. Facing the challenges in ophthalmology clerkship teaching: Is flipped classroom the answer? 2017;12(4):e0174829.
  15. Stuart J, Rutherford RJ. Medical student concentration during lectures. Lancet. 1978;2(8088):514-6.
  16. Lew EK. Creating a contemporary clerkship curriculum: the flipped classroom model in emergency medicine. Int J Emerg Med. 2016;9(1):25.
  17. Tune JD, Sturek M, Basile DP. Flipped classroom model improves graduate student performance in cardiovascular, respiratory, and renal physiology. Advances in physiology education. 2013;37(4):316-20.
  18. O’Connor EE, Fried J, McNulty N, et al. Flipping Radiology Education Right Side Up. Acad Radiol. 2016;23(7):810-22.
  19. Foertsch J, Moses G, Strikwerda J, et al. Reversing the Lecture/Homework Paradigm Using eTEACH® Web-based Streaming Video Software. J Eng Educ. 2002;91(3):267-74.
  20. Lage MJ, Platt GJ, Treglia M. Inverting the Classroom: A Gateway to Creating an Inclusive Learning Environment. J Econ Educ. 2000;31(1):30-43.
  21. Jones KO, Harland J, Reid JMV, et al., editors. Relationship between examination questions and bloom’s taxonomy. 2009 39th IEEE Frontiers in Education Conference; 2009 18-21 Oct. 2009.

 

Posted in .

Leave a Reply

Your email address will not be published. Required fields are marked *