Mariela Skendi , MD, MSc^1,2; Roxane Liard , MD, MSc¹; Charlotte Besacier , MD¹; Jean-Michel Correas , MD, PhD²; Sohela Moussaoui , MD, MSc¹; Julie Chastang , MD, MSc¹; Gladys Ibanez , MD, PhD^1,3

(1) Department of General Practice, Faculty of Medicine, Sorbonne University, Paris, France

(2) Department of Adult Radiology, Necker Hospital, Paris, France

(3) INSERM, Institut Pierre Louis d’Epidémiologie et de Santé Publique (IPLESP) Paris, France

---

Download article PDF – POCUS Journal 2022; 7(2):225-231.

DOI: https://doi.org/10.24908/pocus.v7i2.15458

---

Abstract 

Objective: The main objective of this study is the evaluation of the accuracy and reliability of a handheld point of care ultrasound device (POCUS-hd) for intrauterine pregnancy (IUP) detection compared to comprehensive reference transabdominal ultrasound (TU). The secondary objectives were to evaluate POCUS-hd for intrauterine pregnancy (IUP) detection compared to transabdominal and transvaginal ultrasound (TUTV), evaluate the inter-device agreement and inter-rater reliability of gestational age during early pregnancy. Methods: It is an observational transverse study with consecutive patient recruitment. Two blinded operators systematically used POCUS-hd and reference transabdominal ultrasound for IUP diagnosis. The accuracy of POCUS-hd for IUP diagnosis was expressed as sensitivity, specificity, negative predictive value (NPV) and positive predictive value (PPV). The gestational age (GA) was assessed based on the crown-rump length. The reliability and agreement of gestational age evaluation were assessed by Bland-Altman plots, kappa statistic, and intraclass correlation coefficients (ICC). Results: POCUS-hd compared to TU had a sensitivity of 95-100%, a specifcity of 90-100%, PPV of 95-100%  and NPV of 90-100%. Inter-rater agreement for IUP detection using POCUS-hd was very good, kappa=1.0; CI95% [0.9-1.0]. The inter-device agreement limits (mean difference ± 2SD) for GA were: -3 to +2.3 days by Operator 1, -3.4 to +3.3 days by Operator 2 for POCUS-hd vs. TU and  -3.1 to +2.3 days for POCUS-hd versus TUTV. Conclusion: This handheld POCUS device is an accurate and reliable diagnostic tool that can be used for IUP positive findings and GA assessment during early pregnancy by clinicians in family planning settings or general practice.

Background

The past decade saw the development and increased popularity of new point of care ultrasound (POCUS) devices. These portable devices have very fast start times, when compared to conventional ultrasound machines, and enable clinicians to perform POCUS at the bedside in clinical units. The further miniaturization of the machines gave birth to a new concept – “echoscopy” – defined in 2013 by the European Federation of Societies for Ultrasound in Medicine and Biology (EFSUMB) [1, 2] as part of three levels of ultrasound: echoscopy, POCUS and comprehensive ultrasound. While POCUS highlights the setting where the ultrasound exam is performed, echoscopy is defined by its ability to answer a simple targeted clinical question asked by the clinician at the bedside that can be documented in the patient’s chart and does not require a detailed imaging report. It is the intention of the clinician and the need to answer a specific clinical question that defines the type of ultrasound performed. Should the physician wish to perform a detailed exam to explore a region of organs, sophisticated devices used for comprehensive ultrasound machines are better adapted. Echoscopy and POCUS can be performed with handheld devices while comprehensive ultrasound with more sophisticated equipment [2].

For patients presenting lower abdominal pain and/or vaginal hemorrhage in early pregnancy, it is important for the clinician both to confirm the presence of an intrauterine pregnancy (IUP) as well as estimate the gestational age (GA), for clinical decision making later in pregnancy or voluntary pregnancy termination [3]. Before using these new miniaturized ultrasound devices to answer such clinical questions in everyday practice, it is important to evaluate their accuracy and reliability compared to a high-end system.

The principal objective was the evaluation of the accuracy and reliability of  handheld point of care ultrasound device (POCUS-hd) for IUP detection compared to comprehensive reference transabdominal ultrasound (TU). The secondary objectives were to evaluate POCUS-hd for IUP detection compared to transabdominal and transvaginal ultrasound (TUTV), evaluate the inter-device agreement and inter-rater reliability in calculating GA in the first trimester of pregnancy.

Methods

Study Design

This was an observational transverse monocentric study conducted according to the STARD and GRRAS guidelines for accuracy and reliability [4, 5]. All studies were performed at the Family Planning Clinic at the Cochin Port-Royal University Hospital in Paris, France.

The first part of the study compared the accuracy of a POCUS-hd compared to TU and TUTV for the detection of IUP and the inter-operator agreement for IUP detection.

The second part of the study evaluated the inter-device agreement on GA measurement. The GA obtained using POCUS-hd was compared to the TU measurement. The inter-rater variability for GA measurement was then calculated for each operator. Two blinded operators scanned independently the same population of patients in alternating order, on the same day at 5-10 minutes intervals. Operator 1 performed POCUS followed by TU for all patients and a TUTV for all pregnancies younger than 6 weeks of gestation or whenever the embryo could not be visualized transabdominally according to the usual practice at the clinic. Both POCUS and the reference comprehensive ultrasound were performed on the same day during the patient’s visit to the family planning clinic.

Operator 2 performed POCUS-hd followed only by a TU. At the end of each study, both operators would fill a written report and would communicate the results of their respective scan to the patient. Patients had no specific preparation for the study, like fasting or full bladder requirement.

IUP definition and GA calculation

The presence of an IUP was confirmed by the visualization of the double decidual sac sign on B-mode with either an embryo or a yolk sac.

The GA was assessed based on the crown-rump length (CRL) using the following equation [11]:  gestational age (days) = 8.052*(1.037*CRL)^1/2+23.73.

The mean diameter of the gestational sac was not used for GA calculation due to its higher variability and less precise GA estimation [6-12]. Basic settings were used such as gain, depth, zoom, and use of calipers for measurements of CRL.

Population

We aimed to recruit 65 consecutive patients who visited the Family Planning Clinic at the Cochin Port-Royal University Hospital between May and July 2016. Among this population, pregnancy was either confirmed or suspected. Patients would come in with a positive urinary pregnancy test, a positive plasmatic beta human chorionic gonadotropin (hCG), a delay in the onset of menses, abdominal pain with/or vaginal bleeding and for a follow-up visit to confirm pregnancy termination. Patients were included if they were at least 18 years of age. Patients were excluded from the study if they were under 18 years, had twin gestations, refused to sign a consent form or if the image acquisition was incomplete due to technical difficulties with the ultrasound machine.

Operators

Operator 1 was a general practitioner (GP) who had been working and using ultrasound at the family planning clinic for 5 years. Operator 2 was a GP who had a general 2-year ultrasound diploma and had finished a 6month training at the family planning clinic.

Ultrasound Device

POCUS was performed with a handheld Visiq Philips device that weighed 1 kg, had an average start time of 30 seconds and was connected to a C5-2°MHz transducer through USB port. The comprehensive reference ultrasound was performed on the ProSound Alpha 6 machine using UST-9123 6-2 MHz and UST-9124 7.5-3 MHz transducers.

Data Storage and Interpretation

Images obtained by POCUS-hd were stored as DICOM files on the Visiq Philips ultrasound device. Images obtained by the transabdominal ultrasound machine were stored on the ProSound Alpha 6 machine. Both operators recorded their findings and image interpretations on paper files (Figure 1).

Statistical Analysis

The accuracy of POCUS-hd was calculated in terms of sensitivity, specificity, negative predictive value and positive predictive value using contingency tables for Operators 1 and 2. Inter-rater agreement on IUP detection was evaluated by the kappa statistic. Inter-device agreement for GA evaluation was calculated using Bland-Altman plots. Inter-rater variability for GA measurement with POCUS-hd was calculated using the intraclass correlation coefficient (ICC) and Bland-Altman plots [12]. Data analysis was performed by the Department of General Practice at Sorbonne University, using Stata and R Studio software.

Results

Among the 65 eligible women, 57 were enrolled in the study (Table 1). On standard transabdominal ultrasound, there were 37 IUPs detected, among whom 34 had visible embryos, 3 had gestational sacs with yolk sacs according to POCUS-hd. On TUTV there were 45 IUP detected, among whom 41 had visible embryos, 4 had gestational sacs with yolk sacs according to TUTV (Figure 2, Figure S1 suppl).

Table 1. Patient characteristics (n=57)

Accuracy

POCUS-hd accuracy was calculated through contingency tables. The sensitivity of POCUS-hd for IUP detection was 95-100% (35/37 for Operator 1 and 37/37 for Operator 2) when compared to TU alone. The specificity for POCUS-hd for IUP detection was 90-100% (18/20 for Operator 1 and 20/20 for Operator 2) when compared to TU alone. The positive predictive value (PPV) was 95100% (35/37 Operator 1, 37/37 Operator 2). The negative predictive value (NPV) was 90-100% (18/20 Operator 1, 20/20 Operator 2).

The sensitivity of POCUS-hd for IUP detection was 82% (37/45 by Operator 1) when compared to TUTV. The specificity for POCUS-hd for IUP detection was 100% (20/20 by Operator 1) when compared to TUTV. The PPV was 100% (37/37) and NPV was 60% (12/20) where the reference ultrasound was TUTV.

Inter-rater agreement for IUP detection by POCUS-hd was excellent, kappa=1.0; CI_95% [0.9-1.0] (Table 2).

Table 2. Diagnostic accuracy of echoscopy for intrauterine pregnancy detection compared to comprehensive ultrasound (n=57).

Reliability

Agreement limits of POCUS-hd vs. TU (mean difference ± 2SD) were -3.0 to +2.3 days for Operator 1 and -3.4 to +3.3 days for Operator 2. Agreement limits of POCUS-hd vs. TUTV were -3.1 to +2.3 days. The inter-device agreement (POCUS-hd vs. TU and POCUS-hd vs. TUTV) for GA estimation was very good (Figure 3).

The inter-rater agreement of GA by POCUS-hd was excellent, ICC = 0.99, CI 95% [0.98 – 0.99] and agreement limits on the Bland-Altman plot were -2.7 to +3 days (Figure 4).

Discussion

The handheld POCUS device, compared to transabdominal ultrasound and transabdominal ultrasound completed with transvaginal, was highly accurate for IUP detection and GA assessment. The study showed excellent agreement of POCUS-hd versus TU and POCUS-hd versus TUTV for GA measurement as agreement limits of POCUS-hd  [+/-3 days] are within the precision limits of ultrasound dating of [+/-5 days] days used in clinical practice [6, 13]. The reproducibility of gestational age measurements by POCUS-hd between the 2 operators was very good.

POCUS-hd is intended to be used by clinicians who need to determine the location of a pregnancy in the first trimester at the bedside during the clinical examination. They may then, depending on their abilities, date the pregnancy if needed [7]. In the event of a negative result where intrauterine pregnancy cannot be confirmed, it is up to the clinician to decide whether to continue the investigations and within what time frame to repeat POCUS, or request a comprehensive ultrasound. Of the 65 patients recruited for this study, 8 were excluded because they were under 18 years of age. The remaining 57 patients, all consented to be part of the study.

To this day, only a few studies have evaluated the accuracy of a handheld ultrasound device for routine obstetrical examination during early pregnancy but there is no study that evaluates their accuracy and reliability in a context of pregnancy termination. A systematic review published in 2019 by Rykkje et al. comparing hand-held ultrasound devices with high-end ultrasound showed a good overall agreement for obstetrics and gynecology use. The results of our study with a Visiq Philips handheld device are comparable to the results of 3 obstetrics/gynecology studies during the first semester at an emergency setting in terms of reliability where a Vscan was used [14]. One of the strengths of this study is to evaluate another handheld device, Visiq/Philips, in a context of pregnancy termination in a family planning clinic.

Sayasneh et al. evaluated the validity of a POCUS-hd device, Vscan, in a population of 101 patients with signs of pelvic pain or hemorrhage during their first trimester. There was “good” to “very good” concordance between the Vscan and the transabdominal and transvaginal ultrasound for the detection of an embryo, a gestational sac, cardiac activity, with kappa coefficients of 0.844, 0.843 and 0.729, respectively (p <0.0001). The concordance for CRL and mean diameter of the gestational sac measurement was very good with an ICC > 0.9 (p <0.0001) [15]. These results are in agreement with the results of our study covering 37 women which found a good concordance between POCUS-hd and the transabdominal ultrasound to measure gestational age based on crown-rump length, ICC = 1.0 (p <0.0001).

Several studies on the different fields of application of POCUS suggest that its greatest potential and impact on morbidity and mortality is in obstetrics. A study on the use of POCUS by midwives in Zambia, for example, showed that they can be trained to perform POCUS, answer simple obstetric clinical questions, and impact clinical decision-making [16-19].

One of the limitations of this study is that it does not assess intra-operator variability. This possibility was discussed during the design of the study but additional measures would have extended the duration of the examination and might have become uncomfortable for patients. For this reason and for patients’ comfort, transvaginal ultrasound was not repeated by Operator 2 but was rather performed only once by Operator 1 as part of the usual practice.

Among obstetric studies, bedside ultrasound is easily accepted and allows accurate monitoring of pregnancy after 5 weeks of gestation. Pelvic pain in early pregnancy may be secondary to an ectopic pregnancy in the absence of a uterine gestational sac and the presence of an adnexal mass or intraperitoneal free fluid. The results of a meta-analysis on the diagnosis of ectopic pregnancy by bedside ultrasound performed by emergency physicians show a high specificity and high sensitivity in the localization of a pregnancy but remain operator-dependent [17, 20-23].

The results of our study showed that the diagnostic performance of POCUS-hd to detect intrauterine pregnancy was satisfactory and could be used in the family planning clinic. The reliability of a handheld POCUS-hd device to evaluate GA during the first trimester was comparable to conventional ultrasound with an accuracy of +/- 3 days. The population in this study included adult women in the first trimester of their pregnancy. However, the study may be extended to other populations. POCUS-hd can also be used in other clinical situations such as confirming the proper positioning of an intrauterine device. In the context of gynecological emergencies, it can assess the viability of the pregnancy. POCUS-hd can also be used to diagnose other (non-obstetric) pathologies such as a pelvic mass or intraperitoneal free fluid. Its usefulness in cardiac, renal, vascular, hepatosplenic and musculoskeletal pathologies has been well-established.

As technology progresses, both image resolution and POCUS-hd affordability will undoubtedly improve. In the near future, physicians and medical students will be equipped more easily and educators will teach ultrasound skills during medical school or during continuing medical education activities.

Conclusions

This handheld POCUS device seems to be an accurate and reliable diagnostic tool that can be used for IUP detection and GA assessment during early pregnancy by clinicians in the family planning setting or general practice. These portable devices enable clinicians to perform POCUS at the bedside in clinical units, help improve the accuracy of the physical exam and improve patient care.

Ethics statement and consent

Patients were informed before the study orally and in writing. All patients signed a written consent form before participating in the study. In conformity with  the French regulation, authorizations for the study were obtained by the institution of Advisory Committee on the Processing of Research Information (CCTIRS) and National Data Protection Commission (CNIL) #2070527. Considering the study did not change the usual practice and it did not involve any risk for the patients, a formal ethical approval was not required. Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Funding

The Visiq ultrasound equipment was supplied by Philips on a temporary loan for the study duration, at the request of the author (MS). Philips had no role in study design, data collection, analysis, decision to publish, or preparation of the manuscript.

Acknowledgements

We thank Grace Shaloub for her help on the manuscript.

References

1. Moore CL, Copel JA. Point-of-Care Ultrasonography. New England Journal of Medicine. 2011; 364:749-57.

2. Birth of Echoscopy- The EFSUMB Point of View. Ultraschall in der Medizin – European Journal of Ultrasound. 2013; 34:92-92.

3. Diagnostic Ultrasonography in Women’s Health Care (Position Paper) [Internet]. http://www.aafp.org/about/policies/all/ultrasonography-diagnostic.html [cited 2016 Jul 26]

4. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HC; STARD Group. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Fam Pract. 2004; 21:4-10.

5. Kottner J, Audigé L, Brorson S, Donner A, Gajewski BJ, Hróbjartsson A, Roberts C, Shoukri M, Streiner DL. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol. 2011; 64:96-106.1.

6. Salomon LJ, Alfirevic Z, Bilardo CM, Chalouhi GE, Ghi T, Kagan KO, Lau TK, Papageorghiou AT, Raine-Fenning NJ, Stirnemann J, Suresh S, Tabor A, Timor-Tritsch IE, Toi A, Yeo G. ISUOG practice guidelines: performance of first-trimester fetal ultrasound scan. Ultrasound Obstet Gynecol. 2013; 41:102-13.

7. Lavoué V, Huchon C, Deffieux X, Voltzenlogel MC, Vandenbroucke L, Levêque J. French guidelines for diagnostic criteria for nonviable pregnancy early in the first trimester and for management of threatened miscarriage and nonviable pregnancy (except uterine evacuation)]. J Gynecol Obstet Biol Reprod (Paris). 2014 Dec;43(10):776-93. French. doi: 10.1016/j.jgyn.2014.09.012. Epub 2014 Nov 6. PMID: 25447361.

8. Delabaere A, Huchon C, Lavoue V, Lejeune V, Iraola E, Nedellec S, Gallot V, Capmas P, Beucher G, Subtil D, Carcopino X, Vialard F, Nizard J, Quibel T, Costedoat-Chalumeau N, Legendre G, Venditelli F, Rozenberg P, Lemery D, Deffieux X. Definition of pregnancy losses: Standardization of terminology from the French National college of Obstetricians and Gynecologists (CNGOF). J Gynecol Obstet Biol Reprod (Paris) 2014, Dec; 43(10):756-63.

9. Kaur A, Kaur A. Transvaginal ultrasonography in first trimester of pregnancy and its comparison with transabdominal ultrasonography. J Pharm Bioallied Sci. 2011; 3:329-38.

10. Robinson HP, Sweet EM, Adam AH. The accuracy of radiological estimates of gestational age using early fetal crown-rump length measurements by ultrasound as a basis for comparison. Br J Obstet Gynaecol. 1979; 86:525-528.

11. Salomon LJ. [How to date pregnancy?]. J Gynecol Obstet Biol Reprod (Paris). 2011; 40:726–33.

12. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;  1:307-10.

13. Butt K, Lim K, Society of Obstetricians and Gynaecologists of Canada. Determination of gestational age by ultrasound. J Obstet Gynaecol Can. 2014; 36:171-83.

14. Rykkje, A.; Carlsen, J.F.; Nielsen, M.B. Hand-Held Ultrasound Devices Compared with High-End Ultrasound Systems: A Systematic Review. Diagnostics 2019; 9:61.

15. Sayasneh A, Preisler J, Smith A, Saso S, Naji O, Abdallah Y, Stalder C, Daemen A, Timmerman D, Bourne T. Do pocket-sized ultrasound machines have the potential to be used as a tool to triage patients in obstetrics and gynecology? Ultrasound Obstet Gynecol. 2012; 40:145-50.

16. Richardson A, Hopkisson J, Campbell B, Raine-Fenning N. Use of the double decidual sac sign to confirm intra-uterine pregnancy location prior to ultrasonographic visualisation of embryonic contents: a diagnostic accuracy study. Ultrasound Obstet Gynecol. 2016; 49:643-648.

17. Dietrich CF, Goudie A, Chiorean L, Cui XW, Gilja OH, Dong Y, Abramowicz JS, Vinayak S, Westerway SC, Nolsøe CP, Chou YH, Blaivas M. Point of Care Ultrasound: A WFUMB Position Paper. Ultrasound Med Biol. 2016; 43:49-58.

18. Kimberly HH, Murray A, Mennicke M, Liteplo A, Lew J, Bohan JS, Tyer-Viola L, Ahn R, Burke T, Noble VE. Focused maternal ultrasound by midwives in rural Zambia. Ultrasound Med Biol. 2010; 36:1267–72.

19. Liu L, Hill K, Oza S, Hogan D, Chu Y, Cousens S, Mathers C, Stanton C, Lawn J, Robert BE. Levels and Causes of Mortality under Age Five Years. In: Black RE, Laxminarayan R, Temmerman M, Walker N, editors. Reproductive, Maternal, Newborn, and Child Health: Disease Control Priorities, Third Edition (Volume 2) [Internet]. Washington (DC): The International Bank for Reconstruction and Development / The World Bank; [2016 Sep 28]. https://doi.org/10.1596/978-1-4648-0348-2_ch4

20. Lijmer JG, Mol BW, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JH, Bossuyt PM. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999; 282:1061–6.

21. Stein JC, Wang R, Adler N, Boscardin J, Jacoby VL, Won G, Goldstein R, Kohn MA. Ann Emerg Med. 2010; 56:674–83.

22. Saul T, Lewiss RE, Rivera MDR. Accuracy of emergency physician performed bedside ultrasound in determining gestational age in first trimester pregnancy. Crit Ultrasound J. 2012; 4:22.

23. Bailey C, Carnell J, Vahidnia F, Shah S, Stone M, Adams M, Nagdev A. Accuracy of emergency physicians using ultrasound measurement of crown-rump length to estimate gestational age in pregnant females. Am J Emerg Med. 2012; 30:1627–9.