Advertisement

Personal protective equipment in Covid-19: Evidence-based quality and analysis of YouTube videos after one year of pandemic

Published:November 26, 2021DOI:https://doi.org/10.1016/j.ajic.2021.11.013

      Highlights

      • Using the personal protective equipment (PPE) in wrong ways may lead to higher contagion risks.
      • YouTube is a social media sharing information on several medical issues.
      • Increased interest on donning and doffing of PPE was observed on the worldwide web.
      • Overall reliability and quality of YouTube videos on this topic were reasonable.
      • YouTube may be recommended as a reliable source on correct usage of PPE for Covid-19 patients care.

      Abstract

      Background

      The correct use of personal protective equipment (PPE) during the Covid-19 pandemic is mandatory to minimize the contagion risk. The current study aimed to evaluate quality information of YouTube videos on PPE use during the pandemic.

      Methods

      Using Google Trend tool, the frequency of worldwide YouTube and Google searches for “donning and doffing” was examined. We queried YouTube with terms related to donning and doffing of PPE. Validated quality information assessment tools were used.

      Results

      From the December 1, 2019 to the January 31, 2021, according to YouTube and Google searches, both peaks occurred in April 2020 (69.5% and 72.0%, respectively). Of all videos, 144 were eligible for the analyses. According to misinformation tool, 90 (62.5%) videos contained inaccuracies. The median DISCERN Section 1 ranged from 3 to 5. The median DISCERN Section 3 was 4. According to Global Quality Score, 8.3% (n = 12), 14.6% (n = 21), 22.9% (n = 33), 30.6% (n = 44) and 23.6% (n = 34) were classified as poor, partially poor, moderate, partially good and excellent quality videos, respectively.

      Conclusions

      Nowadays, YouTube may be recommended as a reliable source of information. Nevertheless, a not negligible number of videos contained inaccuracies. Future authors should improve videos contents to provide more complete information.

      Key words

      Introduction

      More than 17,000 health care providers worldwide have lost their lives due to Covid-19 infection and its complications, since the beginning of the pandemic.

      COVID-19: Health worker death toll rises to at least 17000 as organizations call for rapid vaccine rollout | Amnesty International [Internet]. 2021 [cited 2021 Mar 16]. https://www.amnesty.org/en/latest/news/2021/03/covid19-health-worker-death-toll-rises-to-at-least-17000-as-organizations-call-for-rapid-vaccine-rollout/. Accessed March 16, 2021

      ,

      WHO Director-General's opening remarks at the media briefing on COVID-19 - 11 March 2020 [Internet]. 2021 [cited 2021 Mar 16]. https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19-11-march-2020. Accessed March 16, 2021

      Currently, one year after the outbreak declaration, it is estimated that approximately 120 million people got infected and more than 2.5 million died in 223 different countries.

      Coronavirus disease (COVID-19) – world health organization [internet]. 2021 [cited 2021 Mar 16]. https://www.who.int/emergencies/diseases/novel-coronavirus-2019. Accessed March 16, 2021

      ,

      Coronavirus Update (Live): 120,822,563 Cases and 2,673,622 deaths from COVID-19 virus pandemic - worldometer [Internet]. 2021 [cited 2021 Mar 16]. https://www.worldometers.info/coronavirus/. Accessed March 16, 2021

      The transmission mechanisms of Covid-19 have been largely studied by the scientific community.

      COVID-19 transmission—up in the air.:1. 2021

      • Meyerowitz EA
      • Richterman A
      • Gandhi RT
      • Sax PE.
      Transmission of SARS-CoV-2: a review of viral, host, and environmental factors.
      CDC
      COVID-19 and Your Health [Internet].
      • Creta M
      • Sagnelli C
      • Celentano G
      • et al.
      SARS-CoV-2 infection affects the lower urinary tract and male genital system: a systematic review.
      In this regard, international official recommendations primarily focused on the correct use of personal protective equipment (PPE), with the aim to break down the viral spreading.
      CDC
      Healthcare Workers [Internet].

      Rational use of personal protective equipment for coronavirus disease (COVID-19) and considerations during severe shortages [Internet]. 2021 [cited 2021 Mar 16]. https://www.who.int/publications/i/item/rational-use-of-personal-protective-equipment-for-coronavirus-disease-(covid-19)-and-considerations-during-severe-shortages. Accessed March 16, 2021

      Guidance for wearing and removing personal protective equipment in healthcare settings for the care of patients with suspected or confirmed COVID-19 [Internet]. European Centre for Disease Prevention and Control. 2020 [cited 2021 Mar 16]. Accessed March 16, 2021. https://www.ecdc.europa.eu/en/publications-data/guidance-wearing-and-removing-personal-protective-equipment-healthcare-settings

      However, the use of PPE in health care settings may be highly dangerous if the staff executing the donning and doffing procedures is not adequately skilled and trained to the purpose.
      • Wang J
      • Zhou M
      • Liu F.
      Reasons for healthcare workers becoming infected with novel coronavirus disease 2019 (COVID-19) in China.
      Over the years, online medical resources and instruments have increased. YouTube is the main free video platform, containing over 100 million videos and more than 1.9 billion users in over 100 different countries.

      Press - YouTube [Internet]. 2021 [cited 2021 Mar 16]. Accessed March 16, 2021. https://www.youtube.com/intl/en-GB/about/press/

      ,

      youtube.com Competitive Analysis, Marketing Mix and Traffic - Alexa [Internet]. 2021 [cited 2021 Mar 16]. Accessed March 16, 2021. https://www.alexa.com/siteinfo/youtube.com

      YouTube, as well as other social medias,
      • Lim K
      • Kilpatrick C
      • Storr J
      • Seale H.
      Exploring the use of entertainment-education YouTube videos focused on infection prevention and control.
      • Gandhi CK
      • Patel J
      • Zhan X.
      Trend of influenza vaccine Facebook posts in last 4 years: a content analysis.
      • Meadows CZ
      • Tang L
      • Liu W.
      Twitter message types, health beliefs, and vaccine attitudes during the 2015 measles outbreak in California.
      • Odlum M
      • Yoon S.
      What can we learn about the Ebola outbreak from tweets?.
      • Tang L
      • Bie B
      • Park S-E
      • Zhi D.
      Social media and outbreaks of emerging infectious diseases: a systematic review of literature.
      is commonly used as a source of medical information and it is successfully used as a learning and self-training tool, among health care workers as well.
      • Tolu S.
      English-language videos on YouTube as a source of information on self-administer subcutaneous anti-tumour necrosis factor agent injections.
      ,
      • Aldallal SN
      • Yates JM
      • Ajrash M.
      Use of YouTubeTM as a self-directed learning resource in oral surgery among undergraduate dental students: a cross-sectional descriptive study.
      Previous studies have already evaluated the quality of the videos concerning several medical fields.
      • Garg N
      • Venkatraman A
      • Pandey A
      • Kumar N.
      YouTube as a source of information on dialysis: a content analysis: online information on dialysis.
      • Kumar N
      • Pandey A
      • Venkatraman A
      • Garg N.
      Are video sharing Web sites a useful source of information on hypertension?.
      • Capece M
      • Di Giovanni A
      • Cirigliano L
      • et al.
      YouTube as a source of information on penile prosthesis.
      • Morra S
      • Collà Ruvolo C
      • Napolitano L
      • et al.
      YouTubeTM as a source of information on bladder pain syndrome: a contemporary analysis.
      To the best of our knowledge, no previous investigators examined the quality information of YouTube videos on the use of PPE for Covid-19 patients care. The aim of the current study was to evaluate the evidence-based quality of YouTube videos on the correct donning and doffing procedures and usage of PPE at one year from the beginning of Covid-19 pandemic.

      Materials and methods

      Web interest assessment

      We evaluated the interest of the worldwide web users on the donning and doffing procedures. We queried Google Trends

      Google trends [Internet]. Google Trends. 2021 [cited 2021 Mar 30]. Accessed December 15, 2021. https://trends.google.it/trends/?geo=IT

      with the terms “donning and doffing,” using the following search settings: “worldwide,”, “period from December 1, 2019 to January 31, 2021,” “all the categories.” The trends of Google search and YouTube search were independently recorded. To examine changes in search frequency over time, we compared the relative frequency of searches between all 14 months (December 2019-January 2021).

      Search strategy, selection criteria, and videographic characteristics

      We queried YouTube

      YouTube [Internet]. 2021 [cited 2021 Mar 30]. Accessed December 15, 2021. https://www.youtube.com/

      with 40 combinations of keywords (Supplementary Table 1). Internet cache cleared, the log-out from any personal account and a proxy located in United Kingdom through a free VPN software were set to minimise the search history and the geographic related biases.
      The first 50 videos were examined for each of the 40 keywords combinations. A total of 2,000 videos was achieved. The following exclusion criteria were applied: non-English language, live-actions, interviews, news reports, webinars or lectures nontopic related, non-medical content, non-donning/doffing procedures and duplicate videos. Only the videos published after the December 1, 2019 were included. A total of 144 videos were eligible for the analyses (Fig 1).
      Fig 1
      Fig 1PRISMA diagram depicting inclusion and exclusion criteria of YouTube videos search.
      For each of the 144 videos included, the following variables were collected on the 25th of February 2021: length (seconds), views, persistence time on YouTube (days), thumbs-up, thumbs-down, subscribers, authoring entity (public health institutions [such as public and government health institutions and associations, general health institutions], academic hospitals [such as academic hospitals and academic institutions, universities], nonacademic hospitals [such as nonacademic hospitals and institutions, health care centers, private practice hospitals], alternative medicine channels [such as digital learning platforms, health information websites], individuals and others [such as news channels and PPE producers companies]) and target audience (health care workers, general public, and visitors).

      Strategies and instruments for the assessment of videos content

      Videos contents were assessed independently by 2 medical doctors, both staff members in the “Intermediate Care Covid-19 Unit” of the University of Naples “Federico II.” A third investigator, a medical doctor staff member from the same Unit, adjudicated any differences and consensus was achieved among all reviewers.

      Videos content assessment according to the centers for disease control and prevention (CDC) recommendations

      Videos contents were evaluated according to the CDC recommendations,
      CDC
      Healthcare workers [Internet].
      in which the donning and the doffing procedures were reported as step-by-step checklists (Supplementary Table 2). For each of the donning and doffing steps, 5 possible scores from 1 (strongly disagree) to 5 (strongly agree) were assigned.

      Misinformation tool

      After evaluating the content of the videos according to CDC recommendations, the investigators judged if inaccuracies were present in reporting information. The inaccuracies reasons were categorized according to the following non-exclusive groups, as previously done
      • Huang MM
      • Winoker JS
      • ME Allaf
      • Matlaga BR
      • Koo K.
      Evidence-based quality and accuracy of YouTube videos about nephrolithiasis.
      : (1) Incomplete presentation of data; (2) Inconsistent with CDC recommendations; (3) Weak or inconsistent supporting data; (4) Misrepresentation of data; (5) False or without any supporting data (Supplementary Table 3).

      DISCERN instrument

      The DISCERN instrument is a validated tool used to evaluate the good quality evidence-based and the reliability of the information given.
      • Huang MM
      • Winoker JS
      • ME Allaf
      • Matlaga BR
      • Koo K.
      Evidence-based quality and accuracy of YouTube videos about nephrolithiasis.
      ,
      • Akyol A.
      Is YouTube a quality source of information on sarcopenia?.
      The original version of the DISCERN tool, made by The British Library, was used for the study.

      DISCERN - The DISCERN Instrument [Internet]. 2021 [cited 2021 Mar 30]. http://www.discern.org.uk/discern_instrument.php. Accessed March 30, 2021

      The instrument is based on a scale of 5 possible answers per question (1 = no; 2 = partially no; 3 = partially; 4 = partially yes; 5 = yes). DISCERN Section 1 (questions 1 to 8 aimed to evaluate if a publication is reliable) and Section 3 (question 16 aimed to evaluate the overall rating of the publication) were used. DISCERN Section 2 was not applicable to the current study, since it aimed to evaluate information on treatment choices. However, to date, the only possible preventive treatment available for people dealing with Covid-19 patients is the correct usage of PPE.

      Global quality score (GQS)

      The GQS is a validated tool assessing the quality, the feasibility and the clinical utility of each video.
      • Erdem MN
      • Karaca S.
      Evaluating the accuracy and quality of the information in kyphosis videos shared on YouTube.
      • Kocyigit BF
      • Akaltun MS.
      Does YouTube provide high quality information? Assessment of secukinumab videos.
      • Kocyigit BF
      • Akaltun MS
      • Sahin AR.
      YouTube as a source of information on COVID-19 and rheumatic disease link.
      It was firstly used to assess the internet quality content on Crohn's Disease and Ulcerative Colitis.
      • Bernard A
      • Langille M
      • Hughes S
      • Rose C
      • Leddin D
      • Veldhuyzen van Zanten S.
      A systematic review of patient inflammatory bowel disease information resources on the World Wide Web.
      The original English version of the GQS was used for the porpoise of the study. Five possible scores from 1 (poor quality, poor flow, most of information missing, not at all useful) to 5 (excellent quality, excellent flow, completely accurate information, very useful) were assigned.

      Statistical analysis

      Descriptive statistics were presented as medians and interquartile ranges (IQR) for continuously coded variables or counts and percentages for categorically coded variables. Chi-square test and Kruskal-Wallis test examined the statistical significance in proportions and medians differences. In all statistical analyses, R software (www.rproject.org) environment for statistical computing and graphics (R version 4.0.0) and Microsoft Excel 2019 were used. All tests were 2-sided with a level of significance set at P < .05.

      Results

      Web interest

      From the December 1, 2019 to the January 31, 2021, an increased user's interest on both Google and YouTube was recorded (Fig 2). Specifically, according to YouTube search, the relative search frequency ranged from 2.4 to 21.6%, with the peak recorded in April 2020 (69.5%). Similarly, according to Google search, the relative search frequency ranged from 3.2 to 22.6%, with the peak recorded in April 2020 (72.0%).
      Fig 2
      Fig 2Chart-line plot depicting relative frequency of worldwide search for “donning and doffing” on both Google and YouTube searches, observed between the December 1, 2019 and the January 31, 2021.

      Videographic characteristics

      Of 144 videos (Table 1), the median length was 298.5 seconds (IQR: 190.0-472.2; range: 25.0-2442.0), the median number of views was 4443.5 (IQR: 1118.8-19484.8; range: 6.0-2675484.0) and the median persistence time on YouTube was 314.0 days (IQR: 268.0-337.0; range: 50.0-381.0). Moreover, the median number of thumbs-up, thumbs-down and subscribers was 27.0 (IQR: 7.0-120.0; range: 0-44000.0), 2.0 (IQR: 0-7.0; range: 0-1600.0) and 9570.0 (IQR: 1000.0-42800.0; range: 10.0-2500000.0), respectively. Of all videos, 36.8% (n = 53), 12.5% (n = 18), 13.2% (n = 19), 22.2% (n = 32), 6.9% (n = 10), and 8.3% (n = 12), were produced by public health institutions, academic hospitals, nonacademic hospitals, alternative medicine channels, individuals and others, respectively. Additionally, 88.2% (n = 127), 10.4% (n = 15) and 1.4% (n = 2) were targeted to health care workers, general public and visitors, respectively.
      Table 1Videographic characteristics of 144 YouTube videos on donning and doffing of personal protective equipment, recorded on the 25th of February 2021
      Videographic characteristicsValue
      Length, secMedian (IQR)

      Range
      298.5 (190.0-472.2)

      25.0-2442.0
      Views, nMedian (IQR)

      Range
      4443.5 (1118.8-19484.8)

      6.0-2675484.0
      Persistence time, dMedian (IQR)

      Range
      314.0 (268.0-337.0)

      50.0-381.0
      Thumbs-up, nMedian (IQR)

      Range
      27.0 (7.0-120.0)

      0-44000.0
      Thumbs-down, nMedian (IQR)

      Range
      2.0 (0-7.0)

      0-1600.0
      Subscribers, nMedian (IQR)

      Range
      9570.0 (1000.0-42800.0)

      10.0-2500000.0
      Authoring entity, n (%)Public health institutions

      Alternative medicine channels

      Nonacademic hospital

      Academic hospitals

      Individuals

      Others
      53 (36.8)

      32 (22.2)

      19 (13.2)

      18 (12.5)

      10 (6.9)

      12 (8.3)
      Target audience, n (%)Healthcare workers

      General public

      Visitors
      127 (88.2)

      15 (10.4)

      2 (1.4)

      Videos content results

      Videos content results according to CDC recommendations

      According to the donning CDC recommendations step-by-step checklist, the median scores recorded ranged from 3 (for step 1) to 5 (for step 2). According to the doffing CDC recommendations step-by-step checklist, the median scores recorded ranged from 4 (for steps 2-5) to 5 (for steps 1 and 6) (Supplementary Fig 1).

      Misinformation tool results

      According to misinformation tool, 90 videos were defined as inaccurate. Specifically, 90.0% (n = 81), 50.0% (n = 45), 46.7% (n = 42), 35.6% (n = 32) and 24.4% (n = 22) were inaccurate for incomplete presentation of data, for inconsistent with CDC recommendations, for weak or inconsistent supporting data, for misrepresentation of data and for false or without any supporting data, respectively (Table 2).
      Table 2Videos containing inaccurate or non-evidence-based claims about donning and doffing of personal protective equipment (PPE) according to Centers for Disease Control and Prevention recommendations, recorded on 25th of February 2021
      Inaccuracy reasons or non-evidence-based claimsVideos, n (%)
      Incomplete presentation of data81 (90.0)
      Inconsistent with CDC recommendations45 (50.0)
      Weak or inconsistent supporting data42 (46.7)
      Misrepresentation of data32 (35.6)
      False or without any supporting data22 (24.4)

      DISCERN instrument results

      The median DISCERN for the Section 1 ranged from 3 to 5. Specifically, the highest median score was recorded for the question 5. Conversely, the lowest median score was recorded for the question 4, 7, and 8. The median DISCERN for the Section 3 was 4. Moreover, statistically significant differences were recorded when DISCERN Section 1 and Section 3 were compared according to the videos authoring entity. Specifically, videos produced by public health institutions obtained a median score ≥4 in 8 out of 9 questions. Conversely, videos produced by individuals obtained a median score ≤3 in 8 out of 9 questions (Table 3A).
      Table 3DISCERN instrument Section 1 and Section 3 (A) and Global Quality Score (B) of 144 videos recorded on the 25th of February 2021
      OverallPublic health institutions 53 (36.8%)Alternative medicine channels 32 (22.2%)Nonacademic hospitals 19 (13.2%)Academic hospitals 18 (12.5%)Others 12 (8.3%)Individuals 10 (6.9%)P-value
      a. DISCERN Instrument – Section 1 & Section 3
      1. Are the aims clear?4555543< .001
      2. Does it achieve its aims?4444432.5< .001
      3. Is it relevant?4444432< .001
      4. Is it clear what sources were used to compile the publication?3433322.001
      5. Is it clear when the information used or reported was produced?5555555.1
      6. Is it balanced and unbiased?4444432.5< .001
      7. Does it provide details of additional sources of support and information?3433322.5< .01
      8. Does it refer to areas of uncertainty?333332.52.06
      16. Based on the answer to all the above questions, rate overall quality of the publication4444432.5< .01
      b. Global Quality Score
      1. Poor quality12 (8.3)3 (5.7)2 (6.2)0 (0)0 (0)3 (25.0)4 (40.0)< .001
      2. Partially poor quality21 (14.6)11 (20.8)4 (12.5)0 (0)3 (16.7)1 (8.3)2 (20.0)< .001
      3. Moderate quality33 (22.9)11 (20.8)8 (25.0)3 (15.8)3 (16.7)6 (50.0)2 (20.0)< .001
      4. Partially good quality44 (30.6)13 (24.5)9 (28.1)13 (68.4)7 (38.9)1 (8.3)1 (10.0)< .001
      5. Excellent quality34 (23.6)15 (28.3)9 (28.1)3 (15.8)5 (27.8)1 (8.3)1 (10.0)< .001

      GQS results

      Of all videos, 8.3% (n = 12), 14.6% (n = 21), 22.9% (n = 33), 30.6% (n = 44) and 23.6% (n = 34) were classified as poor, partially poor, moderate, partially good and excellent quality videos, respectively. According to videos authoring entity, the highest rate of excellent quality videos was recorded in videos produced by public health institutions (28.3%, P < .001) and by alternative medicine channels (28.1%, P < .001). The highest rate of poor-quality videos was recorded in videos produced by individuals (40.0%, P < .001) and by others (25%, P < .001, Table 3B).

      Discussion

      The current study aimed to evaluate the evidence-based quality and accuracy of YouTube videos on the correct usage of PPE for the care of Covid-19 patients. To the best of our knowledge, no previous investigators examined this topic. We addressed this void and identified several noteworthy observations.
      First, from the December 1, 2019 to the January 31, 2021, we observed an increased worldwide interest on the donning and doffing procedures, both on Google (Δ = 19.2%) and on YouTube (Δ = 19.4%). The peak occurred in April 2020 during the first pandemic wave. In consequence, most of the users interested in the topic obtained information from videos uploaded on the web. Thus, it is mandatory to evaluate the quality of information given in the videos, because learning wrong ways of PPE usage may lead to higher contagion risks.
      • Ortega R
      • Gonzalez M
      • Nozari A
      • Canelli R.
      Personal protective equipment and Covid-19. Ingelfinger JR, editor.
      In the context of a new sanitary emergency outbreak, videos uploaded on YouTube provide a quicker and easier access to the contents of interest, compared to the information available in the scientific literature at the moment of the search. Indeed, the process of studies publication on the international libraries, such as PubMed

      PubMed [Internet]. PubMed. 2021 [cited 2021 Mar 30]. https://pubmed.ncbi.nlm.nih.gov/. Accessed December 15, 2021

      or Cochrane Library,

      Cochrane Reviews | Cochrane Library [Internet]. 2021 [cited 2021 May 24]. https://www.cochranelibrary.com/. Accessed December 15, 2021

      takes time due to multiple revision-making steps. For this reason, the contents shown in the videos should be correctly reported.
      Second, of all videos examined, approximately 40% were produced by public and government health institutions and approximately 90% were targeted to the health care workers, who are the most exposed professionals to the virus. In consequence, it is important that YouTube videos explaining donning and doffing steps report accurate and high-quality information in order to avoid mistakes. In this regard, we recorded higher medians for all the doffing steps (ranged from 4 to 5), relative to the donning steps (ranged from 3 to 5). It is important to underline how the doffing, more than donning, is the most dangerous procedure, due to the prolonged contact with infected patients during the care.
      • Cheng L
      • Chen L
      • Xiao L
      • et al.
      Problems and solutions of personal protective equipment doffing in COVID-19.
      In the future, it will be important for the video authors to maintain good standards on the high-quality represented steps and give more attention to the low-quality represented ones, providing a more complete and exhaustive information.
      Third, we recorded inaccurate contents in 90 videos (62.5%), according to the misinformation tool based on the CDC recommendations.
      CDC
      Healthcare workers [Internet].
      Specifically, the main inaccuracy reason was the incomplete presentation of data. For example, we observed that several YouTube videos have given partial information on the step of washing hands, even though it is widely known that this procedure decreases the risk of infection.
      • Hillier MD.
      Using effective hand hygiene practice to prevent and control infection.
      ,
      • Lotfinejad N
      • Peters A
      • Pittet D.
      Hand hygiene and the novel coronavirus pandemic: the role of healthcare workers.
      Thus, future videos should focus more their attention on the worst represented aspects.
      Fourth, according to DISCERN instrument, the median of the question 16, which is a surrogate of the overall quality, was 4. Moreover, videos produced by public health institutions showed higher quality, relative to videos produced by other entities. Furthermore, according to GQS, 78 (54.2%) videos were evaluated at least as good quality. Moreover, we observed that, among 85 (59.0%) videos produced by both public health institutions and alternative medicine channels, more than half were evaluated as good or excellent quality. Conversely, among 22 (15.2%) videos produced by individuals or others, more than half were partially poor quality or below. The above considerations imply that YouTube videos on donning and doffing of PPE during Covid-19 pandemic show a reasonable quality, especially the ones produced by official institutions and alternative medicine channels. In conclusion, internet users should first consider videos published by these kinds of authoring entities, which may represent the most reliable contents sources on this topic.
      Taken together, we observed that the interest peak on both Google and YouTube occurred in April 2020, concordantly to the first pandemic wave. It confirms how YouTube was highly used to promptly acquire information on donning and doffing of PPE for Covid-19 patients care. Moreover, overall reliability and quality of YouTube videos on this topic were reasonable, as evidenced by high DISCERN score and high number of good and excellent quality videos. Nevertheless, a not negligible number of videos contained inaccuracies. In the future, it will be mandatory for the authors to give more attention to the low-quality items, maintaining good standards in the high-quality ones, providing the most exhaustive information possible. Finally, we have listed the top-quality videos among all the ones included in the current study as they may be effective tools for health care workers training during the pandemic (Supplementary Table 4).
      The current study is not devoid of limitations. First, YouTube search results rely on Google proprietary search algorithms, based on user's previous search activities and location. However, the log-out from any personal accounts and the change of location via VPN proxy were performed. Second, only English-language videos were included in the final sample. Other languages videos could provide different information. Third, some reliable or non-reliable videos might be missed, due to our search terms. However, we used 40 keywords combinations in order to minimize selection errors. Fourth, quality assessment videos were subjectively evaluated. However, 3 investigators were independently involved to analyse video contents. Fifth, YouTube is a constantly expanding multimedia platform and the contents may rapidly change significantly with new updates over time.

      Conclusions

      During the Covid-19 outbreak, YouTube was used as an easy and quick access source of information on donning and doffing of PPE, especially during the first pandemic wave. According to DISCERN instrument and GQS, YouTube may be recommended as a reliable source on correct usage of PPE for the Covid-19 patients care. In the future, it will be important for the video authors to maintain good standards on the high-quality represented steps and give more attention to the low-quality represented ones, providing a more complete and exhaustive information.

      Appendix. SUPPLEMENTARY MATERIALS

      References

      1. COVID-19: Health worker death toll rises to at least 17000 as organizations call for rapid vaccine rollout | Amnesty International [Internet]. 2021 [cited 2021 Mar 16]. https://www.amnesty.org/en/latest/news/2021/03/covid19-health-worker-death-toll-rises-to-at-least-17000-as-organizations-call-for-rapid-vaccine-rollout/. Accessed March 16, 2021

      2. WHO Director-General's opening remarks at the media briefing on COVID-19 - 11 March 2020 [Internet]. 2021 [cited 2021 Mar 16]. https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19-11-march-2020. Accessed March 16, 2021

      3. Coronavirus disease (COVID-19) – world health organization [internet]. 2021 [cited 2021 Mar 16]. https://www.who.int/emergencies/diseases/novel-coronavirus-2019. Accessed March 16, 2021

      4. Coronavirus Update (Live): 120,822,563 Cases and 2,673,622 deaths from COVID-19 virus pandemic - worldometer [Internet]. 2021 [cited 2021 Mar 16]. https://www.worldometers.info/coronavirus/. Accessed March 16, 2021

      5. COVID-19 transmission—up in the air.:1. 2021

        • Meyerowitz EA
        • Richterman A
        • Gandhi RT
        • Sax PE.
        Transmission of SARS-CoV-2: a review of viral, host, and environmental factors.
        Ann Intern Med. 2021 Jan; 174: 69-79
        • CDC
        COVID-19 and Your Health [Internet].
        Centers for Disease Control and Prevention. 2020; ([cited 2021 Mar 16])
        • Creta M
        • Sagnelli C
        • Celentano G
        • et al.
        SARS-CoV-2 infection affects the lower urinary tract and male genital system: a systematic review.
        J Med Virol. 2021 May; 93: 3133-3142
        • CDC
        Healthcare Workers [Internet].
        Centers for Disease Control and Prevention. 2020; ([cited 2021 Mar 16])
      6. Rational use of personal protective equipment for coronavirus disease (COVID-19) and considerations during severe shortages [Internet]. 2021 [cited 2021 Mar 16]. https://www.who.int/publications/i/item/rational-use-of-personal-protective-equipment-for-coronavirus-disease-(covid-19)-and-considerations-during-severe-shortages. Accessed March 16, 2021

      7. Guidance for wearing and removing personal protective equipment in healthcare settings for the care of patients with suspected or confirmed COVID-19 [Internet]. European Centre for Disease Prevention and Control. 2020 [cited 2021 Mar 16]. Accessed March 16, 2021. https://www.ecdc.europa.eu/en/publications-data/guidance-wearing-and-removing-personal-protective-equipment-healthcare-settings

        • Wang J
        • Zhou M
        • Liu F.
        Reasons for healthcare workers becoming infected with novel coronavirus disease 2019 (COVID-19) in China.
        J Hosp Infect. 2020 May; 105: 100-101
      8. Press - YouTube [Internet]. 2021 [cited 2021 Mar 16]. Accessed March 16, 2021. https://www.youtube.com/intl/en-GB/about/press/

      9. youtube.com Competitive Analysis, Marketing Mix and Traffic - Alexa [Internet]. 2021 [cited 2021 Mar 16]. Accessed March 16, 2021. https://www.alexa.com/siteinfo/youtube.com

        • Lim K
        • Kilpatrick C
        • Storr J
        • Seale H.
        Exploring the use of entertainment-education YouTube videos focused on infection prevention and control.
        Am J Infect Control. 2018 Nov; 46: 1218-1223
        • Gandhi CK
        • Patel J
        • Zhan X.
        Trend of influenza vaccine Facebook posts in last 4 years: a content analysis.
        Am J Infect Control. 2020 Apr; 48: 361-367
        • Meadows CZ
        • Tang L
        • Liu W.
        Twitter message types, health beliefs, and vaccine attitudes during the 2015 measles outbreak in California.
        Am J Infect Control. 2019 Nov; 47: 1314-1318
        • Odlum M
        • Yoon S.
        What can we learn about the Ebola outbreak from tweets?.
        Am J Infect Control. 2015 Jun; 43: 563-571
        • Tang L
        • Bie B
        • Park S-E
        • Zhi D.
        Social media and outbreaks of emerging infectious diseases: a systematic review of literature.
        Am J Infect Control. 2018 Sep; 46: 962-972
        • Tolu S.
        English-language videos on YouTube as a source of information on self-administer subcutaneous anti-tumour necrosis factor agent injections.
        Rheumatol Int.:8. 2021; 38: 1285-1292
        • Aldallal SN
        • Yates JM
        • Ajrash M.
        Use of YouTubeTM as a self-directed learning resource in oral surgery among undergraduate dental students: a cross-sectional descriptive study.
        Br J Oral Maxillofac Surg. 2019 Dec; 57: 1049-1052
        • Garg N
        • Venkatraman A
        • Pandey A
        • Kumar N.
        YouTube as a source of information on dialysis: a content analysis: online information on dialysis.
        Nephrology. 2015 May; 20: 315-320
        • Kumar N
        • Pandey A
        • Venkatraman A
        • Garg N.
        Are video sharing Web sites a useful source of information on hypertension?.
        J Am Soc Hypertens. 2014 Jul; 8: 481-490
        • Capece M
        • Di Giovanni A
        • Cirigliano L
        • et al.
        YouTube as a source of information on penile prosthesis.
        Andrologia. 2021 Sep 14; : e14246
        • Morra S
        • Collà Ruvolo C
        • Napolitano L
        • et al.
        YouTubeTM as a source of information on bladder pain syndrome: a contemporary analysis.
        Neurourol Urodyn. 2021 Sep 24;
      10. Google trends [Internet]. Google Trends. 2021 [cited 2021 Mar 30]. Accessed December 15, 2021. https://trends.google.it/trends/?geo=IT

      11. YouTube [Internet]. 2021 [cited 2021 Mar 30]. Accessed December 15, 2021. https://www.youtube.com/

        • CDC
        Healthcare workers [Internet].
        Centers for Disease Control and Prevention. 2020; ([cited 2021 Mar 30])
        • Huang MM
        • Winoker JS
        • ME Allaf
        • Matlaga BR
        • Koo K.
        Evidence-based quality and accuracy of YouTube videos about nephrolithiasis.
        BJU Int. 2021 Feb; 127: 247-253
        • Akyol A.
        Is YouTube a quality source of information on sarcopenia?.
        Eur Geriatr Med.:5. 2021; (Accessed March 30, 2021)
      12. DISCERN - The DISCERN Instrument [Internet]. 2021 [cited 2021 Mar 30]. http://www.discern.org.uk/discern_instrument.php. Accessed March 30, 2021

        • Erdem MN
        • Karaca S.
        Evaluating the accuracy and quality of the information in kyphosis videos shared on YouTube.
        Spine. 2018 Nov 15; 43: E1334-E1339
        • Kocyigit BF
        • Akaltun MS.
        Does YouTube provide high quality information? Assessment of secukinumab videos.
        Rheumatol Int. 2019 Jul; 39: 1263-1268
        • Kocyigit BF
        • Akaltun MS
        • Sahin AR.
        YouTube as a source of information on COVID-19 and rheumatic disease link.
        Clin Rheumatol. 2020 Jul; 39: 2049-2054
        • Bernard A
        • Langille M
        • Hughes S
        • Rose C
        • Leddin D
        • Veldhuyzen van Zanten S.
        A systematic review of patient inflammatory bowel disease information resources on the World Wide Web.
        Am J Gastroenterol. 2007 Sep; 102: 2070-2077
        • Ortega R
        • Gonzalez M
        • Nozari A
        • Canelli R.
        Personal protective equipment and Covid-19. Ingelfinger JR, editor.
        N Engl J Med. 2020 Jun 25; 382: e105
      13. PubMed [Internet]. PubMed. 2021 [cited 2021 Mar 30]. https://pubmed.ncbi.nlm.nih.gov/. Accessed December 15, 2021

      14. Cochrane Reviews | Cochrane Library [Internet]. 2021 [cited 2021 May 24]. https://www.cochranelibrary.com/. Accessed December 15, 2021

        • Cheng L
        • Chen L
        • Xiao L
        • et al.
        Problems and solutions of personal protective equipment doffing in COVID-19.
        Open Med Wars Pol. 2020; 15: 605-612
        • Hillier MD.
        Using effective hand hygiene practice to prevent and control infection.
        Nurs Stand. 2020 Apr 29; 35: 45-50
        • Lotfinejad N
        • Peters A
        • Pittet D.
        Hand hygiene and the novel coronavirus pandemic: the role of healthcare workers.
        J Hosp Infect. 2020 Aug; 105: 776-777