Advertisement

Clinical Evaluation of an Electronic Hand Hygiene Monitoring System

Open AccessPublished:June 19, 2022DOI:https://doi.org/10.1016/j.ajic.2022.06.017

      Highlights

      • The system can capture hand hygiene behavior of staff under clinical conditions
      • The sensitivity and positive predictive value (PPV) of sanitizations were 100 %
      • The sensitivity and PPV of patient contact were 100% and 94% (nurses and doctors)
      • The system shows less accuracy for cleaning assistants. More data are needed
      • The system can be used as a supporting tool to provide reliable hand hygiene data

      ABSTRACT

      Background

      : We aimed to test the accuracy of an electronic hand hygiene monitoring system (EHHMS) during daily clinical activities in different wards and with varying healthcare professions.

      Method

      : The accuracy of an EHHMS ([blinded for reviewers]) was assessed during real clinical conditions by comparing events registered by two observers in parallel with events registered by the EHHMS. The events were categorized as true-positive, false-positive, and false-negative registrations. Sensitivity and positive predictive value (PPV) were calculated.

      Results

      : A total of 103 events performed by 25 healthcare workers (9 doctors, 11 nurses, and 5 cleaning assistants) were included in the analyses. The EHHMS had a sensitivity of 100% and a positive predictive value of 100% when measuring alcohol-based hand rub. When looking at the hand hygiene opportunities of all healthcare workers combined taking place in the patient rooms and working rooms, the sensitivity was 75% and the PPV 95%. For doctors‘ and nurses’ taking care of patients in their beds the EHHMS had a sensitivity of 100% and a PPV of 94%.

      Conclusion

      : The objective accuracy measures demonstrate that this EHHMS can capture hand hygiene behavior under clinical conditions in different settings with clinical healthcare workers but show less accuracy with cleaning assistants.

      Key words

      BACKGROUND

      Healthcare-associated infections (HAIs) are the most frequent adverse events occurring during patient care and are estimated to cost a 200-bed facility more than $1.7 million per year (
      • Saleem Z
      • Godman B
      • Hassali MA
      • Hashmi FK
      • Azhar F
      • Rehman IU.
      Point prevalence surveys of health-care-associated infections: a systematic review.
      ,
      • Zarb P
      • Coignard B
      • Griskeviciene J
      • Muller A
      • Vankerckhoven V
      • Weist K
      • et al.
      The European Centre for Disease Prevention and Control (ECDC) pilot point prevalence survey of healthcare-associated infections and antimicrobial use.
      ,
      • Pittet D
      • Hugonnet S
      • Harbarth S
      • Mourouga P
      • Sauvan V
      • Touveneau S
      • et al.
      Effectiveness of a hospital-wide programme to improve compliance with hand hygiene.
      ,
      • Klevens RM
      • Edwards JR
      • Richards CL
      • Horan TC
      • Gaynes RP
      • Pollock DA
      • et al.
      Estimating health care-associated infections and deaths in U.S. hospitals.
      ). Inadequate hand hygiene (HH) leads to cross-transmission of microorganisms and HAIs (
      World HealthOrganization
      WHO guidelines on hand hygiene in health care: first global patient safety challenge: clean care is safer care.
      ). Even during the Covid-19 pandemic, HH compliance (HHC) among healthcare workers (HCWs) is a challenge, and hospitals are struggling to find solutions with sustained effect (

      [blinded for reviewers]. 2022

      ,
      • Williams V
      • Kovacs-Litman A
      • Muller MP
      • Hota S
      • Powis JE
      • Ricciuto DR
      • et al.
      Impact of COVID-19 on hospital hand hygiene performance: a multicentre observational study using group electronic monitoring.
      ,
      • Casaroto E
      • Generoso JR
      • Tofaneto BM
      • Bariani LM
      • Auler M de A
      • Xavier N
      • et al.
      Hand hygiene performance in an intensive care unit before and during the COVID-19 pandemic.
      ).
      To evaluate HH interventions and the cost-effectiveness of new initiatives, HHC must be measured reliably. Direct observation by trained observers is the most used method, but it is subject to bias and resource-heavy for the already strained health systems managing the pandemic (
      • Dhar S
      • Tansek R
      • Toftey EA
      • Dziekan BA
      • Chevalier TC
      • Bohlinger CG
      • et al.
      Observer bias in hand hygiene compliance reporting.
      ,
      • Boyce JM.
      Hand Hygiene, an Update.
      ). Healthcare organizations are starting to use electronic HH monitoring systems (EHHMSs) as part of the World Health Organization's (WHO) multimodal strategy for HH improvement because they require fewer human resources, provide larger and more representative data sets, and are less subject to observation bias. EHHMS measures a proxy for HH. Both direct observations and EHHMSs have pros and cons, and together they can supplement each other (
      • Dhar S
      • Tansek R
      • Toftey EA
      • Dziekan BA
      • Chevalier TC
      • Bohlinger CG
      • et al.
      Observer bias in hand hygiene compliance reporting.
      ,
      • Boyce JM.
      Hand Hygiene, an Update.
      ,
      • Srigley JA
      • Furness CD
      • Baker GR
      • Gardam M.
      Quantification of the Hawthorne effect in hand hygiene compliance monitoring using an electronic monitoring system: a retrospective cohort study.
      ,
      • Boyce JM.
      Electronic monitoring in combination with direct observation as a means to significantly improve hand hygiene compliance.
      ).
      New EHHMSs must be validated in clinical practice to be widely adopted (
      • Blomgren P-O
      • Lytsy B
      • Hjelm K
      • Swenne CL.
      Healthcare workers’ perceptions and acceptance of an electronic reminder system for hand hygiene.
      ,
      • Granqvist K
      • Ahlstrom L
      • Karlsson J
      • Lytsy B
      • Andersson AE.
      Learning to interact with new technology: Health care workers’ experiences of using a monitoring system for assessing hand hygiene - a grounded theory study.
      ). A systematic review of 42 articles mentioning automated measurement systems found that fewer than 20% of the studies included calculations for accuracy (
      • Ward MA
      • Schweizer ML
      • Polgreen PM
      • Gupta K
      • Reisinger HS
      • Perencevich EN.
      Automated and electronically assisted hand hygiene monitoring systems: a systematic review.
      ). In [blinded for reviewers], the only EHHMS used is [blinded for reviewers] (

      [blinded for reviewers]. 2022

      ). The system's accuracy was recently validated in a [blinded for reviewers] hospital under simulated conditions, which showed an accuracy rate of 100% (

      [blinded for reviewers]. 2022

      ). However, EHHMSs also need to be validated during real clinical conditions to assess reliability and generalizability, as suggested by Limper et al. (
      • Limper HM
      • Garcia-Houchins S
      • Slawsky L
      • Hershow RC
      • Landon E.
      A Validation Protocol: Assessing the Accuracy of Hand Hygiene Monitoring Technology.
      ,
      • Limper HM
      • Slawsky L
      • Garcia-Houchins S
      • Mehta S
      • Hershow RC
      • Landon E.
      Assessment of an Aggregate-Level Hand Hygiene Monitoring Technology for Measuring Hand Hygiene Performance Among Healthcare Personnel.
      ).
      We aimed to test the accuracy of the [blinded for reviewers] system during real clinical conditions in different wards with varying healthcare professions.

      METHODS

      Setting

      At three randomly selected days in December 2020 and June 2021, we conducted the study in the Department of Oncology (32 beds) and the Department of Hematology (34 beds) at [blinded for reviewers], a tertiary care university hospital center with 85,000 hospital admissions per year and covering a population of up to 2.5 million inhabitants.

      Electronic hand hygiene monitoring system

      [blinded for reviewers] (

      [blinded for reviewers]. 2022

      ,

      [blinded for reviewers]. 2022

      ,

      [blinded for reviewers]. 2022

      ,

      [blinded for reviewers]. 2022

      ,

      [blinded for reviewers]. 2022

      ,

      [blinded for reviewers]. 2022

      ) is an advanced type 5 EHHMS according to the classification by Gould et al. A type 5 EHHMS is capable of taking previous workflow into considerations instead of only looking at room entry or patient zones as separate events (
      • Gould D
      • Lindström H
      • Purssell E
      • Wigglesworth N.
      Electronic hand hygiene monitoring: accuracy, impact on the Hawthorne effect and efficiency.
      ). This system captures a proxy measure for WHO's moment 1, 4, and 5 using three main hardware components (Fig. 1): (
      • Saleem Z
      • Godman B
      • Hassali MA
      • Hashmi FK
      • Azhar F
      • Rehman IU.
      Point prevalence surveys of health-care-associated infections: a systematic review.
      ) the [blinded for reviewers] zone sensor: A sensor placed on the wall above the patient bed and in workrooms (e.g., medication rooms) that registers if the healthcare worker (HCW) was near the sensor; (
      • Zarb P
      • Coignard B
      • Griskeviciene J
      • Muller A
      • Vankerckhoven V
      • Weist K
      • et al.
      The European Centre for Disease Prevention and Control (ECDC) pilot point prevalence survey of healthcare-associated infections and antimicrobial use.
      ) the [blinded for reviewers] dispenser sensor: A sensor on soap and alcohol-based hand rub (ABHR) dispensers that measures when the HH action happened. For this study, the sensors were not placed on soap dispensers; (
      • Pittet D
      • Hugonnet S
      • Harbarth S
      • Mourouga P
      • Sauvan V
      • Touveneau S
      • et al.
      Effectiveness of a hospital-wide programme to improve compliance with hand hygiene.
      ) the [blinded for reviewers]: An anonymous Bluetooth tag on the HCW's name badge, key hanger, or clothes which connects a HH action to a HCW and registers if the HH action happened in relation to a HH opportunity.
      Fig 1
      Fig. 1Three hardware components of ([blinded for reviewers]):1) the zone sensor, 2) the dispenser sensor, 3) the individual Bluetooth tag.
      The system uses time and distance measures as part of the algorithms to register if a HH opportunity takes place. It does not qualitatively distinguish between moments 4 and 5.

      Validation approach

      The study design aimed to compare HH actions and HH opportunities between direct observations and the EHHMS in order to establish true positive, true negative and false negative events.
      We adapted a validation approach described by Limper et al. (
      • Limper HM
      • Garcia-Houchins S
      • Slawsky L
      • Hershow RC
      • Landon E.
      A Validation Protocol: Assessing the Accuracy of Hand Hygiene Monitoring Technology.
      ,
      • Limper HM
      • Slawsky L
      • Garcia-Houchins S
      • Mehta S
      • Hershow RC
      • Landon E.
      Assessment of an Aggregate-Level Hand Hygiene Monitoring Technology for Measuring Hand Hygiene Performance Among Healthcare Personnel.
      ). Because the EHHMS had already been validated during real clinical conditions, we focused on the final phase of the validation approach testing how the EHHMS performed under real clinical conditions. HCWs (nurses, doctors, and cleaning assistants) from the two departments (four wards) volunteered to wear a Bluetooth test tag ([blinded for reviewers]) during their daily clinical activities. The test tag had a known identification number to ensure that each event could be identified in the database retrospectively. The observations were included in the analyses when the following inclusion criteria were met: the observers registered contact between the HCW and the patient/patient surroundings the system registered the HCW in the patient zone in sufficient time for contact with the patient/patients near surroundings the observers registered the HCW's use of ABHR the system registered the HCW's use of ABHR
      A pre-requisite for the event to be included was that: 1) the HCWs agreed to wear the test tag; 2) the HCWs used an ABHR dispenser with a sensor; and 3) the patient bed was placed correctly under the bed sensor. The pre-requisite variables were checked for each event.
      Two trained and experienced observers (nurses) documented all HH actions and HH opportunities by direct observation using a pre-defined observation sheet (Table 1). The two observers documented the behavior of each HCW at the same time. The events are reported in two main categories 1) HH actions (HCWs use of ABHR) and 2) HH opportunities (HCWs physical contact with a patient, patient surroundings, or work zones)). In case of discrepancies between the two observers, the event was excluded.
      Table 1Example of registration of observation data
      Time (HH:MM)Procedure/BehaviorUse of ABHR
      08.36Hand hygienex
      08.37Touch the patient bed
      08.41Take a notebook from the uniform pocket and write a note
      08.43Hand hygienex
      08.43Put on gloves
      08.44Touch patient leg and stomach
      08.55Hand hygienex
      Table 2.
      Table 2Hand Hygiene actions performed by doctors, nurses and cleaning assistants Comparisons of results between the direct observers and the electronic hand hygiene monitoring system for nurses, doctors, and cleaning staff.
      Sanitizations observedSanitizations not observed
      Detected by the EHHMS780
      Not detected by the EHHMS0ND
      Table 3.
      Table 3Hand Hygiene opportunities performed by doctors, nurses and cleaning assistants. Comparison of results between the direct observers and the electronic hand hygiene monitoring system.
      Contact observedContact not observed
      Detected by the EHHMS181
      Not detected by the EHHMS6ND

      Ethics

      This was a substudy to a quality improvement project. According to the [blinded for reviewers] law, approval was queried and evaluated as not needed by both the Ethics Committee ([blinded for reviewers]) and the [blinded for reviewers] Data Protection Agency ([blinded for reviewers]). After approval from department management, all the observed HCWs were verbally informed of the aim of the study and agreed to use a test sensor while they were being observed. Patients were also verbally informed of the purpose of the observers’ presence in the patient room.

      Statistical analysis

      We used an independent-event approach treating each device encounter as an independent event to allow identification of inaccuracies during the observations. The direct observation data from the two observers and data from the EHHMS were categorized into three scenarios as suggested by Limper et al. (
      • Limper HM
      • Garcia-Houchins S
      • Slawsky L
      • Hershow RC
      • Landon E.
      A Validation Protocol: Assessing the Accuracy of Hand Hygiene Monitoring Technology.
      ): (
      • Saleem Z
      • Godman B
      • Hassali MA
      • Hashmi FK
      • Azhar F
      • Rehman IU.
      Point prevalence surveys of health-care-associated infections: a systematic review.
      ) True-positive events were defined as actions/opportunities captured by the direct observers and the EHHMS. (
      • Zarb P
      • Coignard B
      • Griskeviciene J
      • Muller A
      • Vankerckhoven V
      • Weist K
      • et al.
      The European Centre for Disease Prevention and Control (ECDC) pilot point prevalence survey of healthcare-associated infections and antimicrobial use.
      ) False-positive events were actions/opportunities that were not registered by the direct observers but captured by the EHHMS. (
      • Pittet D
      • Hugonnet S
      • Harbarth S
      • Mourouga P
      • Sauvan V
      • Touveneau S
      • et al.
      Effectiveness of a hospital-wide programme to improve compliance with hand hygiene.
      ) False-negative events were actions/opportunities registered by the observers but not captured by the EHHMS. The truth was defined by the two observers recording the same HH action and HH opportunity of the HCW. Based on the events, we calculated the sensitivity and positive predictive value. The sensitivity was defined as the probability that a true HH event was captured by the EHHMS. The positive predictive value (PPV) was defined as the probability that the event captured by the EHHMS really occurred. True-negative events (events not captured by the observers or the EMHHS) were not possible to report in this study because these events could not be identified.
      Statistical analyses were conducted using GraphPad Prism (version 9.3.1, GraphPad Inc.) and Excel (version 16.47.1, Microsoft).

      RESULTS

      Overall, 120 events were performed by 25 HCWs (doctors, n=9; nurses, n=11; cleaning assistants, n=5). Twelve events did not meet the inclusion criteria. Of the remaining events, we found a discrepancy between the registrations of the two observers in five cases (percentage of agreement between observers of 95%). Thus, 103 events were included in the accuracy analyses, of which 78 were HH actions and 25 were HH opportunities. The nurses accounted for 45 (44%) of the registrations, doctors 35 (34%), and the cleaning assistants 22 (21%).
      When looking at HH actions the overall accuracy analyses show a sensitivity of 100% (95% CI: 95%-100%) meaning that all HH events were detected by the EHHMS. The positive predictive value was 100% (95% CI: 95%-100%).
      When looking at HH opportunities the overall accuracy of the EHHMS shows a sensitivity of 75% (95% CI: 55%-88%) and a positive predictive value of 95% (95% CI: 75%-100%). Three of the six false-negative events concerned cleaning activities of the patient bed or patient surroundings by the cleaning assistants, which was not registered by the system. Two false-negative events were doctors examining the patient in a chair close (approx. 1-2 meters) to the bed and the sensor. The last false-negative event was a nurse picking up a plate from the patient table in the patient room, which the EHHMS did not detect. The nurse did not touch the patient. The one false-positive event concerned a doctor standing near the patient's bed talking with the patient for a longer period of time. The doctor did not have patient contact, and did not touch the near surroundings, but the EHHMS registered the doctor in the patient zone. So, the system detected all HH opportunities of nurses and doctors with the patient in bed, but also one event without contact.
      The system was developed to detect HH events related to workflow of nurses and doctors), who have patient contact while the patient is in the bed or while they perform work in the medication room, rinsing rooms, storerooms, and staff toilets. When looking into these events only, the sensitivity was 100% (95% CI: 81%-100%) and the positive predictive value 94% (95% CI: 73%-100%) (Table 4). When looking at the cleaning assistants for whom the system was not intended, the observers only registered three events with contact between a cleaning assistant and a patient or patient near surroundings which the system did not capture.
      Table 4Hand Hygiene opportunities performed by doctors and nurses. Comparisons of results between the direct observers and the electronic hand hygiene monitoring system.
      Contact observedContact not observed
      Detected by the EHHMS161
      Not detected by the EHHMS0ND
      The median length of the patient contacts registered by the EHHMS was 63 seconds (95% CI: 23%–215%). The patient contact with the shortest length of duration was 15 seconds. The patient contact with the longest length of duration was 587 seconds.

      DISCUSSION

      In this validation study, we investigated the accuracy of an EHHMS during real clinical conditions in different wards and with varying types of healthcare professionals. We found a high sensitivity for detecting HH opportunities by nurses and doctors, which was comparable to the findings in a previous validation study of the EHHMS under simulated conditions (

      [blinded for reviewers]. 2022

      ).
      It is the first time cleaning assistants has used the EHHMS and is, to the best of our knowledge, the first study to publish data for this group. The EHHMS calculates HHC based on algorithms designed for the workflow of nurses and doctors. We found it interesting to investigate if the EHHMS and the algorithms could be used on the cleaning assistant's workflow. The results indicate that this staff group can use the EHHMS, but the system might not register all HH opportunities of the cleaning assistants when cleaning equipment and surfaces of patient surroundings. Importantly, the missed HH opportunities will not have a negative impact on their HHC because they are simply not registered. Further studies are necessary to determine if the EHHMS can be used to measure the HHC of cleaning assistants.
      Two false-negative cases were in relation to doctors examining patients in a chair next to the bed, and one false-negative case was a nurse picking up a plate from the patient room while the patient was sitting in a chair. We decided to include HH opportunities in the patient chairs nearby the bed to investigate the possibility of false-positive events. However, we did not detect any false-positive events with patients sitting in chairs. The chairs were placed randomly in each patient room with a distance between one and two meters from the patient bed. If we only include patient contacts occurring when in bed, the system registered all HH opportunities.
      Only a few studies have tested the accuracy of an EHHMS using a methodology like this study, which was suggested by Limper et al. (
      • Limper HM
      • Garcia-Houchins S
      • Slawsky L
      • Hershow RC
      • Landon E.
      A Validation Protocol: Assessing the Accuracy of Hand Hygiene Monitoring Technology.
      ). One EHHMS study found a sensitivity of 88.7% under simulated conditions and 92.7% under real clinical conditions (GOJO/Purell SMARTLINKTM system) (
      • Limper HM
      • Slawsky L
      • Garcia-Houchins S
      • Mehta S
      • Hershow RC
      • Landon E.
      Assessment of an Aggregate-Level Hand Hygiene Monitoring Technology for Measuring Hand Hygiene Performance Among Healthcare Personnel.
      ). A second study found that the accuracy of measuring HH events decreased from 88.5% under simulated conditions to 52.4% under real clinical conditions (nGage™ system) (
      • Pineles LL
      • Morgan DJ
      • Limper HM
      • Weber SG
      • Thom KA
      • Perencevich EN
      • et al.
      Accuracy of a radiofrequency identification (RFID) badge system to monitor hand hygiene behavior during routine clinical activities.
      ). A third study found an 84% agreement between an EHHMS and the manual observations (Tork Vision Hand Hygiene System) (
      • Gould D
      • Lindström H
      • Purssell E
      • Wigglesworth N.
      Electronic hand hygiene monitoring: accuracy, impact on the Hawthorne effect and efficiency.
      ). Post-hoc analyses of the study with the Tork Vision Hand Hygiene System by Cawthorne et al. showed a sensitivity of 75%, specificity of 97%, PPV of 97%, and NPV of 72% (
      • Cawthorne K-R
      • Cooke RP.
      A standardised approach to validate both the accuracy and precision of electronic hand hygiene monitoring systems is needed.
      ). Our data suggest that [blinded for reviewers] may be as or more accurate than other EHHMSs when assessing the HH behavior of doctors and nurses.
      When implementing an EHHMS, the algorithms can be adjusted to the setting. This study found a median length of the patient contacts of 63 seconds with the shortest contact of 15 seconds, which is relevant knowledge when optimizing EHHMSs because the time parameter is often part of the algorithms used. The threshold needed for estimation of patient contact might differ in other settings and types of professions. However, a strength of the study is that we used different wards and different healthcare professions, which increases the reliability and generalizability.
      A strength of this study is that the EHMMS was compared to human evaluation of the HCWs’ HH behavior by two observers. Human evaluation depends on the observer's experience. We overcame this challenge by using two trained observers documenting the behavior of each participating HCW at the same time. We found an percentage of agreement between observers of 95%, highlighting the importance of having two observers present when conducting EHHMS validation studies to minimize the risk of wrongly classified events. However, having two observers collecting the same observation data by following the HCWs through different room types and tasks resulted in relatively few data points, which is a major limitation of this study.
      We could have collected more data points if we only looked at room entry and exits, but the strength of this type 5 EHHMS is that it measures HHC by following the HCW around in the different rooms in the ward, and therefore the observers also did that. Continued study with more data points is necessary to determine the validity under real clinical conditions. A limitation of the study is that the study setup did not make it possible to detect true-negative events. This is not possible because a non-event could not be defined in time and place as described by Limper et al. (
      • Limper HM
      • Garcia-Houchins S
      • Slawsky L
      • Hershow RC
      • Landon E.
      A Validation Protocol: Assessing the Accuracy of Hand Hygiene Monitoring Technology.
      ). Without true-negative events, we cannot calculate specificity and negative predictive values.

      CONCLUSION

      We found the [blinded for reviewers] system to be accurate when tested during real clinical conditions. The EHHMS captured WHO's Moments 1, 4, and 5 of varying healthcare professions in different settings with high objective accuracy. The findings indicate that the EHHMS can be used as a supporting tool to provide reliable data for some of the key elements of infection prevention and control.

      Declaration of Competing Interest

      MBH is working in Konduto ApS, which has developed Sani NudgeTM. The other authors declare that they have no competing interests. All authors have approved the final article.

      Source of funding

      This study was funded by the Health Research Foundation of the ([blinded for reviewers])

      Acknowledgements

      We thank all the healthcare workers who took part in the study.

      REFERENCES

        • Saleem Z
        • Godman B
        • Hassali MA
        • Hashmi FK
        • Azhar F
        • Rehman IU.
        Point prevalence surveys of health-care-associated infections: a systematic review.
        Pathog Glob Health. 2019 Jun; 113: 191-205
        • Zarb P
        • Coignard B
        • Griskeviciene J
        • Muller A
        • Vankerckhoven V
        • Weist K
        • et al.
        The European Centre for Disease Prevention and Control (ECDC) pilot point prevalence survey of healthcare-associated infections and antimicrobial use.
        Euro Surveill. 2012 Nov 15; 17: 20316
        • Pittet D
        • Hugonnet S
        • Harbarth S
        • Mourouga P
        • Sauvan V
        • Touveneau S
        • et al.
        Effectiveness of a hospital-wide programme to improve compliance with hand hygiene.
        Infection Control Programme. Lancet. 2000 Oct 14; 356: 1307-1312
        • Klevens RM
        • Edwards JR
        • Richards CL
        • Horan TC
        • Gaynes RP
        • Pollock DA
        • et al.
        Estimating health care-associated infections and deaths in U.S. hospitals.
        2002. Public Health Rep. 2007 Apr; 122: 160-166
        • World HealthOrganization
        WHO guidelines on hand hygiene in health care: first global patient safety challenge: clean care is safer care.
        (editor) World Health Organization, Geneva, Switzerland2009: 262 (Patient Safetyp)
      1. [blinded for reviewers]. 2022

        • Williams V
        • Kovacs-Litman A
        • Muller MP
        • Hota S
        • Powis JE
        • Ricciuto DR
        • et al.
        Impact of COVID-19 on hospital hand hygiene performance: a multicentre observational study using group electronic monitoring.
        CMAJ Open. 2021 Dec; 9: E1175-E1180
        • Casaroto E
        • Generoso JR
        • Tofaneto BM
        • Bariani LM
        • Auler M de A
        • Xavier N
        • et al.
        Hand hygiene performance in an intensive care unit before and during the COVID-19 pandemic.
        Am J Infect Control. 2022 Jan 31; S0196-6553 (00045-1)
        • Dhar S
        • Tansek R
        • Toftey EA
        • Dziekan BA
        • Chevalier TC
        • Bohlinger CG
        • et al.
        Observer bias in hand hygiene compliance reporting.
        Infect Control Hosp Epidemiol. 2010 Aug; 31: 869-870
        • Boyce JM.
        Hand Hygiene, an Update.
        Infect Dis Clin North Am. 2021 Sep; 35: 553-573
        • Srigley JA
        • Furness CD
        • Baker GR
        • Gardam M.
        Quantification of the Hawthorne effect in hand hygiene compliance monitoring using an electronic monitoring system: a retrospective cohort study.
        BMJ Qual Saf. 2014 Dec; 23: 974-980
        • Boyce JM.
        Electronic monitoring in combination with direct observation as a means to significantly improve hand hygiene compliance.
        Am J Infect Control. 2017 May 1; 45: 528-535
        • Blomgren P-O
        • Lytsy B
        • Hjelm K
        • Swenne CL.
        Healthcare workers’ perceptions and acceptance of an electronic reminder system for hand hygiene.
        J Hosp Infect. 2021 Feb; 108: 197-204
        • Granqvist K
        • Ahlstrom L
        • Karlsson J
        • Lytsy B
        • Andersson AE.
        Learning to interact with new technology: Health care workers’ experiences of using a monitoring system for assessing hand hygiene - a grounded theory study.
        Am J Infect Control. 2021 Oct 2; S0196-6553 (00641-6)
        • Ward MA
        • Schweizer ML
        • Polgreen PM
        • Gupta K
        • Reisinger HS
        • Perencevich EN.
        Automated and electronically assisted hand hygiene monitoring systems: a systematic review.
        Am J Infect Control. 2014 May; 42: 472-478
      2. [blinded for reviewers]. 2022

      3. [blinded for reviewers]. 2022

        • Limper HM
        • Garcia-Houchins S
        • Slawsky L
        • Hershow RC
        • Landon E.
        A Validation Protocol: Assessing the Accuracy of Hand Hygiene Monitoring Technology.
        Infect Control Hosp Epidemiol. 2016 Aug; 37: 1002-1004
        • Limper HM
        • Slawsky L
        • Garcia-Houchins S
        • Mehta S
        • Hershow RC
        • Landon E.
        Assessment of an Aggregate-Level Hand Hygiene Monitoring Technology for Measuring Hand Hygiene Performance Among Healthcare Personnel.
        Infect Control Hosp Epidemiol. 2017 Mar; 38: 348-352
      4. [blinded for reviewers]. 2022

      5. [blinded for reviewers]. 2022

      6. [blinded for reviewers]. 2022

        • Gould D
        • Lindström H
        • Purssell E
        • Wigglesworth N.
        Electronic hand hygiene monitoring: accuracy, impact on the Hawthorne effect and efficiency.
        J Infect Prev. 2020 Jul; 21: 136-143
        • Pineles LL
        • Morgan DJ
        • Limper HM
        • Weber SG
        • Thom KA
        • Perencevich EN
        • et al.
        Accuracy of a radiofrequency identification (RFID) badge system to monitor hand hygiene behavior during routine clinical activities.
        Am J Infect Control. 2014 Feb; 42: 144-147
        • Cawthorne K-R
        • Cooke RP.
        A standardised approach to validate both the accuracy and precision of electronic hand hygiene monitoring systems is needed.
        J Infect Prev. 2021 Jan; 22: 42-43