If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Corresponding Author. Anne-Mette Iversen, Department of Oncology, Aarhus University Hospital, Palle Juul-Jensens Boulevard 99, DK, 8200, Aarhus N, Denmark
The system can capture hand hygiene behavior of staff under clinical conditions
•
The sensitivity and positive predictive value (PPV) of sanitizations were 100 %
•
The sensitivity and PPV of patient contact were 100% and 94% (nurses and doctors)
•
The system shows less accuracy for cleaning assistants. More data are needed
•
The system can be used as a supporting tool to provide reliable hand hygiene data
ABSTRACT
Background
: We aimed to test the accuracy of an electronic hand hygiene monitoring system (EHHMS) during daily clinical activities in different wards and with varying healthcare professions.
Method
: The accuracy of an EHHMS ([blinded for reviewers]) was assessed during real clinical conditions by comparing events registered by two observers in parallel with events registered by the EHHMS. The events were categorized as true-positive, false-positive, and false-negative registrations. Sensitivity and positive predictive value (PPV) were calculated.
Results
: A total of 103 events performed by 25 healthcare workers (9 doctors, 11 nurses, and 5 cleaning assistants) were included in the analyses. The EHHMS had a sensitivity of 100% and a positive predictive value of 100% when measuring alcohol-based hand rub. When looking at the hand hygiene opportunities of all healthcare workers combined taking place in the patient rooms and working rooms, the sensitivity was 75% and the PPV 95%. For doctors‘ and nurses’ taking care of patients in their beds the EHHMS had a sensitivity of 100% and a PPV of 94%.
Conclusion
: The objective accuracy measures demonstrate that this EHHMS can capture hand hygiene behavior under clinical conditions in different settings with clinical healthcare workers but show less accuracy with cleaning assistants.
Healthcare-associated infections (HAIs) are the most frequent adverse events occurring during patient care and are estimated to cost a 200-bed facility more than $1.7 million per year (
). Even during the Covid-19 pandemic, HH compliance (HHC) among healthcare workers (HCWs) is a challenge, and hospitals are struggling to find solutions with sustained effect (
To evaluate HH interventions and the cost-effectiveness of new initiatives, HHC must be measured reliably. Direct observation by trained observers is the most used method, but it is subject to bias and resource-heavy for the already strained health systems managing the pandemic (
). Healthcare organizations are starting to use electronic HH monitoring systems (EHHMSs) as part of the World Health Organization's (WHO) multimodal strategy for HH improvement because they require fewer human resources, provide larger and more representative data sets, and are less subject to observation bias. EHHMS measures a proxy for HH. Both direct observations and EHHMSs have pros and cons, and together they can supplement each other (
Learning to interact with new technology: Health care workers’ experiences of using a monitoring system for assessing hand hygiene - a grounded theory study.
Am J Infect Control.2021 Oct 2; S0196-6553 (00641-6)
). A systematic review of 42 articles mentioning automated measurement systems found that fewer than 20% of the studies included calculations for accuracy (
). The system's accuracy was recently validated in a [blinded for reviewers] hospital under simulated conditions, which showed an accuracy rate of 100% (
). However, EHHMSs also need to be validated during real clinical conditions to assess reliability and generalizability, as suggested by Limper et al. (
We aimed to test the accuracy of the [blinded for reviewers] system during real clinical conditions in different wards with varying healthcare professions.
METHODS
Setting
At three randomly selected days in December 2020 and June 2021, we conducted the study in the Department of Oncology (32 beds) and the Department of Hematology (34 beds) at [blinded for reviewers], a tertiary care university hospital center with 85,000 hospital admissions per year and covering a population of up to 2.5 million inhabitants.
) is an advanced type 5 EHHMS according to the classification by Gould et al. A type 5 EHHMS is capable of taking previous workflow into considerations instead of only looking at room entry or patient zones as separate events (
) the [blinded for reviewers] zone sensor: A sensor placed on the wall above the patient bed and in workrooms (e.g., medication rooms) that registers if the healthcare worker (HCW) was near the sensor; (
) the [blinded for reviewers] dispenser sensor: A sensor on soap and alcohol-based hand rub (ABHR) dispensers that measures when the HH action happened. For this study, the sensors were not placed on soap dispensers; (
) the [blinded for reviewers]: An anonymous Bluetooth tag on the HCW's name badge, key hanger, or clothes which connects a HH action to a HCW and registers if the HH action happened in relation to a HH opportunity.
Fig. 1Three hardware components of ([blinded for reviewers]):1) the zone sensor, 2) the dispenser sensor, 3) the individual Bluetooth tag.
The system uses time and distance measures as part of the algorithms to register if a HH opportunity takes place. It does not qualitatively distinguish between moments 4 and 5.
Validation approach
The study design aimed to compare HH actions and HH opportunities between direct observations and the EHHMS in order to establish true positive, true negative and false negative events.
We adapted a validation approach described by Limper et al. (
). Because the EHHMS had already been validated during real clinical conditions, we focused on the final phase of the validation approach testing how the EHHMS performed under real clinical conditions. HCWs (nurses, doctors, and cleaning assistants) from the two departments (four wards) volunteered to wear a Bluetooth test tag ([blinded for reviewers]) during their daily clinical activities. The test tag had a known identification number to ensure that each event could be identified in the database retrospectively. The observations were included in the analyses when the following inclusion criteria were met: the observers registered contact between the HCW and the patient/patient surroundings the system registered the HCW in the patient zone in sufficient time for contact with the patient/patients near surroundings the observers registered the HCW's use of ABHR the system registered the HCW's use of ABHR
A pre-requisite for the event to be included was that: 1) the HCWs agreed to wear the test tag; 2) the HCWs used an ABHR dispenser with a sensor; and 3) the patient bed was placed correctly under the bed sensor. The pre-requisite variables were checked for each event.
Two trained and experienced observers (nurses) documented all HH actions and HH opportunities by direct observation using a pre-defined observation sheet (Table 1). The two observers documented the behavior of each HCW at the same time. The events are reported in two main categories 1) HH actions (HCWs use of ABHR) and 2) HH opportunities (HCWs physical contact with a patient, patient surroundings, or work zones)). In case of discrepancies between the two observers, the event was excluded.
Table 1Example of registration of observation data
Time (HH:MM)
Procedure/Behavior
Use of ABHR
08.36
Hand hygiene
x
08.37
Touch the patient bed
08.41
Take a notebook from the uniform pocket and write a note
Table 2Hand Hygiene actions performed by doctors, nurses and cleaning assistants Comparisons of results between the direct observers and the electronic hand hygiene monitoring system for nurses, doctors, and cleaning staff.
Table 3Hand Hygiene opportunities performed by doctors, nurses and cleaning assistants. Comparison of results between the direct observers and the electronic hand hygiene monitoring system.
This was a substudy to a quality improvement project. According to the [blinded for reviewers] law, approval was queried and evaluated as not needed by both the Ethics Committee ([blinded for reviewers]) and the [blinded for reviewers] Data Protection Agency ([blinded for reviewers]). After approval from department management, all the observed HCWs were verbally informed of the aim of the study and agreed to use a test sensor while they were being observed. Patients were also verbally informed of the purpose of the observers’ presence in the patient room.
Statistical analysis
We used an independent-event approach treating each device encounter as an independent event to allow identification of inaccuracies during the observations. The direct observation data from the two observers and data from the EHHMS were categorized into three scenarios as suggested by Limper et al. (
) False-negative events were actions/opportunities registered by the observers but not captured by the EHHMS. The truth was defined by the two observers recording the same HH action and HH opportunity of the HCW. Based on the events, we calculated the sensitivity and positive predictive value. The sensitivity was defined as the probability that a true HH event was captured by the EHHMS. The positive predictive value (PPV) was defined as the probability that the event captured by the EHHMS really occurred. True-negative events (events not captured by the observers or the EMHHS) were not possible to report in this study because these events could not be identified.
Statistical analyses were conducted using GraphPad Prism (version 9.3.1, GraphPad Inc.) and Excel (version 16.47.1, Microsoft).
RESULTS
Overall, 120 events were performed by 25 HCWs (doctors, n=9; nurses, n=11; cleaning assistants, n=5). Twelve events did not meet the inclusion criteria. Of the remaining events, we found a discrepancy between the registrations of the two observers in five cases (percentage of agreement between observers of 95%). Thus, 103 events were included in the accuracy analyses, of which 78 were HH actions and 25 were HH opportunities. The nurses accounted for 45 (44%) of the registrations, doctors 35 (34%), and the cleaning assistants 22 (21%).
When looking at HH actions the overall accuracy analyses show a sensitivity of 100% (95% CI: 95%-100%) meaning that all HH events were detected by the EHHMS. The positive predictive value was 100% (95% CI: 95%-100%).
When looking at HH opportunities the overall accuracy of the EHHMS shows a sensitivity of 75% (95% CI: 55%-88%) and a positive predictive value of 95% (95% CI: 75%-100%). Three of the six false-negative events concerned cleaning activities of the patient bed or patient surroundings by the cleaning assistants, which was not registered by the system. Two false-negative events were doctors examining the patient in a chair close (approx. 1-2 meters) to the bed and the sensor. The last false-negative event was a nurse picking up a plate from the patient table in the patient room, which the EHHMS did not detect. The nurse did not touch the patient. The one false-positive event concerned a doctor standing near the patient's bed talking with the patient for a longer period of time. The doctor did not have patient contact, and did not touch the near surroundings, but the EHHMS registered the doctor in the patient zone. So, the system detected all HH opportunities of nurses and doctors with the patient in bed, but also one event without contact.
The system was developed to detect HH events related to workflow of nurses and doctors), who have patient contact while the patient is in the bed or while they perform work in the medication room, rinsing rooms, storerooms, and staff toilets. When looking into these events only, the sensitivity was 100% (95% CI: 81%-100%) and the positive predictive value 94% (95% CI: 73%-100%) (Table 4). When looking at the cleaning assistants for whom the system was not intended, the observers only registered three events with contact between a cleaning assistant and a patient or patient near surroundings which the system did not capture.
Table 4Hand Hygiene opportunities performed by doctors and nurses. Comparisons of results between the direct observers and the electronic hand hygiene monitoring system.
The median length of the patient contacts registered by the EHHMS was 63 seconds (95% CI: 23%–215%). The patient contact with the shortest length of duration was 15 seconds. The patient contact with the longest length of duration was 587 seconds.
DISCUSSION
In this validation study, we investigated the accuracy of an EHHMS during real clinical conditions in different wards and with varying types of healthcare professionals. We found a high sensitivity for detecting HH opportunities by nurses and doctors, which was comparable to the findings in a previous validation study of the EHHMS under simulated conditions (
It is the first time cleaning assistants has used the EHHMS and is, to the best of our knowledge, the first study to publish data for this group. The EHHMS calculates HHC based on algorithms designed for the workflow of nurses and doctors. We found it interesting to investigate if the EHHMS and the algorithms could be used on the cleaning assistant's workflow. The results indicate that this staff group can use the EHHMS, but the system might not register all HH opportunities of the cleaning assistants when cleaning equipment and surfaces of patient surroundings. Importantly, the missed HH opportunities will not have a negative impact on their HHC because they are simply not registered. Further studies are necessary to determine if the EHHMS can be used to measure the HHC of cleaning assistants.
Two false-negative cases were in relation to doctors examining patients in a chair next to the bed, and one false-negative case was a nurse picking up a plate from the patient room while the patient was sitting in a chair. We decided to include HH opportunities in the patient chairs nearby the bed to investigate the possibility of false-positive events. However, we did not detect any false-positive events with patients sitting in chairs. The chairs were placed randomly in each patient room with a distance between one and two meters from the patient bed. If we only include patient contacts occurring when in bed, the system registered all HH opportunities.
Only a few studies have tested the accuracy of an EHHMS using a methodology like this study, which was suggested by Limper et al. (
). A second study found that the accuracy of measuring HH events decreased from 88.5% under simulated conditions to 52.4% under real clinical conditions (nGage™ system) (
). Post-hoc analyses of the study with the Tork Vision Hand Hygiene System by Cawthorne et al. showed a sensitivity of 75%, specificity of 97%, PPV of 97%, and NPV of 72% (
). Our data suggest that [blinded for reviewers] may be as or more accurate than other EHHMSs when assessing the HH behavior of doctors and nurses.
When implementing an EHHMS, the algorithms can be adjusted to the setting. This study found a median length of the patient contacts of 63 seconds with the shortest contact of 15 seconds, which is relevant knowledge when optimizing EHHMSs because the time parameter is often part of the algorithms used. The threshold needed for estimation of patient contact might differ in other settings and types of professions. However, a strength of the study is that we used different wards and different healthcare professions, which increases the reliability and generalizability.
A strength of this study is that the EHMMS was compared to human evaluation of the HCWs’ HH behavior by two observers. Human evaluation depends on the observer's experience. We overcame this challenge by using two trained observers documenting the behavior of each participating HCW at the same time. We found an percentage of agreement between observers of 95%, highlighting the importance of having two observers present when conducting EHHMS validation studies to minimize the risk of wrongly classified events. However, having two observers collecting the same observation data by following the HCWs through different room types and tasks resulted in relatively few data points, which is a major limitation of this study.
We could have collected more data points if we only looked at room entry and exits, but the strength of this type 5 EHHMS is that it measures HHC by following the HCW around in the different rooms in the ward, and therefore the observers also did that. Continued study with more data points is necessary to determine the validity under real clinical conditions. A limitation of the study is that the study setup did not make it possible to detect true-negative events. This is not possible because a non-event could not be defined in time and place as described by Limper et al. (
). Without true-negative events, we cannot calculate specificity and negative predictive values.
CONCLUSION
We found the [blinded for reviewers] system to be accurate when tested during real clinical conditions. The EHHMS captured WHO's Moments 1, 4, and 5 of varying healthcare professions in different settings with high objective accuracy. The findings indicate that the EHHMS can be used as a supporting tool to provide reliable data for some of the key elements of infection prevention and control.
Declaration of Competing Interest
MBH is working in Konduto ApS, which has developed Sani NudgeTM. The other authors declare that they have no competing interests. All authors have approved the final article.
Source of funding
This study was funded by the Health Research Foundation of the ([blinded for reviewers])
Acknowledgements
We thank all the healthcare workers who took part in the study.
REFERENCES
Saleem Z
Godman B
Hassali MA
Hashmi FK
Azhar F
Rehman IU.
Point prevalence surveys of health-care-associated infections: a systematic review.
Learning to interact with new technology: Health care workers’ experiences of using a monitoring system for assessing hand hygiene - a grounded theory study.
Am J Infect Control.2021 Oct 2; S0196-6553 (00641-6)
00503-X&id=gr1.jpg){kind=link}