If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to Gerard Lacey, PhD, MBA, School of Computer Science and Statistics, O'Reilly Institute, Trinity College Dublin, Dublin 2, Ireland.
Poor quality handwashing contributes to the spread of nosocomial infections. We investigate the impact of automatic video auditing (AVA) with feedback on the quality and quantity of handwashing in a hospital setting.
Methods
AVA systems were mounted over all handwash sinks in a surgical unit. Phase 1 established baseline handwashing quality and quantity. Phase 2 examined the impact of real-time performance feedback, and phase 3 examined the incremental impact of weekly team performance reports. Phase 4 remeasured the baseline without feedback.
Results
A total of 3,606 handwash events were audited. During phase 2 and 3, compliance with the World Health Organization technique improved from 15.7%-46% (P < .0001), and the average number of handwash events per patient per day increased from 0.91-2.25 (P < .0001). Performance returned to baseline in phase 4.
Conclusions
AVA with real-time feedback significantly improved the quality and quantity of handwashing. The combination of AVA with electronic monitoring will allow simultaneous auditing of hand hygiene quantity and quality. The impact of cognitive offloading onto the technology may have contributed to the return to baseline at the end of the study, and suggests further research is required in this area.
The hands of health care workers (HCW) are identified as the primary vector of transmission for health care–acquired infections, and as a consequence hand hygiene is critical to patient safety.
These studies show low adherence, 8%-15%, with the WHO 6-step technique for hand hygiene. Various strategies have been used to improve hand hygiene quality, and frequent short training sessions have reduced methicillin-resistant Staphylococcus aureus infection rates.
Individualized regular training was more effective than lectures at creating a “culture of prevention,” and resulted in a reduction in bloodstream infections.
In the video auditing of hand hygiene practices, videos are manually coded by researchers to classify and count different behaviors. Video auditing has been used to record activates at handwashing sinks
but real-time feedback was not provided. Remote video auditing is where remote human operators examine samples of live video footage from the clinical setting and deliver near real-time feedback to a cohort of staff. This approach has been used to improve the quantity of hand hygiene from 10% opportunities to 85%.
Remote video auditing does not directly address hand hygiene quality, and concerns have been raised regarding its impact on privacy.
Automatic video auditing (AVA) is where a computer automatically analyzes the video and provides real-time feedback to each individual user. Privacy is ensured as no videos are stored on the device and no videos leave the device. AVA has been shown to improve training outcomes.
To our knowledge, this study represents the first use of AVA to measure handwashing quality at sinks and to provide individualized real-time feedback on technique.
Our study measured the quality and quantity of handwashing by HCWs using AVA with real-time feedback at handwash sinks. The AVA can also be used for measuring the use of alcohol-based handrub (ABHR). However, we chose to focus on handwashing because the AVA does not interrupt clinical workflow at sinks and ensures that all handwashes, no matter the quality, are measured. Unlike handwashing at sinks, ABHR usage is not tied to a specific location and is often performed “on the move,” therefore to use AVA for auditing, HCWs would have to perform handrubs standing in view of the camera. This would change clinical practice and would have allowed HCWs to self-select for auditing, which would have skewed the data.
The auditing of handwashing in this study was carried out as part of the normal hand hygiene compliance auditing procedure for the hospital. As the study captured less personal information than the normal observational audit procedures, the Department of Health determined that the study did not need additional ethical approval.
Methods
A custom-designed AVA system with a computer screen for feedback (SureWash, Dublin, Ireland) was used in this study. An AVA unit was placed over all the handwash sink within an active surgical unit, as shown in Figure 1. In total, 8 AVA units were used to ensure all handwash events were captured. Toilets, bathrooms, and the dirty utility were excluded from the study.
Fig 1Map of ward B7 in Broomfield Hospital showing the numbered locations of each of the units.
The AVA systems provided real-time training feedback to each HCW on their hand hygiene technique. The screen showed each step of the WHO protocol, each with a red/green traffic light symbol. As the HCW completed each step, its associated traffic light changed from red to green. The aim was to get all the traffic lights to turn from red to green. To ensure anonymity, the AVA camera faced straight down so that only the sink could be seen and the HCW's face or any other identifying information was excluded from view.
The AVA systems operated from 6 AM to 8 PM each day for approximately 4 months. The study was divided into 4 phases: phase 1 (10 days) measured the baseline of handwash quality and quantity without providing any feedback. Phase 2 (32 days) and phase 3 (32 days) aimed to assess the longer-term impact of the AVA system and the associated training feedback on handwashing quality and quantity. In phase 3, incremental feedback was provided by means of a weekly performance report for staff meetings. The aim of this report was to determine if group targets had any impact on HCW hand hygiene performance. During phase 4 (6 days) feedback was turned off and the baseline of handwash performance was measured again.
up to 32 different types of events happen at a hand hygiene sinks. To avoid counting these other events as handwash events, we only included handwash events taking more than 5 seconds long. In total, 3,606 handwash events were audited for quality. In the surgical unit, the number of patients and staffing varied daily with a peak on Tuesdays and a minimum over the weekend. To address this issue, the data were normalized using the number of hand hygiene events per patient per day, that is the total number of hand hygiene events per day divided by the number of patients that day.
We measured quality using the percentage of hand hygiene events that met the full WHO protocol, that is the total number of hand hygiene events that turned all the “traffic lights” green, divided by the total number of hand hygiene events. The automatic analysis of the images was validated using 2 human observers. The inter-rater reliability between the observers and the computer was 0.85. As expected, there was some variance between the 6 different human reviewers in the study, but in each pairing the computer agreed with 1 human reviewer to a greater degree than the human reviewers agreed with each other, demonstrating that computer analysis produced classifications as consistent as a human reviewer.
The data on hand hygiene quantity and quality were tested for normality and for the presence of daily or weekly patterns using an autoregressive integrated moving average model. The data were also tested to verify that there was no correlation between the number of staff or patients on the ward. To verify the impact of the intervention, a homogeneity test was applied to each of the phase transitions. The baseline values for quality and quantity was calculated in phases 1 and 4. The study effect size was large (Hedges g = 1.54), resulting in a power of 0.8 after just 16 hand hygiene events. The study had ample power as baseline values were calculated using 240 and 160 hand hygiene events, respectively. Each phase was modeled using a segmented linear regression model and significance was measured using the Mann–Whitney–Wilcoxon 2-tailed test. Statistical calculations were performed using XLstat (Addinsoft, Paris, France).
Results
Analysis of both hand hygiene quality and quantity showed that the addition of a weekly dashboard report in phase 3 had no impact, and therefore data from phases 2 and 3 have been combined for the purposes of further analysis. The difference between the baseline measured at phase 4 and in phase 1 were not statistically significant.
After the introduction of feedback in phase 2, the average number of handwash events per patient day increased from the baseline of 0.91-2.25, and the average percentage of handwash events meeting the quality standard increased from a baseline of 15.7%-46%. Both of these increases were highly statistically significant (P < .0001). Table 1 shows the data for the study along with the 95% confidence intervals. The segmented linear regression in Figure 2 shows that this positive impact on pass rates was sustained for the intervention.
Table 1Study data showing the duration of each phase and the high-level data produced by the system
Handwash events per patient day
Percentage of handwash events that comply with the WHO protocol
Days
Number of handwash events
Average
95% CI
Average
95% CI
Phase 1
10
240
0.91
0.75-1.07
15.7%
10.6%-20.9%
Phases 2 and 3
64
3,206
2.25
2.10-2.39
46.0%
43.8%-48.1%
Phase 4
6
160
1.15
0.73-1.58
13.7%
5.9%-21.5%
CI, confidence interval, WHO, World Health Organization.
After the provision of feedback at the beginning of phase 2, the average quantity handwash events increased by 147% (2.25/0.91), and the average quality of the handwashes improved by 193% (46/15.7). The provision of real-time feedback did change the behavior of a significant cohort of the HCWs, but behavior returned to baseline when feedback was removed.
Discussion
The limitations of the study are that it was not possible to track ABHR usage for 14 hours per day during the 88 days of the study, nor was it possible to count all of the hand hygiene opportunities during this period. Because the study was anonymous, it was not possible to associate improved performance to a specific subset of the staff, or to a more general change in behavior across all the staff.
The main purpose of the study was to assess the impact of using AVA with feedback on the quality and quantity of handwash events. The study did show a very significant increase in both the quality and quantity of hand hygiene over baseline. The baseline quality of 15% is very similar to that reported in the studies by Tschudin-Sutter et al
Because it was not possible to measure the overall rate of hand hygiene, any displacement in the use of ABHR by extra handwashing could not be determined. The combination of AVA and electronic measurement of ABHR usage would deliver a study that can simultaneously measure the quantity and quality of hand hygiene.
The study design compared hand hygiene events on a binary pass–fail basis, but work by Reilly et al
showed that even partial completion of the WHO protocol delivers a significant reduction in the microbial load. Our future evaluations will also consider gradual improvements in technique.
There were no performance incentives used in the study, nor were there any punishments if HCWs achieved a low score. Peer-to-peer learning was reported as HCWs challenged each other to achieve a perfect score. Interaction with the AVA system promoted personal reflection on hand hygiene, as evidenced by comments such as “it made us think about what we are doing.” However, some HCWs simply ignored the system. This anecdotal evidence is in line with studies that report the level of compliance is strongly influenced by cultural factors
the authors suggest a cultural explanation, that the removal of technology is interpreted by HCWs as a signal from management that hand hygiene is no longer a priority. Another explanation, based on cognitive science, is that the constant availability of feedback acts as transitive memory,
the task of memory onto the technology. We see this effect in smartphone use, in which we no longer need to remember things because we have easy access to the information. Whatever the underlying mechanism, this effect should be considered when interpreting the results of technology studies that use an interrupted time series design. Our future research will examine ways of varying the feedback so as to minimize cognitive offloading and encourage the building of muscle memory.
Although the difference between phases 2 and 3 of the study are not statistically significant, there was a small downward trend. Staats et al
points out that engagement with all initiatives reduces over time, and this points to a need to vary the feedback to maintain engagement. In learning technology, adaptive gamification
has been shown to improve learning by tailoring the reward to individual learners’ preferences. Our future research will focus on personalizing the feedback based on individual performance so as to maintain engagement.
Conclusions
The AVA with real-time feedback supports the WHO multimodal strategy for hand hygiene by providing “frequent easy access to training, evaluation, and feedback,” a “reminder in workplace,” and provides a visible “demonstration of the hospital's safety climate.” In line with other studies,
feedback on handwash performance delivered a substantial improvement in the average quantity of handwashes by 147% (2.25/0.91), and the average quality of the handwashes by 193% (46/15.7). The provision of a weekly performance report on hand hygiene at staff meetings had no measurable impact on hand hygiene practice.
Future studies that combine AVA systems with electronic hand hygiene monitoring technology will allow, for the first time, the assessment of both hand hygiene quality and quantity. When the feedback was removed, performance returned to baseline, this was also seen in other studies
and suggests that care should be taken interpreting the results of interrupted times series designs using technology interventions. Further research is required into the impact of cognitive offloading when using technology with real-time feedback to support hand hygiene.
Acknowledgements
The authors would like to thank the patients and staff of Mid Essex Hospital Services NHS Trust for their support during this study.
References
Allegranzi B
Pittet D.
Role of hand hygiene in healthcare-associated infection prevention.
Funding: The authors would like to acknowledge the financial support of the HCAI Technology Innovation Program of the UK Department of Health under the Showcase Hospitals programme and Enterprise Ireland under research grant CP/08/204
30646-7&id=gr1.jpg){kind=link}
30646-7&id=gr2.jpg){kind=link}