Advertisement

Validity and reliability of the proposed core competency for infection control nurses of hospitals in Hong Kong

Published:November 15, 2010DOI:https://doi.org/10.1016/j.ajic.2010.06.019
      Literature review and the Delphi approach were used to draft the core competency items of hospital infection control nurses in Hong Kong. Content validity, internal consistency, and test-retest reliability of the proposed core competency were ensured. The result serves as the foundation of developing training and assessment tools for infection control nurses in Hong Kong.

      Key Words

      There is only half a century of history of infection control nurses (ICNs) in the world with the first ICN appointed in the United Kingdom in 1959.
      • Gardner A.M.N.
      • Stamp M.
      • Bowgen J.A.
      The infection control sister: a new member of the control of infection team.
      Infection control programs were introduced to Hong Kong in the 1980s with limited resources.
      • Seto W.H.
      The progress of hospital infection control in Hong Kong.
      • Chan R.
      My experience on certification of infection control (CIC).
      After the severe acute respiratory syndrome epidemic in Hong Kong in 2003, infection control became popular. The most prominent change was that more and more ICNs were deployed. Hence, the competency level of ICNs became the concern of hospital management and clients.
      Infection control professional groups in the United Kingdom and North America had developed their own competency standards.
      • King D.
      Development of core competencies for infection prevention and control.
      • Horan-Murphy E.
      • Barnard B.
      • Chenoweth C.
      • Friedman C.
      • Hazuka B.
      • Russell B.
      • et al.
      APIC/CHICA-Canada infection control and epidemiology: professional and practice standards.
      However, these publications from overseas standards cannot be employed directly in the local setting because of differences in health care structure, philosophy and management approach, setting and resources, disease epidemiology, and education background. Therefore, we propose a core competency for specialist ICNs in Hong Kong. This paper describes the process of development of the proposed core competency for ICNs in Hong Kong.

      Methods

      Design

      The study was divided into 3 stages. First, the Delphi approach was used to draft the core competency. Next, both qualitative and quantitative approaches were employed to establish the content validity. Last, the reliability was developed based on traditional statistical methods and the Rasch model. This study was approved by the Human Subjects Ethics Subcommittee of The Hong Kong Polytechnic University, Hong Kong, SAR.

      Stage 1: drafting the core competency using the Delphi approach

      E-mail was used as the main method of communication for the Delphi approach. At the beginning, the investigator drafted the competency categories of ICNs based on a literature review and transformed them into a questionnaire for collecting opinions. The experts were asked whether they agreed or disagreed with the categories on the questionnaire. The investigator acted as the study coordinator and decision maker. She reviewed the returned comments and modified the items when necessary. Direct discussions with experts were used when clarification of returned information was needed. A few rounds were held until the consensus was achieved. Next, the investigator drafted the competency items from the literature based on the agreed competency categories. The drafted competency items were transformed into a questionnaire. The experts were then asked whether they agreed or disagreed with the listed items. Each questionnaire had a free-text area to allow the experts to fill in additional categories, items, or other comments. Several rounds were conducted until consensus was achieved.

      Sample

      Purposive sampling was used. A panel of 6 subject matter experts consisted of 2 infection control officers (ICOs) and 4 ICNs who were the leaders of large infection control teams in public hospitals was established.

      Data analysis

      Only categories or items on which 80% or more of the members agreed were included.
      • Powell C.
      The Delphi technique: myths and realities.
      • Staggers N.
      • Gassert C.A.
      • Curran C.
      A Delphi study to determine informatics competencies for nurses at four levels of practice.
      The κ statistics were worked out to quantify the quality of the content validity by the online calculator of the University of Joensuu, Finland.
      • Wynd C.A.
      • Schmidt B.
      • Schaefer M.A.
      Two quantitative approaches for estimating content validity.

      Randolph JJ. Free-marginal multi-rater κ: An alternative to Fleiss’ fixed-marginal multi-rater κ. Paper presented at the Joensuu University Learning and Instruction Symposium 2005, Joensuu, Finland, 14–15 October 14-15, 2005. Available from: http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/1b/c3/27.pdf. Accessed April 10, 2010.

      • Randolph J.J.
      Online κ calculator 2008.

      Results of stage 1

      A total of 4 rounds were conducted. The first 2 rounds were used to develop the competency categories and the latter 2 to develop the competency items. We finally obtained 10 categories and 51 items. The categories included (1) surveillance, (2) program management and evaluation, (3) evidence-based practice, (4) education, (5) team and service management, (6) collaboration and partnership, (7) outbreak investigation and control, (8) research and development, (9) expert knowledge and continuing education, and (10) professional development. The free-marginal multirater κ for 51 items among six panel experts was 0.84 showing excellent agreement among experts.
      • Wynd C.A.
      • Schmidt B.
      • Schaefer M.A.
      Two quantitative approaches for estimating content validity.
      In addition, some of the draft competency items consisted of double-barreled sentences. We split these items to obtain 64 competency items.

      Stage 2: establishing content validity

      A questionnaire consisting of the 64-item draft core competency of ICNs that was developed in stage 1 was employed. The experts were asked to rate these items in terms of relevance to the core competency of ICNs. A 4-point ordinal rating scale from Lynn’s study was used.
      • Lynn M.R.
      Determination and quantification of content validity.
      At the end of the questionnaire, the experts were asked whether all the content domains were included in the draft core competency list. They were requested to specify the missing content in the case of a negative answer.

      Sample

      An expert panel that consisted of 3 subject matter experts who were the ICOs from different major public hospitals was established.
      • Lynn M.R.
      Determination and quantification of content validity.
      • Grant J.S.
      • Davis L.L.
      Selection and use of content experts for instrument development.

      Data analysis

      The content validity index (CVI) and inter-rater agreement was calculated. Free-marginal multirater κ statistics were worked out using the online calculator of University of Joensuu, Finland.

      Randolph JJ. Free-marginal multi-rater κ: An alternative to Fleiss’ fixed-marginal multi-rater κ. Paper presented at the Joensuu University Learning and Instruction Symposium 2005, Joensuu, Finland, 14–15 October 14-15, 2005. Available from: http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/1b/c3/27.pdf. Accessed April 10, 2010.

      • Randolph J.J.
      Online κ calculator 2008.

      Results of stage 2

      The content experts commented that all the dimensions of the core competency were available in the questionnaire. CVI was 0.75, which was fairly satisfactory.
      • Polit E.F.
      • Beck C.T.
      Nursing research: principles and methods.
      The inter-rater agreement was 75% (48/64 items). The free-marginal multirater κ was 0.67 representing a good level of agreement.
      • Wynd C.A.
      • Schmidt B.
      • Schaefer M.A.
      Two quantitative approaches for estimating content validity.

      Stage 3: establishing reliability

      The 64 items developed in stage 1 were transformed into a questionnaire. Subjects were invited to answer 2 identical questionnaires 2 weeks apart.
      • Lynn M.R.
      Determination and quantification of content validity.
      • Rossen E.K.
      • Gruber K.J.
      Development and psychometric testing of the relocation self-efficacy scale.
      • Norweg A.M.
      • Whiteson J.
      • Demetis S.
      • Rey M.
      A new functional status outcome measure of dyspnea and anxiety for adults with lung disease: the dyspnea management questionnaire.
      To minimize the memory effect, the sequence of items in each questionnaire was randomly assigned. Subjects were asked to rate each competency item as very important, important, undecided, not important or not very important. Their demographic information was also requested.

      Sample

      Because of the limited number of ICNs in Hong Kong, their input was reserved for the follow-up studies. Nurses who had previously worked as ICNs (ie, ex-ICNs) were invited to participate in this study. Eighteen ex-ICNs were invited through snowball sampling.

      Data analysis

      Demographic data and test-retest reliability were analyzed by SPSS version 15.0 (SPSS, Inc, Chicago, IL). Rating scale diagnositics were performed using Winsteps (Lawrence Erlbaum Associates, Mahwah, NJ) version 1.0.0, which was a Rasch analysis software package. Coefficient α was worked out by both SPSS and Winsteps.

      Results of stage 3

       Response rate and demographics

      In total, 17 (94.4%) of 18 ex-ICNs completed the second questionnaire. Among the 18 subjects, most of them (77.8%) were female. The majority (50.0%) fell into the 31- to 40-year-old age group. Their work experience in the infection control field ranged from 1 to 10 years with the mean of 5.5. Most of them (94.4%) worked as full-time ICNs, and 83.3% worked in an acute setting. All of them were working in public hospitals, and 50.0% of them worked in a large hospital that had more than 1,000 patient beds.

      Reliability

      The internal consistency for the first returns of 18 ex-ICNs for 64 items was high, with a coefficient α of .98. Seventeen pairs of data were used to check the test-retest reliability. The means of each item on the first and second tests were calculated. The correlation between 2 respective means was then compared and Spearman correlation coefficient was 0.839 (P < .000). This showed that there was no significant difference between the means of 2 returns.

      Rating scale diagnostics

      The category structure is summarized in Table 1. Among the 5 rating categories, only 4 were used. No item was rated as “not very important.” Only 2 frequencies rated as “not important.” This illustrated that the ex-ICNs generally agreed with the competency items that were drafted by the field experts, which matched the validity content. The Rasch-Andrich thresholds increased monotonically from −2.97 to 3.95 showing that the scale was ordered. The thresholds between categories were within the optimal distance, which should be between 1.4 and 5 logits.
      • Bond T.G.
      • Fox C.M.
      Applying the Rasch model. Fundamental measurement in the human sciences.
      This illustrated that each category defined a distinct position on the scale. Results showed that rating scale of the questionnaire was functional. The probability graph of the rating scale is visualized as Fig 1. Each category has a distinct peak in the graph. The thresholds were presented at the intersections of rating scale categories.
      Table 1Summary of category structure
      CategoryObserved
      RatingLabelCount%Rasch-Andrich threshold
      1Not very important00-
      2Not important20None
      3Undecided464−2.97
      4Important58551−0.98
      5Very important519453.95

      Conclusion

      The core competency of ICNs drafted by panel experts using Delphi approach was valid and reliable. The information on content validity was enhanced by free-marginal multirater κ in addition to CVI. The functional rating scale of the questionnaire ensured the quality of data collected.
      • Bond T.G.
      • Fox C.M.
      Applying the Rasch model. Fundamental measurement in the human sciences.
      To promote the quality of infection control service, it is important to equip ICNs in Hong Kong with reliable and valid training and assessment. Core competency is the major component for the development. The result of this study builds the foundation on this future direction.

      References

        • Gardner A.M.N.
        • Stamp M.
        • Bowgen J.A.
        The infection control sister: a new member of the control of infection team.
        Lancet. 1962; 280: 710-711
        • Seto W.H.
        The progress of hospital infection control in Hong Kong.
        J Hong Kong Med Assoc. 1989; 41: 222-223
        • Chan R.
        My experience on certification of infection control (CIC).
        Hong Kong Infection Control Nurses’ Association Newsletter. 2005; 7: 10-12
        • King D.
        Development of core competencies for infection prevention and control.
        Nurs Stand. 2005; 19: 50-54
        • Horan-Murphy E.
        • Barnard B.
        • Chenoweth C.
        • Friedman C.
        • Hazuka B.
        • Russell B.
        • et al.
        APIC/CHICA-Canada infection control and epidemiology: professional and practice standards.
        Am J Infect Control. 1999; 27: 47-51
        • Powell C.
        The Delphi technique: myths and realities.
        J Adv Nurs. 2003; 41: 376-382
        • Staggers N.
        • Gassert C.A.
        • Curran C.
        A Delphi study to determine informatics competencies for nurses at four levels of practice.
        Nurs Res. 2002; 51: 383-390
        • Wynd C.A.
        • Schmidt B.
        • Schaefer M.A.
        Two quantitative approaches for estimating content validity.
        West J Nurs Res. 2003; 25: 508-518
      1. Randolph JJ. Free-marginal multi-rater κ: An alternative to Fleiss’ fixed-marginal multi-rater κ. Paper presented at the Joensuu University Learning and Instruction Symposium 2005, Joensuu, Finland, 14–15 October 14-15, 2005. Available from: http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/1b/c3/27.pdf. Accessed April 10, 2010.

        • Randolph J.J.
        Online κ calculator 2008.
        (Available from) (Accessed April 10, 2010)
        • Lynn M.R.
        Determination and quantification of content validity.
        Nurs Res. 1986; 35: 382-385
        • Grant J.S.
        • Davis L.L.
        Selection and use of content experts for instrument development.
        Res Nurs Health. 1997; 20: 269-274
        • Polit E.F.
        • Beck C.T.
        Nursing research: principles and methods.
        7th ed. Lippincott Williams & Wilkins, Philadelphia2004 (p. 423-4)
        • Rossen E.K.
        • Gruber K.J.
        Development and psychometric testing of the relocation self-efficacy scale.
        Nurs Res. 2007; 56: 244-251
        • Norweg A.M.
        • Whiteson J.
        • Demetis S.
        • Rey M.
        A new functional status outcome measure of dyspnea and anxiety for adults with lung disease: the dyspnea management questionnaire.
        J Cardiopulm Rehabil. 2006; 26: 395-404
        • Bond T.G.
        • Fox C.M.
        Applying the Rasch model. Fundamental measurement in the human sciences.
        2nd ed. Lawrence Erlbaum Associates, New Jersey2007 (p. 219-33)