Developing valid test bank of surveillance case study scenarios for inter-rater reliability

Published:February 11, 2022DOI:


      • Case study scenarios for surveillance were developed and distributed to infection preventionists of varying levels of experience.
      • Item difficulty and item discrimination were applied to each question to determine validity of written scenarios.
      • Results highlight that case studies developed by expert Ips may not elicit the intended response in a test group of individuals with vary experience and written scenarios can benefit from measurement and evaluation.


      Case studies are utilized for training on National Healthcare Safety Network (NHSN) healthcare associated infection surveillance definitions. Item discrimination and item analysis were applied to case studies to identify questions that most accurately assess infection preventionists (IPs) application of surveillance definitions.

      Key Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to American Journal of Infection Control
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. Hospital Acquired Conditions. Center for Medicare and Medicaid Services. Accessed on August 12, 2021.

        • Pogorzelska-Maziarz M
        • Gilmartin H
        • Reese S.
        Infection prevention staffing and resources in U.S. acute care hospitals: results from the APIC MegaSurvey.
        Am J Infect Control. 2018; 46: 852-857
        • Wright MO
        • Allen-Bridson K
        • Hebden J.
        Assessment of the accuracy and consistency in the application of standardized surveillance definitions: a summary of the American Journal of the Infection Control and National Healthcare Safety Network case studies, 2010-2016.
        Am J Infect Control. 2017; 45: 607-611
        • Holmes K
        • Moinuddin M
        • Steinfeld S.
        Interrater reliability: item analysis to develop validquestions for case-study scenarios.
        Infect Control Hosp Epidemiol. 2022; 50: 960-962
        • Quaigrain K
        • Arhin A.K.
        Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation.
        Cogent Educ. 2017; 41301013