Confiabilidade da autoavaliação das práticas de segurança do paciente instituída pelo Sistema Nacional de Vigilância Sanitária: um estudo piloto

Abstract

Introduction: The Self-Assessment of Patient Safety Practices is an initiative of the National Health Surveillance System for the identification and monitoring of risks in Brazilian health services through self-inspection. Annually, participating services can be classified as highly compliant with safe practices based on 21 indicators. Although this initiative is promising, it is necessary to assess the reliability of the simple indicators and the high adherence composite indicator. Objective: To analyze self-assessment indicators of Patient Safety Practices in hospitals in Rio Grande do Norte. Method: The design was a cross-sectional observational pilot study with analysis of the agreement between information from self-assessment and face-to-face inspection, which is considered the gold standard. The sample was random with 10 participants from the 27 ICUbed hospitals in the state of Rio Grande do Norte who responded to the Self-Assessment in 2019. After collecting self-assessment data (raw and revised by health surveillance), a health surveillance team performed an independent face-to-face inspection at hospitals to verify onsite compliance. The reliability of the 21 simple indicators and the high adherence composite indicator was analyzed with general agreement index, Kappa coefficient, intraclass correlation coefficient, Bland and Altman plots, and chi-square test. Results: although the ICG of the criteria individually did not present a percentage above 90%, this was significant for most criteria. And the composite indicator ICG agreed to compare the revised self-assessment by VISA and face-to-face inspection. Likewise, the ICC of 0.89 and 0.80 (95% CI) in both comparisons. Kappa values were quite divergent, ranging from -0.32 to 1.00, for simple indicators, but the composite indicator showed relative reliability. The criteria with the least reliability are those related to assistance protocols. The Bland and Altman graph showed adequate agreement between the pairs of measurements from the Self-Assessment revised by VISA and face-to-face inspection (p> 0.05). Conclusions: This pilot study provides a potentially useful method for periodically measuring the reliability of patient safety practice self-assessments. Indicators that need a review of their description and clarification have been identified to increase their reliability. Although problems were detected in some simple indicators, the high adherence composite indicator was reliable both in terms of raw data sent by hospitals and those reviewed by health surveillance and face-to-face inspection.