High quality of evidence is uncommon in Cochrane systematic reviews in Anaesthesia, Critical Care and Emergency Medicine

Abstract

Background: The association between the quality of evidence in systematic reviews and authors' conclusions regarding the effectiveness of interventions relevant to anaesthesia has not been examined.

Objective: The objectives of this study were: to determine the proportion of systematic reviews in which the authors made a conclusive statement about the effect of an intervention; to describe the quality of evidence derived from outcomes in reviews that used the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) working group system for grading the quality of evidence; and to identify review characteristics associated with conclusiveness.

Design: Cross-sectional analysis of Cochrane systematic reviews from the Anaesthesia, Critical Care and Emergency Review Group was undertaken.

Data sources: The Cochrane webpage was used to identify reviews for inclusion (http://.ace.cochrane.org/).

Eligibility criteria: New and updated versions of systematic reviews published up to 17 September 2015 were eligible. Protocols for systematic reviews were excluded.

Results: A total of 159 reviews were included. GRADE was used in 103 reviews (65%). Of these, high-level evidence for the primary outcome was identified in 11 reviews (10%). The main reasons that quality of evidence for the primary outcome was downgraded were risk of bias (n = 44; 43%) and imprecision (n = 36; 35%). Authors of 47% (n = 75) of the total number of reviews made conclusive statements about the effects of interventions. Independent predictors of conclusiveness in the subgroup of reviews with GRADE assessments were quality of evidence for the primary outcome (odds ratio 2.03; 95% confidence interval: [1.18 to 3.52] and an increasing number of studies included in reviews (OR 1.05; 95% CI: [1.01 to 1.09]).

Conclusion: It was common for conclusive statements to be made about the effects of interventions despite evidence for the primary outcome being rated less than high quality. Improving methodological quality of trials would have the greatest impact on improving the quality of evidence.