Influence of the way results are presented on research interpretation and medical decision making: the PRIMER collaboration randomized studies

Abstract

Background: The manner of presentation of research results may affect how clinicians interpret research and make clinical decisions. The authors evaluate whether the use of confidence levels improve research interpretation and decision making compared with P values and 95% confidence intervals.

Methods: The 2 Presentation and Interpretation of Medical Research (PRIMER) studies were 3-arm randomized trials. PRIMER 1 presented results of 5 fictitious scenarios with P values (P), P plus 95% confidence intervals (P + CI), or P, CI, and confidence levels (P + CI + CL); PRIMER 2 compared P + CI + CL, P + CI, and P + CL. Clinicians were asked to identify the correct interpretation of scenarios in terms of statistical and clinical significance and then indicate the intended decision making in terms of treatment recommendation.

Results: Seventy-five and 246 clinicians participated in PRIMER 1 and PRIMER 2, respectively. In PRIMER 1, P+CI+CL was superior to P + CI and P (P < 0.05); the latter 2 arms did not differ significantly. Decision making was not significantly different between arms. In PRIMER 2, P+CI+CL resulted in better interpretation than P + CI (P = 0.03), with no difference between P + CI and P + CL. In combined analysis, the odds of correct interpretation were higher for P+CI+CL than P+CI (odds ratio = 1.73, P=0.005, 95% CI= 1.19--2.52). Decision making was better for P + CI+ CL (P = 0.03). On multivariate analysis, the P + CI+ CL arm and clinicians with statistics training, not in private practice, or participating in PRIMER 1 had better interpretation. The P + CI+ CL arm was the only factor improving decision making.

Conclusions: Presenting research with a combination of P values, confidence intervals, and confidence levels leads to better interpretation and decision making by clinicians.