Systematic Review and Evidence Synthesis

What is risk of bias assessment?

Risk of bias assessment (sometimes called "quality assessment" or "critical appraisal") helps to establish transparency of evidence synthesis results and findings. A risk of bias assessment is a defining element of systematic reviews and often performed for each included study in the review.  

Evidence syntheses strive to eliminate bias in their findings.  Individual studies that are included in a synthesis may include biases in their results or conclusions, for example design flaws that raise questions about validity of findings or an overestimate of intervention effect.  Risk of bias assessment generally is not required with evidence synthesis methods outside of systematic reviews. However, this may depend on the evidence synthesis method that you are utilizing.

Disciplinary considerations for risk of bias assessment.

This Cochrane Training presentation helps to navigate the steps of risk of bias assessment in systematic reviews in human medicine, but some things might not apply to other disciplines. For disciplines outside of human medicine, the Critical Appraisal Skills Programme provides checklists that can be applied to a diverse array of study types, and the Mixed Methods Appraisal Tool can be a helpful resource if included studies use mixed methods. 

Sources for identifying risk of bias tools

Risk of bias in scoping reviews

Scoping reviews don't typically include a risk of bias assessment

“A key difference between scoping reviews and systematic reviews is that the former are generally conducted to provide an overview of the existing evidence regardless of methodological quality or risk of bias (4, 5). Therefore, the included sources of evidence are typically not critically appraised for scoping reviews.”

From Tricco, A., Lillie, E., Zarin, W., O'Brien, K., Colquhoun, H., Levac, D., . . . Straus, S. (2018). PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169(7), 467-473. https://doi.org/10.7326/M18-0850.  

Representing risk of bias assessment

Risk of bias assessments can be represented in table format in an evidence synthesis, showing each included study and how strong it is across several quality criteria for that particular study type for example*: 

 

If a high proportion of studies are assessed with a high risk of bias, caution should be used when interpreting results for your evidence synthesis.  More information can be found in Chapter 8 of the Cochrane Handbook.    

Last Updated: May 31, 2024 5:03 PM