The librarians at the Bio-Medical Library are expert searchers who can support and guide investigators in the following systematic review tasks. See our Systematic Review Service webpage to get started.
Writing a Systematic Review
“A Systematic Review attempts to collate all empirical evidence that fits pre-specified eligibility criteria to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing reliable findings from which conclusions can be drawn and decisions made."
The key characteristics of a systematic review are:
- a clearly stated set of objectives with an explicit, reproducible methodology;
- a systematic search that attempts to identify all studies that would meet the eligibility criteria;
- an assessment of the validity of the findings of the included studies, for example through the assessment of risk bias; and
- a systematic presentation, and synthesis, of the characteristics and findings of the included studies.” (1)
Systematic reviews are reproducible, transparent and the methods used are documented. A systematic review answers specific questions which are fully described in the protocol. Bias is minimized by having at least two people on the team do each step. A systematic review may take a year or more to complete.
Recommended guidelines for conducting a systematic review have been published by Cochrane, PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and IOM (Institute of Medicine). IOM guidelines recommend partnering with a librarian.
Formulate Research Question
Consider whether a systematic review is necessary and whether you have the time and resources to conduct a systematic review. Has a systematic review of your topic already been done? Library staff can help you search the literature to determine this. A well-formulated research question will typically include five elements, designated by the acronym PICOS:
- Population(s) - A precise description of the group of participants or patients to be studied, the clinical problem and the health care setting, e.g., the disease being addressed and possibly where the care is provided, a hospital, clinic, etc. The population should be a clinically suitable group of participants or patients.
- Interventions(or Exposures) - A clear description of the main interventions being considered, e.g. a new treatment, a new process of care, social or educational intervention etc.
- Comparison(s):- A clear description of what the intervention is being compared to, e.g. a treatment already in use, a placebo. The comparison is usually between an experimental group, which is treated with the new therapy, and a control group, which is treated with the standard therapy or with a placebo.
- Outcome(s) - The outcomes are being assessed. What were the changes in health status of the participants due to the intervention? Were there changes in mortality, morbidity, symptoms, quality of life?
- Study Design - The kind of studies to be included in the review: What kind of studies will be excluded and an explanation of the inclusion and exclusion criteria? Were the participants in the research study properly recruited? Was the intervention adequately measured?
2. Develop Research Protocol
A research protocol is “a precise and detailed plan for the study of a biomedical problem or for a regimen of therapy.”(2) “A protocol is important because it pre-specifies the objectives and methods of the systematic review. For instance, a protocol specifies outcomes of primary interest, how reviewers will extract information about those outcomes, and methods that reviewers might use to quantitatively summarize the outcome data." (1)
If your systematic review addresses effects of interventions and strategies to prevent, diagnose, treat and monitor health conditions, for which there is a health related outcome, consider free registration of your review protocol with PROSPERO: an international prospective register of systematic reviews.
3. Conduct Literature Search
The literature search should be exhaustive enough to develop a comprehensive list of potentially relevant studies. With a librarian’s assistance, select core and specialized databases to search. Databases searched will depend on the topic, but may include Medline, Cochrane Controlled Trials Registry, Web of Science, Scopus, Embase. Other literature might include conference proceedings, bibliographies of relevant articles, unpublished studies, hand searching key journals, pharmaceutical companies and personal communication with experts in the field. Utilize librarians to provide assistance with formulation of search strategy and execution of searches.
Typically, a broader search is done first and the strategy is then refined and increased in complexity. Records should be kept to document the dates of inclusion for each database searched, dates searches were performed and updated, details of strategies used and numbers of results obtained. Consider using a reference management tool such as Endnote, which will enable bibliography creation, citation sharing and elimination of duplicate records.
4. Select Studies per Protocol
After eliminating duplicates, at least two researchers should screen titles and abstracts based on inclusion and exclusion criteria stated in the protocol. You should define the selection criteria, consider the nature of the question being posed, the type of relevant study design, and the weakest acceptable study design acceptable for selection.3 Selected full text articles should be obtained for second stage screening by at least two reviewers. A log of excluded studies with reason for exclusion should be kept. Use of Excel will facilitate this process.
5. Appraise Studies per Protocol
Selected full text articles should be appraised for methodological quality by at least two reviewers. Use of a study quality checklist is recommended to help determine whether studies meet the criteria of the protocol. Such a checklist should help determine questions such as: Were the patients assigned to groups using a random allocation? Was the allocation sequence concealed to patients and clinicians? Was the study unduly influenced by bias that either over or under estimated the true effect of the intervention? (2)
6. Extract Data
Design a data extraction form and use more than one reviewer to do data extraction. Cochrane makes a good practice data extraction form available as a guide for developing your own form.
7. Analyze Results
Tabulate results from individual studies. Examine forest plot ( a statistical graph designed to measure the relative strength of treatment effects from several quantitative scientific papers studying the same question). Explore sources of heterogeneity. Consider meta-analysis. Perform sensitivity analysis; examine funnel plots (a statistical graph designed to check for the existence of publication bias in systematic reviews and meta-analyses). Make list of excluded studies available to interested readers.
8. Interpret Results
Consider limitations (including biases), strength of evidence, applicability, economic implications and implications for future research. When interpreting the results of the various studies these are the kinds of questions you should have in mind:
- Is there evidence on critical and important outcomes for the population and intervention of interest described in the question?
- Are the searches adequate?
- Is there a risk of publication and related bias?
- Is the methodological quality of the included studies good enough?
- Are the results consistent from study to study?
- Are there enough data for precise estimation of effect?
- Are the observed effects of substantial clinical, not just statistical significance? (3)
- Liberati, Allessandro, et al, “The PRISMA Statement for Reporting Systematic Reviews and Meta-Analysis of Studies that Evaluate Health Care Interventions: Explanation and Elaboration,” PLoS Med. 2009; 6(7): 1-28.
- Stedman TL. Stedman's Medical Dictionary. Philadelphia: Lippincott, Williams & Wilkins; 2006.
- Kahn, Kalid, et al, Systematic Reviews to Support Evidence-Based Medicine, Hodder Arnold, London, 2011.