Systematic Reviews
This page provides CROWN partners and stakeholders with essential information, tools, and guidance for conducting systematic reviews. Explore specific topics such as search strategy and bias assessment, each leading to dedicated pages with comprehensive details and relevant links.
What is Systematic Review?
- Volume of Research: The growing volume of research literature makes it difficult for individual decision-makers to assess this vast quantity of primary research to make the best healthcare decisions. A Systematic Review consolidates the information across research findings and presents it in a digestible manner.
- Purpose: Systematic reviews provide an up-to-date summary of research knowledge on interventions, diagnostic tests, prognostic factors, or other healthcare topics by systematically assessing primary research.
- Addressing Bias: Systematic reviews address the primary issue with ad hoc research and selection, which is bias. They employ methods to minimize bias in the summaries and syntheses of research, similar to those used in primary research studies.
- Identifying Gaps in Knowledge: Systematic reviews also aim to identify gaps in existing knowledge or limitations in previous studies, which can guide future research efforts.
Steps of a Systematic Review
Cochrane Reviews developed by the Cochrane Collaboration are considered the gold standard in systematic reviews due to their rigorous methodology and comprehensive approach. The rigorous and transparent approach of Cochrane Reviews ensures that healthcare providers, policymakers, and researchers can access reliable and up-to-date evidence to inform their decisions.
Here is a step-by-step process for conducting a systematic review, as suggested by Cochrane.
1. Prepare for a Review
- Formulate research question(s): Systematic reviews should address answerable questions and fill important gaps in knowledge. The detailed specification of the review question requires consideration of several key components, such as ‘PICO’, an acronym for Population, Intervention, Comparison(s), and Outcome.
- Define eligibility criteria: Systematic review researchers should pre-specify criteria for including and excluding studies in the review. Eligibility criteria may be based on the PICO elements of the review question or include other information from studies that have addressed the questions. In addition to the PICO elements, authors need to consider a priori what study designs, publication status (peer-reviewed articles and/or ‘grey’ literature), and language are likely to provide reliable data.
2. Search for Relevant Studies
Different databases tend to be used in different fields. It is thus important to choose databases most relevant to one’s research area. Listed below are some important recommendations:
- Refer to established guidelines such as PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) or the Cochrane Handbook for guidance on database selection.
- Review similar systematic reviews to identify commonly used databases.
- To minimize bias and ensure comprehensive coverage, use multiple databases and combine general and specialized databases.
- Finally, do not forget to document the search process in enough detail so that others can replicate it if needed. It is recommended to work closely with an experienced medical/healthcare librarian or information specialist.
3. Screen Studies
Select studies based on pre-defined inclusion and exclusion criteria. Decisions on selecting and excluding studies should be documented for all records identified by the search. Numbers of records are sufficient for exclusions based on the initial screening of titles and abstracts. Broad categorizations are sufficient for records classed as potentially eligible during an initial screening of the full text.
Each study rather than each report is the unit of interest. Multiple reports of the same study should be collated, and review authors will need to choose and justify which report (the primary report) to use as a source for study results, particularly if two reports include conflicting results.
4. Extract Data and Assess Risk of Bias
‘Data’ can be any information about a study, including methods, participants, settings, interventions, outcomes, results, and investigators. Review authors should plan in advance what data will be required for their systematic review and develop a strategy for obtaining them. At least two people should work independently to extract study characteristics from reports of each study and decide on ways to resolve disagreements.
Detailed information about a study should be collected to conduct an assessment of the risk of bias. Risk-of-bias assessment should be conducted using the tool most appropriate for the design of each study depending on the study design of an included study.
5. Analyze and Synthesize Data
Guided by the study’s research questions, conduct synthesis of data and study outcomes. The types of outcome data can be dichotomous, continuous, ordinal, count or rate, and time-to-event data. Depending on the data, different ways can be used to compare outcome data between studies. For example, dichotomous outcomes can be compared between intervention groups using a risk ratio, an odds ratio, a risk difference, etc. Results extracted from study reports may need to be converted to a consistent format for analysis.
Conduct meta-analysis when appropriate to combine results from two or more separate studies. Although meta-analyses can improve precision and answer questions not posed by individual studies, they may also mislead seriously, particularly if specific study designs, within-study biases, and variation across studies, are not carefully considered.
6. Report Findings
A clear statement of findings and discussion can help people make informed decisions and increase the usability of systematic reviews. Review authors are discouraged from making recommendations about healthcare decisions, but they can highlight different actions that show particular patterns of values or factors that determine a decision, such as cost.
Search Strategy
Here are two tools to help you find systematic reviews and clinical topics on PubMed:
- PubMed Systematic Reviews Filter: This filter assists in finding citations specifically related to systematic reviews. To use it, add systematic [sb] to the search terms.
- PubMed Clinical Queries: This filter assists in finding citations specifically related to systematic reviews. To use it, add systematic [sb] to the search terms.
Bias Assessment
Systematic reviews are crucial for informed clinical and policy decisions, but their quality can vary significantly and may be influenced by biases, particularly when non-randomized studies are included. Assessing the quality of these reviews is essential to ensure they are rigorous, reproducible, and comprehensive.
Risk of Bias 2 (RoB 2)
Risk of Bias 2 (RoB 2) is suitable for analyzing the risk of bias in the findings of individually-randomized, parallel-group trials. The assessment is specific to a single randomized trial that reports the relative effect of two interventions or intervention strategies on a particular outcome.
The updated RoB2 guide and template can be found here.
ROBINS-I
ROBINS-I is used to assess the risk of bias in the results of non-randomized studies that compare the health effects of two or more interventions. Observational studies, such as cohort studies, case-control studies, pre- and post- studies can be analyzed using this tool.
The most recent guide and template can be found here.
ROBINS-E
ROBINS-E provides a structured approach to assessing the risk of bias in observational epidemiological studies that evaluate the effects of exposures (including environmental, occupational and behavioural exposures) on human health. The assessment should examine the strength of evidence about the presence, nature, or potential effect of an exposure on an outcome.
The tool can be found here.
AMSTAR 2
AMSTAR 2 evaluates systematic reviews of healthcare interventions, encompassing both randomized and non-randomized studies. It includes 16 items, simplified response categories, and a detailed user guide to identify high-quality reviews without assigning an overall score.
The tool can be found here.
Reporting Guidelines
The PRISMA statement sets evidence-based guidelines for reporting systematic reviews and meta-analyses, ensuring high reporting standards. PRISMA 2020 updates include refined guidance on identifying, selecting, appraising, and synthesizing studies, with improved item structure for easier use.
The PRISMA reporting guidelines include:
- A 27-item Checklist (with an abridged version)
- Flow Diagram
- Extension for Scoping Reviews
- Extension for Network Meta-Analysis
The Cochrane Handbook for Systematic Reviews of Interventions
This handbook is a comprehensive guide focused on systematic reviews that assess the effects of interventions. It details the specific methodologies employed by Cochrane to conduct these reviews. Cochrane Reviews synthesize primary research to generate new insights into the effects of various interventions in clinical, public health, or policy contexts. These reviews aim to provide users with a balanced summary of the potential benefits and harms of interventions, along with an indication of the certainty of the findings. They also compare the effectiveness of different interventions, helping users select the most appropriate intervention for specific situations. The primary goal of Cochrane Reviews is to inform decision-makers about health and healthcare.
Methods Manual for Community Guide Systematic Reviews
The manual provides a comprehensive overview of the methods used in Community Guide systematic reviews, which underpin the recommendations and findings of the Community Preventive Services Task Force (CPSTF). Developed by the Community Guide Program (CGP), this manual explains the processes and methodologies employed by CPSTF and CGP in conducting these reviews.
It is important to note that the manual is not a step-by-step guide to performing a systematic review. Instead, it aims to help users understand the systematic review process that supports specific CPSTF recommendations and findings. This understanding is intended to increase decision-makers’ confidence in selecting and implementing recommended interventions and to assist researchers in critically assessing Community Guide findings and identifying research areas that need further investigation.
JBI Manual for Evidence Synthesis
This manual is a comprehensive guide for authors conducting systematic and scoping reviews using JBI methodologies. Each chapter focuses on synthesizing various types of evidence to address different clinical and policy-related questions. Presented in an online wiki format, the manual allows for the rapid inclusion of updates and developments in JBI methodologies and methods for evidence synthesis. This dynamic format ensures that the manual remains current and relevant.
This study aims to evaluate the effectiveness of interventions to improve adherence to micronutrient supplementation during pregnancy, a critical factor in the success of prenatal nutrition programs. Employing the Cochrane Collaboration Methodology, the systematic review will analyze data from randomized and nonrandomized controlled studies, focusing on key outcomes like heterogeneity and publication bias. The results will provide crucial insights for clinicians and policymakers, offering evidence-based strategies to enhance supplement adherence and improve maternal health outcomes.
Read the article summary here.
This study assesses IFAS adherence among pregnant women in Sub-Saharan Africa to address anemia-related health issues. The systematic review involved a comprehensive literature search, applying the Newcastle-Ottawa quality assessment scale and PRISMA guidelines. The meta-analysis, using Review Manager and Comprehensive Meta-Analysis software, pooled effect sizes from 23 studies. Key findings indicate that counseling, education on IFAS, and frequent antenatal care visits significantly improve compliance. The study highlights the need for targeted interventions and educational programs to enhance IFAS compliance and inform policy-makers.
Read the article summary here.
This study estimates the national level of adherence to iron and folic acid (IFA) supplementation and its determinants among pregnant women in Ethiopia. Following PRISMA guidelines, an extensive database search and quality assessment using the Newcastle-Ottawa tool were conducted, with meta-analysis performed on 20 studies using a random-effects model. The pooled adherence rate was 46.15%, with higher rates observed in Addis Ababa and Tigray. Factors contributing to higher adherence included receiving supplemental information, good knowledge, early and frequent antenatal care visits. Barriers included fear of side effects and forgetfulness. The study highlights the need to enhance counseling services, improve antenatal care, and address adherence barriers to boost maternal health and reduce anemia risk.
Read the article summary here.
This study analyzed evidence surrounding self-management of iron and folic acid (IFA) supplementation pre-pregnancy, during pregnancy, and postnatally. A systematic review of articles published through November 2020 included studies comparing self-managed IFA supplementation to health worker-initiated supplementation, focusing on health outcomes, user preferences, and cost-effectiveness. However, no studies met the review criteria, highlighting a lack of primary research on this topic. The review underscores the need for further research to better understand self-care practices in dietary supplementation during pregnancy, aiming to provide more choices to enhance reproductive health and rights.
Read the article summary here.
This study reviews qualitative research on how women of reproductive age and healthcare providers perceive and experience iron supplementation programs. A search of published literature identified 12 studies from 17 countries, using systematic review methods to analyze social determinants of iron supplementation. The review highlighted seven key domains: cultural norms, political and socio-economic factors, education and communication, social organization, healthcare access, food and nutrition availability, and adherence. The findings underscore the importance of incorporating social, cultural, and economic factors into iron supplementation guidelines to improve relevance and effectiveness in diverse contexts.
Read the article summary here.
This study globally identifies adherence to iron and folic acid supplementation (IFAS) among pregnant women and associated factors. A systematic search covered databases up to October 2020, including cohort, case-control, and cross-sectional studies, with quality assessed using the Joanna Briggs Institute tool. A meta-analysis synthesized pooled odds ratios using a random-effects model, with 18 studies selected. Key findings showed that adherence to IFAS was higher in primipara vs. multipara (OR 3.91), anemic vs. non-anemic (OR 1.09), and those knowledgeable about IFAS and anemia (OR 2.48 and 0.32, respectively). The study highlights the need for targeted education and interventions to improve IFAS adherence.
Read the article summary here.
References:
Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. J. H. W. (2019). Cochrane handbook for systematic reviews of interventions. Hoboken: Wiley.
Ervin, A.-M. (2022, January 24). Welcome to the SRMA course. Lecture presented in 340.606.71: Methods of Conducting Systematic Reviews and Meta-Analyses at Johns Hopkins Bloomberg School of Public Health.
Lasserson, T. J., Thomas, J., & Higgins, J. P. T. (2023). Chapter 1: Starting a review. In J. P. T. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, & V. A. Welch (Eds.), Cochrane handbook for systematic reviews of interventions (version 6.4, updated August 2023). Cochrane. Available from www.training.cochrane.org/handbook
University of Maryland. (2024, May 8). Research Guides: Systematic Review: Steps of a Systematic Review. Steps of a Systematic Review – Systematic Review – Research Guides at University of Maryland Libraries. https://lib.guides.umd.edu/SR/steps