Welcome
Welcome
What does “evidence synthesis” mean, and why this title rather than “systematic review”?
 
Evidence synthesis, simply put, is the collecting, analyzing, and synthesizing of “evidence” to answer a question. It is a term that is being used more recently to capture the full body of work. You can read a short summary of how it is used by various disciplines for the purpose of providing evidence in The Value of Evidence Synthesis. Nat Hum Behav.  2021 May;5(5):539. PMID: 34017128 DOI: 10.1038/s41562-021-01131-7.
 
Review articles, from literature reviews to qualitative meta-aggregations to systematic reviews, make up most evidence synthesis writing. However, if you are writing a new or revising a current policy or guideline, you also need to synthesize the best evidence to support the policy or guideline.
 
A systematic review, which requires a comprehensive, reproducible search strategy, to find all the evidence, and formal reporting structure to minimize bias, is one type of review article, but there are many others as well. You can read about the various types here: Meeting the review family: exploring review types and associated information retrieval requirements. Sutton A, Clowes M, Preston L, Booth A. Health Info Libr J. 2019 Sep;36(3):202-222. PMID: 31541534. doi: 10.1111/hir.12276.
 
To get started, submit an Evidence Synthesis Assistance Request and the librarian will get in touch with you within 1-2 business days.
 
Introduction to Evidence Synthesis Types

Choosing which evidence synthesis to conduct

Is this your first review article? Are you are confused by all the types – narrative, critical, rapid, scoping, integrative, unbrella, qualitative research synthesis, systematic, meta-aggregation, meta-ethnography, etc.? Some of these may be more familiar to you than others and several sound similar. I would like to point out that Policy Review is listed under family 9 (Pupose Specific Reviews). So yes, although it isn't always considered a review, writing and/or reviewing/revising policy absolutely falls within the scope of evidence synthesis writing. 
 
Even if you scanned Meeting the review family: exploring review types and associated information retrieval requirements. Sutton A, Clowes M, Preston L, Booth A. Health Info Libr J. 2019 Sep;36(3):202-222. PMID: 31541534. doi: 10.1111/hir.12276, you may have seen this statement "Forty-eight distinct review types were identified." Think about that - 48 types of review articles, over half of which have been created within the last decade. How in the world do you choose? These review types were categorised into 7 broad review ‘families’: Traditional, Systematic, Review of Reviews, Rapid, Qualitative, Mixed Methods and Purpose Specific.
 
There are tabs on the side where you can read more about each review type within a family, but as an overview this is a simplified chart showing the famililes. This more detailed PDF iconChart of review types may, or may not, help sort through them. But for a quick ssearch, you can use this online tool developed by staff at Knowledge Translation Program.

Simple chart of review families

Systematic Reviews (with or without meta-analysis)
 
What is a systematic review?
 
A systematic review is a type of evidence synthesis that uses systematic, unbiased and reproducible methods to identify, screen and critically appraise ALL relevant evidence to answer a clearly defined question, and to analyze and synthesize data from the studies included in the review.

Carrie Price, Health Professions Librarian at Towson University, created a short video highlighting the basics of systematic reviews.
 
Not sure what the difference is between a literature review and a systematic review is? Here is a short video, explaining the basic differences. Or are you wondering if you should do a scoping or a systematic review? Librarian Carrie has a video explaining the differences between those.

Basics
  • On average, it takes 18 months to complete and you will need at least 2, preferably 3, people on your team.
  • You need to have a clearly defined question (a scoping search may help you with this) and a protocol developed and registered prior to beginning your conduct your comprehensive literature search. You can, but not necessary, follow the PRISMA-P for developing and reporting your protocol.
  • To locate all the evidence, you will need to conduct extensive complex literature searches of all the major biomedical databases, grey literature and possibly do some hand searching. If you don’t feel you have the time or skills for advanced literature searching, contact us for assistance (or look at the “How We Can Help” tab).
  • Contact IS to have EndNote added to your profile (this can ONLY be accessed through MyDesk) so you can manage citations and screening. If you’ve not used EndNote before, they have tutorials you can access, or contact us for assistance.
  • Remember to follow the PRISMA reporting guidelines and checklist, and include a flowchart within your report.
 
A systematic review, especially if it is your first, can be quite intimidating. So we've developed an Interactive guide that might help (remember, right click on all links on the graphic to open them in a new tab). 
 
Systematic Review - Stages and Steps graphic
 
How can we help you?
In 2011, the Institute of Medicine published Finding What Works in Health Care: Standards for Systematic Reviews. Within these standards, librarians were identified as key members of the evidence writing team (Standard 2.1.3 p. 47-8), and within Standard 3 have roles and expertise that makes their involvement necessary.
Standard 3.1 Conduct a comprehensive systematic search for evidence
Required elements:
3.1.1 Work with a librarian or other information specialist trained in performing systematic reviews (SRs) to plan the search strategy
3.1.2 Design the search strategy to address each key research question
3.1.3 Use an independent librarian or other information specialist to peer review the search strategy

Beyond providing support services as defined by these standards, librarians can assist in many aspects of evidence synthesis research to provide support, consultative guidance or co-authorship – depending on the level of assistance needed. To get started, submit an Evidence Synthesis Assistance Request and the librarian will get in touch with you within 1-2 business days.
 
Support from Library Staff 
  • performing a search for you
  • retrieving full-text articles
  • advising on determining the best review type
  • doing a search for any existing current report or protocol on your research question
 
Librarians as Consultants*
Librarians are available to serve as consultants. In this capacity, librarians can offer assist by 
  • providing guidance on the process for your evidence synthesis
  • providing help formulating the review question and framework (PICO, PICo, PEO, etc. see other options here
  • providing training on search techniques for different databases and/or grey literature
  • providing training on setting up alerts for new articles when your research topic has new literature published during your evidence synthesis writing time frame
  • providing training on using EndNote and setting up groups for screening
  • working with you to iteratively develop a comprehensive search strategy 
  • translating the search strategy to each database, conduct the search and submit abstrcts to the EndNote library
  • providing documentaion of search strategies for each database and all searches conducted per PRISMA reporting guidelines

*As a consultant, it is expected that the librarian will be acknowledged for their contribution on the completed review submitted to any journal for publication.


Librarians as Co-authors**
As a co-author the librarian joins your project team and contributes by partnering on all the above plus:
  • assisting with the development and registration of a protocol (when applicable)
  • updating the search – specified based on expected length of review – to locate new reports
  • improving transparency and reproducibility of the literature search using the PRISMA-S checklist (when applicable)
  • writing the search strategy methodology for the review
  • assisting in completing the appropriate PRISMA checklist and flow chart for publication (when applicable)
  • reviewing and editing any review prior to submission to a publication

**Following the guidelines from ICMJE on the role of authors, it is expected that the librarian will be listed as an author on the completed review submitted to any journal for publication.

    Databases
    For a systematic review, you need to find all the relevant information before you begin the screening process. It is necessary that you systematically, comprehensively and iteratively search all the relevant databases you have access to. But what exactly is a database? And what is the difference between a database and a publisher site like Science Direct?
    • Publisher sites will have indexes of all the publications they publish, and only their publications, in all the disciplines. Whether you can access any articles, however, depends on whether it is an open-access article or journal, or you pay for a subscription. This will be discussed more in Hand Searching.
    • Databases, such as ClinicalKey for Nursing, MEDLINE and CINAHL (Cumulative Index of Nursing and Allied Health Literature), are subscribed to through a vendor (such as EBSCO) and index articles from journals from hundreds of publishers combined. Databases are usually discipline specific, so the ones we have here at Children’s are all within the medical field in some way. (There are exceptions, such as PubMed, PEDro and Trip, which are freely accessible). Even when you pay to access the database, your access to articles is still limited to whether the article or journal is open-access, or you pay for a subscription to all the journals within that database.
    While you can search all the databases listed in the Databases link, it wouldn’t be the best use of your time and wouldn’t be necessary depending on your research question. It’s good to know where to search to gather all the relevant information. Do you know what each database specializes in? If not, here is a basic guide.
     
    Biomedical
    Nursing
    PT, OT, SLP
    Business and Leadership
    Other
    Grey Literature
    What is it?
    Grey Literature is work produced by individuals or organizations for non-commercial purposes. This includes conference abstracts/presentations/proceedings; trial data; government publications; reports (such as white papers, working papers); dissertations/theses; patents;  policies & procedures, non-commercial translations, bibliographies, and official documents not published commercially (primarily government reports and documents) and more. If you want to learn more, this is a short but concise video from the Univ of MN Libraries reviewing what it includes and this short video from Yale Univ offers some tips on finding it..
     
    Why include this?
    Remember, a systematic review “uses systematic, unbiased and reproducible methods to identify, screen and critically appraise ALL relevant evidence”. The problem with published studies is that too often they are biased and only published if they show positive, or significant results. It is important to know about other evidence that might show no results as well.
     
    How does one find Grey Literature?
    • If you are, or a member of your team is, a member of an organization, you can look through their website for conference papers, webinars (which may have PDFs or PowerPoints included), reports, consensus papers, etc. on the topic you are reviewing.
    • Dissertations and thesis papers can sometimes be found through Google Scholar, but we also have access to some dissertations from scholarly universities through Open Dissertations via EBSCOhost.
    • Ask a colleague who may have access or a connection to a conference, technical report, or other on a topic they are an expert in related to your review.
    • Contact the librarians.
    Some sources
    Websites (just a few of the many)
    Dissertations, Theses and Preprints
    • Center for Research Libraries - A resource for institutions outside of the U.S. and Canada.
    • EThOS - Electronic Theses Online Service contains doctoral theses from candidates in the UK.
    • OSF Preprints - A general preprint repository that covers architecture, business, engineering, life sciences, physical sciences and mathematics, arts and humanities, education, law, medicine and health sciences and social and behavioral sciences, among others.
    Clinical Trials
    e-Books, articles and checklists
     For researchers interested in building their awareness of and skills in finding the best grey literature, this is the definitive guide.
     
     
    Looking for more?
    Temple University Libraries – The librarians at Temple University have put together an exhaustive list of grey literature sources, far more than I want to add here.
    Hand Searching
    What is it?
    Hand searching is what it sounds like. Once you have searched the databases, done your grey literature searching, it’s time to do some additional searching “by hand”. Some of this you will have done already when you searched for grey literature, and some you may already do whenever you search for relevant information. But some you may not have considered.
     
    Why include this?
    It is your last step in ensuring that you searched for “ALL relevant evidence” for your research topic.
     
    How does one do hand searching?
    Hand searching includes:
    • Looking over conference proceedings
    • “Hand searching” tables of contents of relevant journals
    • Checking reference lists of articles you have included in your appraisal
    • Checking sources such as ResearchGate, OSF, or other open source repositories for articles
    • Contacting authors if articles are unavailable, or they have grey literature/research that would be useful to incorporate
     
    When doing a hand search it is vital that you record your finding
    • Journals searched, if relevant citations found and used, should be described by journal name and years searched.
    • Conference proceedings/abstracts listed by organization, conference title and year as well as abstract or poster referenced
    • Authors contacted should also be included with any information they provide.
     
    Registering Your Protocol
    As you plan your evidence synthesis, you should, and sometimes need to, think about registering a protocol for your review. Why? As explained in Tawfik, G.M., Giang, H.T.N., Ghozy, S. et al. Protocol registration issues of systematic review and meta-analysis studies: a survey of global researchers. BMC Med Res Methodol 20, 213 (2020). https://doi.org/10.1186/s12874-020-01094-9, registering a protocol of a systematic review, similarly to registering a study, "helps minimize bias in the conduct and reporting of the review, reduce duplication of effort between groups, and keep systematic reviews updated."
     
    This needs to be completed BEFORE you begin screening - but during or right after you develop your title and inclusion criteria. As with all things related to evidence synthesis, the type of synthesis you are reporting will determine if, how, and where you register your protocol. You can register your protocol for systematic, rapid, and umbrella reviews with PROSPERO, but NOT scoping reviews. OSF (from the Center for Open Science) has options for various levels of protocol registration for any type of research/evidence synthesis project. So, for instance, while you cannot register a protocol for a scoping review at PROSPERO, you can at OSF. 
    Finding Additional Studies

    Now that you have learned, at least a little, about some of the databases, how can you know if you've found ALL relevant studies for your review before you begiin? Below are some tools that will assist you in finding additional studies. While I have listed the tool and the link to it, along with a brief description, if you need assistance don't hesitate to contact me for training on the tool.

    • Yale MeSH Analyzer - analyzes citations for keywords,/MeSH terms: this tool helps you determine if there are additional keywords/terms you could use by comparing ones used at a glance, rather than scrolling down abstracts.
       
    • PubMed PubReMiner - in comparing abstracts this tool shows you keywords,/MeSH terms used, how often they were used, and how many studies they were used in: helpful in seeing patterns.
       
    • NLM MeSH on Demand - finds relevant MeSH terms within abstracts, that may or may not have been listed within the list provided in the MeSH listing.
       
    • SR Accelerator: Polyglot Search Translator - translates search across different databases. A HUGE time saver as keywords/MeSH terms/truncation/modifiers/etc. differ between databases and it can take time to know how to best configure the search for each.
       
    • CitationChaser - increases efficiency and reduces time in hand searching, by allowing you to look for articles that referenced a citation, or references used by that citation: saves considerable time in hand-searching!
    Citation Management and Screening Articles
    • EndNote - Used as a citation management tool to collect literature during your literature search process. Also very helpful as a citation/reference tool when writing your report. This tip sheet show you how to request this from IS. PDF iconRequesting EndNote 20.pdf.
    • Rayyan - Free screening tool for blind screening between reviewers to reduce bias. Easy to use, and saves time.
    Clinical Appraisal and Risk of Bias Tools
    After you’ve screened the full-text articles, it’s time to critically appraise the included studies and determine their risk of bias. There is no “right” way of doing this, it will largely depend on the type of you are conducting. Below are some of the possible options.
    Reporting Your Review

    Guidelines, frameworks, and expectations from journal publishers have grown in the last few decade to improve reproducibility, reduce bias, and eliminate waste.

    • PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) is "an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating the effects of interventions, but can also be used as a basis for reporting systematic reviews with objectives other than evaluating interventions"1. It has reporting guidelines (called extensions) for a range of evidence synthesis reviews, with more in development. Citing use of the PRISMA statement, or another reporting guideline, in articles has been a source of some confusion. To assist authors in properly stating how they used the PRISMA reporting guidelines, a statement was released in 2021. PDF iconHow to properly use the PRISMA Statement. 2021.pdf
       
    • EQUATOR Network "is an “umbrella” organisation that brings together researchers, medical journal editors, peer reviewers, developers of reporting guidelines, research funding bodies and other collaborators with mutual interest in improving the quality of research publications and of research itself." 2 One of the resources available through EQUATOR is a page of Toolkits. Included here are tools to help you select the best reporting guideline for your article. While many use the PRISMA reporting guideline, there are other options available for different types - just as there are different frameworks for question formation depending on the type of evidence synthesis.
       
    • Scoping Review Framework - JBI provides an overview of the enhancement of the original framework developed by Arksey and O’Malley in 2005.

    References
    1. http://www.prisma-statement.org/ 
    2. https://www.equator-network.org/about-us/equator-network-what-we-do-and-how-we-are-organised/
    3. Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil, H. Chapter 11: Scoping Reviews (2020 version). In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis, JBI, 2020. 
    4. Updated methodological guidance for the conduct of scoping reviews. Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, McInerney P, Godfrey CM, Khalil H.JBI Evid Synth. 2020 Oct;18(10):2119-2126. doi: 10.11124/JBIES-20-00167.PMID: 33038124.