What is an evidence map and how does it differ from an systematic review?
Evidence on the effectiveness or prevalence of a health condition or an environmental exposure and health outcome can be presented in many different ways. In those situations where primary studies are available, it is possible to conduct evidence synthesis using the primary studies based on systematic reviews and meta-analyses. In systematic reviews, such syntheses follow the steps of framing an answerable question using the population under consideration, the specific intervention or the exposure, the comparison groups, and the specific outcomes. Beyond identifying the studies and a narrative summary of the key findings, no specific summary statistic of the findings from the primary studies is presented \cite{khan2003five}. In a meta-analysis, on the other hand, in addition to narrative summaries, findings from the primary studies are combined to produce summary estimates; besides, in a meta analysis a formal statistical examination of the biases involved in selecting only those studies that were large in size and had positive outcomes is also conducted -- this is referred to as "publication bias". Further, a meta analysis provides an opportunity to examine how the findings would differ or examination of the relationships between different entities in the analyses, referred to as sensitivity analyses or meta-regression \cite{deeks2019analysing}. In meta-analysis and systematic review, the analyst aims to collect and process the primary sources and then critically appraise the body of the literature to identify biases in the conduct of the study or whether a cause and effect association can be justified based on the nature of the studies, and also attempts to conduct a narrative or a statistical summary of the key findings.
Sometimes this is not necessarily the aim of the research, in particular when information about an exposure and an outcome is emerging on a particular topic. In those circumstances, the strategy is to identify what literature are available out there without necessarily conducting either a full critical appraisal of the body of the literature, or statistically summarising the findings. The goal in these situations is to identify the body of evidence that has accrued and the populations studied, the interventions or the exposures assessed, the comparison groups studied, and the key outcomes. In addition, analysts may also be interested to study the data sources, and other details that are not necessarily the main aims of a formal systematic review. Here, rather the aim is to "map" the "landscape" of the research. This is where an "evidence map" or "evidence gap map" is useful, as these information are useful for the analyst or the researcher to identify "what is out there" and what gaps in the available evidence exists.
Hence an evidence map is about laying the "landscape" of what evidence exists about a research question and they provide a visual overview of existing evidence. A
World Bank document on evidence gap maps states that the idea is to conduct a visual summary of the evidence. In the context of environmental health, defining evidence maps, the journal
"Environmental Evidence" states,
Systematic Maps are overviews of the distribution and abundance of evidence in relation to multifaceted elements of a broad question of policy or management relevance. The process and rigour of the mapping exercise is the same as for systematic review except that no evidence synthesis is attempted to seek an answer to the question.
For example, suppose you are interested to find out what recent evidence exists in terms of heart disease risks of exposure to ambient air pollution in the studies published in the past one year, but you do not want to critically appraise each article or formally synthesise the literature, you can construct an evidence map about how many studies on heart health effects of air pollution were reported in the scholarly literature databases since first January of 2019 (and up to a certain date of your choice, say 31st December 2019), in order to understand the landscape of the current research and then on the basis of your findings, you could construct a visual overview of these topics. These might include for example, how many studies were published on the relationship between ambient air quality and exposure to particulate pollution and people suffering from heart attacks (acute myocardial infarction), how many studies investigated the association between long term exposure to air pollution and ischaemic heart disease. You would then construct the maps of these various considerations and would be able to obtain a snapshot of the research and policy landscape. What you would not do would be to summarise the results of these studies. That would be the objective of a scoping review rather than an evidence map \cite{munn2018systematic}. So in summary:
- An evidence map is a "map" like structure
- It uses some principles of systematic review
- It provides a list of articles and resources
- It provides a visual summary in the form of tables and graphs as to what evidence are available on a particular topic
- There is no data analysis or narrative summary of the findings involved unlike that of a scoping review
- There may be some quality appraisal, but it is not the main purpose of an evidence map unlike that of systematic review or meta-analysis.
In this tutorial, I'd like to present the steps of a rapid evidence map construction. For full details and more detailed work, please review the list of citations and references I have included at the end of this document. Here, we will aim to construct a rapid and "miniaturised" evidence map on an environmental topic. Using the skills of searching literature and some data analyses such as using a spreadsheet or an open source statistical tool such as R or Python, you can construct a visual or knowledge map or evidence map.
Steps of constructing an evidence map
Step 1: Frame a PECO or PICO (population, exposure or intervention, comparison group, outcomes) formatted question
This is the first step. Develop a question based on the four criteria:
P: represents population or people.
Who are the human beings or which population will benefit from this question? For example, if you are interested in learning about what evidence exists that exposure to air pollution is associated with increased heart disease risks among elderly, then the population should reflect elderly (those above 65 years old). You can further narrow down the population to mean certain ethnicity (Whites or Maori in New Zealand), or only men or only women, or restrict the age band to above 65 years and below 85 years to exclude the old olds, or those who live in cities and towns and not in villages and so on.
E or I: If you are studying the effects of interventions, then the preferred entity would be interventions, otherwise E.
Suppose you want to study the effectiveness of improved stoves on respiratory illnesses as improved stoves are known to reduce indoor air pollution \cite{smith2007monitoring}. Besides, you know that indoor air pollution is a risk factor for heart disease \cite{Fatmi2016}. In that situation, you will use an "intervention". Alternatively, you can also use E or exposure, if all you are interested in studying what are the heart health risks of those who are exposed to air pollution.
Comparator or comparison group or C:
The comparison group will be in comparison with the exposure. So, if you are studying exposure to air pollution and heart disease among elderly people, then your comparison group will be elderly people who are either NOT exposed to air pollution at all, or if that is not possible to lesser extent of air pollution (this is perhaps more realistic). If you are studying effectiveness of improved stoves as intervention and reduced heart disease risk in the elderly, then improved stoves is the intervention whose effect you want to study. Your comparison group will be those who do not use improved stoves in the household.
O is for outcomes
After you have defined P, E or I, and C, define the outcomes that you want to study. For example, in our case we want to study what evidence exists in the literature since 2019 about the excess heart disease risk among people who are exposed to air pollution? Outcome for us can be defined in terms of heart disease risk or death from heart disease. This might include acute myocardial infarction (also abbreviated as AMI, or known as heart attack or stroke), or hospital admission with AMI. We have mentioned this as "heart disease". However, the more precise you can be in defining the outcome, the more focused will be your review.
Once you have settled on the P, E or I, C, and O, it is time for you to put them together in the form of a table and a question. The question is referred to as PICO (or PECO) question. You accompany this question by developing a table (Table \ref{794788}). To give you an example, we have put together a question and a table for our evidence map of all studies on air pollution and heart disease published since 2019.
Here is the PICO formatted question
"Compared with those with lower levels of exposure to air pollutants, those who are exposed to higher levels of air pollution, what is the risk of heart disease among older adults?"
Here is the accompanying table: