In the chart below we explore how the latest available estimates of cumulative excess deaths during this pandemic compare to mortality patterns prior to the pandemic. This comparison is anchored to 2019 and we compare the severity of the pandemic’s excess death toll to 123 causes of death documented in the Global Burden of Disease database (level-3 aggregation).
Why? Comparisons with top causes of death help dimension the severity of the pandemic’s mortality toll by providing an intuitive reference point. Statements such as “the pandemic is claiming more lives than heart attack or stroke before the pandemic” may foster a better appreciation of the severity of the pandemic than “the excess death ratio is X per 100,000”.
The chart shows the top cause of death in 2019 that is exceeded by estimated excess death toll during the pandemic (see below for more details on the tweaks that need to be applied to make this comparison valid). We focus on the leading causes of death, which we take to be the Top 3. A country is colored red when excess deaths exceed cause of death #1. The different shades of orange apply this logic to causes of death #2 and #3. Green refers to countries where excess deaths do not exceed the top 3 causes. Excess deaths may be negative for some countries, in which case they do not exceeded the death toll of any of the 123 disease families considered.
This analysis is subject to important caveats about standard errors and bias. See below for a detailed discussion of results and methods. Note also that this is a clickable map. The tooltips provide a step-by-step guide for each country on how the results were derived. For a non-interactive version of the map, click on “download image” within the map or here.
New excess death estimates
A myriad of factors complicate our assessment of the true mortality impact of the pandemic. Inadequacies in CRVS systems (civil registration and vital statistics) are one source of uncertainty. Another is the presence of indirect impacts on mortality. These can inflate the mortality toll in the case of un- or under-managed diseases due to strained hospital resources for example. Or they could reduce the mortality toll – think of the virtual absence of the flu season as more people wear masks and keep distance from others.
The concept of excess deaths helps alleviate these challenges. By capturing mortality patterns beyond what would be expected under normal circumstances, we get an amalgamated insight into the effects of factors such as the misattribution of underlying cause of death or the indirect channels through which the pandemic raises mortality.
Excess death measures, however, are far from perfect. First, even in countries with solid CRVS systems that provide timely data, excess deaths remain estimates since they require an assessment of the counterfactual of what would have happened under normal circumstances. Second, and more important, excess death estimates require all-cause mortality data, which creates a serious limitation to getting a globally comprehensive sample. As a result, excess deaths are available only for some 80 countries.
That has recently changed. New estimates have become available that directly estimate excess deaths in data-poor environments. The excess death estimates by The Economist are currently the best-in-class. They’re not without limitations themselves – see details in the methods section below – but they give us a solid starting point and are notable for their transparent disclosure of assumptions, methods and standard errors around the results.
They represent a top-down approach that shouldn’t be seen as substituting for more detailed bottom-up country-level or even subnational analysis; rather the estimates complement such efforts and the true assessment of the pandemic’s impact will in any case need to take into account information based on more than just one method.
Interpretation of results
The chart simply shows what the cumulative excess death estimates of The Economist imply for the severity of the pandemic. We take the estimates as given and focus in this map on the mid-points of the confidence intervals that surround them.
A companion post contrasts the results based on mid-point excess death estimates with those based on officially reported COVID-19 stats as well as those based on the lower bounds and upper bounds of the excess death estimates.
Taking the mid-point estimates at face value for now, what does this map tell us?
The results also suggest that the pandemic has been severe in countries where one would have predicted a priori a milder impact. Many countries in Africa, for example, are very young. Their demographic structure should provide some protection against this age-discriminating infectious disease pandemic. The excess death estimates suggest that these advantages may have been offset by other factors, which must be a combination of higher infection prevalence, higher age-adjusted infection fatality rates, greater measurement challenges and/or a greater contribution from indirect deaths.
Finally, below is a chart that shows the number of countries that have suffered high severity of excess mortality with reference to the leading causes of death. For example, the first bar shows the number of countries on the Y axis (and the share of countries in the global total of 189 as a label on the bar), for which the Top #1 cause of death is exceeded by the measure of excess mortality. The last bar shows the number of countries where none of the leading causes of death (the top 3) are exceeded. The results suggest that the leading causes are exceeded for the vast majority of countries.
First, the standard errors of the estimates. The two visualization immediately above illustrate the sometimes large confidence intervals around the mid-point estimates of excess deaths. The first map shows the country distribution by leading cause of death if we were to take the upper-bound estimates of the 90% confidence interval around the mid-points. The second map shows the same for the lower-bound estimates of that confidence interval.
Second, the possibility of bias. Not only are the excess death measures estimated with a considerable degree of imprecision in especially the poorer countries, there are also reasons why the estimates may be biased (i.e. misrepresentative of reality). The Economist acknowledges two main reasons why and it turns out these are not just some after-thought but important qualifiers:
“Because most countries that report excess deaths are rich or middle-income, the bulk of the data used to train our model comes from such places. The patterns that the model detects in these areas could thus be an inaccurate guide to the dynamics of the pandemic in poor countries. A similar caveat applies to our estimates for countries that have suffered lots of excess deaths for reasons other than the pandemic, such as war or natural disasters.”
The second bullet point on bias is especially important.
The estimates may be well-calibrated against the available observations but we cannot tell with much confidence how well they hold up for those countries where information is sparse or not available. Should additional information be published in the future, the estimates can be checked and improved. But until then we have to accept that not only the margins of uncertainty are large, but also there may be an element of bias in the results.
The direction of bias is likely upward since reports on the ground in especially lower-income countries do not suggest a very considerable excess death toll. If we were to simplistically divide the world into groups of lower- and higher-income countries which are respectively characterized by data environments that are poor and rich, then structural differences between these groups could easily lead to an upward bias. Let’s focus on four structural differences:
For all of the above these reasons, we should approach the estimates as tentative and treat the results shown here as preliminary. We should also use complementary information at the country level to assess the true severity of the pandemic. Having said that, the estimates and results represent the best possible effort to convey a globally consistent picture of the impact of the pandemic and remain a good starting point.
Details on methodology
In what follows, we expand on the extensive footnote to this map, which explains the methodology and articulates the various caveats that apply to this analysis.
The excess death estimates. As noted, we take the cumulative excess death estimates by The Economist, which start for each and every country on Jan 1, 2020. Please note:
We will compare the excess death estimates with pre-pandemic mortality data on a yearly basis. For this reason, we need to adjust the excess death estimates into yearly averages by scaling them down by 365 / the number of days passed since Jan 1, 2020.
The “excess severity” ratio. We express cumulative excess deaths as a ratio to pre-pandemic mortality (and not a proportion since it is a merely a comparison and not a share, i.e. the numerator is not part of the denominator). A few points on this:
Comparing “excess severity” with top causes of death. The next step in the analysis is to compare the ratio of cumulative excess deaths (adjusted and mid-points) to the proportionate mortality rates of the top causes of deaths. Let’s break this down: