Consensus on consensus: a synthesis of consensus estimates on human-caused global warming

Most scientists agree that current climate change is mainly caused by human activity. That has been repeatedly demonstrated on the basis of surveys of the scientific opinion as well as surveys of the scientific literature. In an article published today in the journal Environmental Research Letters (ERL) we provide a review of these different studies, which all arrive at a very similar conclusion using different methods. This shows the robustness of the scientific consensus on climate change.

This meta-study also shows that the level of agreement that the current warming is caused by human activity is greatest among researchers with the most expertise and/or the most publications in climate science. That explains why literature surveys generally find higher levels of consensus than opinion surveys. After all, experienced scientists who have published a lot about climate change have, generally speaking, a good understanding of the anthropogenic causes of global warming, and they often have more peer-reviewed publications than their contrarian colleagues.

Figure: Level of consensus on human-induced climate change versus expertise in climate science. Black circles are data based on studies of the past 10 years. Green line is a fit through the data.

The initial reason for this review article was a specific comment by Richard Tol on John Cook’s literature survey as published in ERL in 2013. Cook found a 97% consensus on anthropogenic global warming in the scientific literature on climate change. This article has both been vilified and praised. Tol argued that Cook’s study is an outlier, but he did so by misrepresenting most other consensus studies, including the survey I undertook while at the Netherlands Environmental Assessment Agency (PBL). To get a gist of the discussion with Tol see e.g. this storify I made based on my twitter exchanges with him (warning: for climate nerds only). Suffice to say the authors of these other consensus studies were likewise not impressed by Tol’s caricature of their work. This is how the broad author team for the current meta-analysis arose, which shows that Cook’s literature survey fits well within the spectrum of other studies.

The video below provides a great overview of the context and conclusions of this study:

Surveys show that among the broad group of scientists who work on the topic of climate change the level of consensus is roughly between 83 and 97% (e.g. Doran, Anderegg, Verheggen, Rosenberg, Carlton, Bray, Stenhouse, Pew, Lichter, Vision Prize). If you zoom in on the subset of most actively publishing climate scientists you find a consensus of 97% (Doran, Anderegg). Analyses of the literature also indicate a level of consensus of 97% (Cook) or even 100% (Oreskes). The strength of literature surveys lies in the fact that they sample the prime locus of scientific evidence and thus they provide the most direct measure of the consilience of evidence. On the other hand, opinion surveys can achieve much more specificity about what exactly is agreed upon. The latter aspect – what exactly is agreed upon and how does that compare to the IPCC report- is something we investigated in detail in our ES&T article based on the PBL survey.

As evidenced by the many –unfounded- criticisms on consensus studies, this is still a hot topic in the public debate, despite the fact that study after study has confirmed that there is broad agreement among scientists about the big picture: our planet is getting warmer and that is (largely) due to human activity, primarily the burning of fossil fuels. A substantial fraction of the general public however is still confused even about the big picture. In politics, schools and media climate change is often not communicated in accordance with the current scientific understanding, even though the situation here in the Netherlands is not as extreme as e.g. in the US.

Whereas the presence of widespread agreement is obviously not proof of a theory being correct, it can’t be dismissed as irrelevant either: As the evidence accumulates and keeps pointing in the same general direction, the experts’ opinion will logically converge to reflect that, i.e. a consensus emerges. Typically, a theory either rises to the level of consensus or it is abandoned, though it may take considerable time for the scientific community to accept a theory, and even longer for the public at large.

Although science can never provide absolute certainty, it is the best method we have to understand complex systems and risks, such as climate change. If you value science it is wise not to brush aside broadly accepted scientific insights too easily, lest you have very good arguments for doing so (“extraordinary claims require extraordinary evidence”). I think it is important for proper democratic decision making that the public is well informed about what is scientifically known about important issues such as climate change.

More info/context/reflections:

Dutch version at sister-blog “klimaatverandering”

Column by first author John Cook in Bulletin of the Atomic Scientists

Stephan Lewandowsky on the psychology of consensus

Collin Maessen tells the backstory starting with Richard Tol’s nonsensus

Ken Rice at …And Then There’s Physics

Dana Nuccitelli in the Guardian

Sou at HotWhopper

Amsterdam University College (AUC) news item

 

Richard Tol misrepresents consensus studies in order to falsely paint John Cook’s 97% as an outlier

John Cook warned me: if you attempt to quantify the level of scientific consensus on climate change, you will be fiercely criticized. Most of the counterarguments don’t stand up to scrutiny however. And so it happened.

The latest in this saga is a comment that Richard Tol submitted to ERL, as a response to John Cook’s study in which they found 97% agreement in the scientific literature that global warming is human caused. Tol tries to paint Cook’s 97% as an outlier, but in doing so misrepresents many other studies, including the survey that I undertook with colleagues in 2012. In his comment and his blogpost he shows the following graph:

Richard Tol comes to very different conclusions regarding the level of scientific consensus than the authors of the respective articles themselves (Oreskes, 2004; Anderegg et al., 2010; Doran and Kendall Zimmerman, 2009; Stenhouse et al., 2013; Verheggen et al., 2014). On the one hand, he is using what he calls “complete sample” results, which in many cases are close to meaningless as an estimate of the actual level of agreement in the relevant scientific community (that counts most strongly for Oreskes and Anderegg et al). On the other hand he is using “subsample” results, which in some cases are even more meaningless (the most egregious example of which is the subsample of outspoken contrarians in Verheggen et al).

The type of reanalysis Tol has done, if applied to e.g. evolution, would look somewhat like this:

• Of all evolutionary biology papers in the sample 75% explicitly or implicitly accept the consensus view on evolution. 25% did not take positon on whether evolution is accepted or not. None rejected evolution. Tol would conclude from this that the consensus on evolution is 75%. This number could easily be brought down to 0.5% if you sample all biology papers and count those that take an affirmative position in evolution as a fraction of the whole. This is analogous to how Tol misrepresented Oreskes (2004).

• Let’s ask biologists what they think of evolution, but to get an idea of dissenting views let’s also ask some prominent creationists, e.g. from the Discovery Institute. Never mind that half of them aren’t actually biologists. Surprise, surprise, the level of agreement with evolution in this latter group is very low (the real surprise is that it’s not zero). Now let’s pretend that this is somehow representative of the scientific consensus on evolution, alongside subsamples of actual evolutionary biologists. That would be analogous to how Tol misrepresented the “unconvinced” subsample of Verheggen et al (2014).

No @RichardTol, a survey of vocal opponents of the mainstream is not a valid estimate of scientific consensus https://t.co/G8b2oK6Ojw (1/2)

— Bart Verheggen (@BVerheggen) September 15, 2015

Especially not as that sub-sample includes non-scientists who publicly oppose mainstream science: meaningless to base estimate on. (2/2)

— Bart Verheggen (@BVerheggen) September 15, 2015

Collin Maessen provide an detailed take-down of Richard Tol on his blog, quoting extensively from the scientists whose work was misrepresented by Tol (myself included). The only surveys which are not misrepresented are those by Bray and von Storch (2007; 2010). This is how I am quoted at Collin’s blog RealSkeptic:

Tol selectively quotes results from our survey. We provided results for different subsamples, based on different questions, and based on different types of calculating the level of agreement, in the Supporting Information with our article in ES&T. Because we cast a very wide net with our survey, we argued in our paper that subgroups based on a proxy for expertise (the number of climate related peer reviewed publications) provide the best estimate of the level of scientific consensus. Tol on the other hand presents all subsamples as representative of the scientific consensus, including those respondents who were tagged as “unconvinced”. This group consists to a large extent of signatories of public statements disapproving of mainstream climate science, many of whom are not publishing scientists. For example, some Heartland Institute staffers were also included. It is actually surprising that the level of consensus in this group is larger than 0%. To claim, as Richard Tol does, that the outcome for this subsample is somehow representative of the scientific consensus is entirely nonsensical.

Another issue is that Richard Tol bases the numbers he uses on just one of the two survey questions about the causes of recent climate change, i.e. a form of cherry picking. Moreover, we quantified the consensus as a fraction of those who actually answered the question by providing an estimate of the human greenhouse gas contribution. Tol on the other hand quantifies the consensus as a fraction of all those who were asked the question, including those who didn’t provide such an estimate. We provided a detailed argument for our interpretation in both the ES&T paper and in a recent blogpost.

Tol’s line of reasoning here is similar to his misrepresentation of Oreskes’ results, by taking the number of acceptance papers not just as a fraction of papers that take position, but rather as a fraction of all papers, including those that take no position on current anthropogenic climate change. Obviously, the latter should be excluded from the ratio, unless one is interested in producing an artificially low, but meaningless number.

Some quotes from the other scientists:

Oreskes:

Obviously he is taking the 75% number below and misusing it. The point, which the original article made clear, is that we found no scientific dissent in the published literature.

Anderegg:

This is by no means a correct or valid interpretation of our results.

Neil Stenhouse:

Tol’s description omits information in a way that seems designed to suggest—inaccurately—that the consensus among relevant experts is low.

Doran:

To pull out a few of the less expert groups and give them the same weight as our most expert group is a completely irresponsible use of our data.

You can read their complete quotes at RealSkeptic.

See also this storify of my twitter discussion with Richard Tol.

Survey confirms scientific consensus on human-caused global warming

• A survey among more than 1800 climate scientists confirms that there is widespread agreement that global warming is predominantly caused by human greenhouse gases.
• This consensus strengthens with increased expertise, as defined by the number of self-reported articles in the peer-reviewed literature.
• The main attribution statement in IPCC AR4 may lead to an underestimate of the greenhouse gas contribution to warming, because it implicitly includes the lesser known masking effect of cooling aerosols.
• Self-reported media exposure is higher for those who are skeptical of a significant human influence on climate.

In 2012, while temporarily based at the Netherlands Environmental Assessment Agency (PBL), my colleagues and I conducted a detailed survey about climate science. More than 1800 international scientists studying various aspects of climate change, including e.g. climate physics, climate impacts and mitigation, responded to the questionnaire. The main results of the survey have now been published in Environmental Science and Technology (doi: 10.1021/es501998e).

Level of consensus regarding attribution

The answers to the survey showed a wide variety of opinions, but it was clear that a large majority of climate scientists agree that anthropogenic greenhouse gases are the dominant cause of global warming. Consistent with other research, we found that the consensus is strongest for scientists with more relevant expertise and for scientists with more peer-reviewed publications. 90% of respondents with more than 10 climate-related peer-reviewed publications (about half of all respondents), agreed that anthropogenic greenhouse gases (GHG) are the dominant driver of recent global warming. This is based on two different questions, of which one was phrased in similar terms as the quintessential attribution statement in IPCC AR4 (stating that more than half of the observed warming since the 1950s is very likely caused by GHG).

Figure 1. The more publications the respondents report to have written, the more important they consider the contribution of greenhouse gases to global warming. Responses are shown as a percentage of the number of respondents (N) in each subgroup, segregated according to self-reported number of peer-reviewed publications.

Literature analyses (e.g. Cook et al., 2013; Oreskes et al., 2004) generally find a stronger consensus than opinion surveys such as ours. This is related to the stronger consensus among highly published – and arguably the most expert – climate scientists. The strength of literature surveys lies in the fact that they sample the prime locus of scientific evidence and thus they provide the most direct measure of the consilience of evidence. On the other hand, opinion surveys such as ours can achieve much more specificity about what exactly is agreed upon and where the disagreement lies. As such, these two methods for quantifying scientific consensus are complementary. Our questions possibly set a higher bar for what’s considered the consensus position than some other studies. Furthermore, contrarian viewpoints were likely overrepresented in our study compared with others.

No matter how you slice it, scientists overwhelmingly agree that recent global warming is to a great extent human caused.

The role of scientific consensus in moving the public debate forward

Mike Hulme had an interesting essay at The Conversation, the main message of which was

In the end, the only question that matters [for the public debate about climate change] is, what are we going to do about it?

Hulme correctly argues that the basic science is clear enough so that for society the important issues to discuss are not science related, but policy related. I argued much the same here. He writes:

What matters is not whether the climate is changing (it is); nor whether human actions are to blame (they are, at the very least partly and, quite likely, largely); nor whether future climate change brings additional risks to human or non-human interests (it does).

Let’s leave the minor quibble aside that AR5 puts the anthropogenic contribution at ‘extremely likely’ having caused more than half of the recent global warming.

The part where I disagree with Hulme is where he argues that showing the existence of a scientific consensus on the above (it is warming; it’s due to us; it’s bad news) somehow stands in the way of  getting society to discuss that most important question. I think the opposite is true. It is the continuous doubt about the science, sowed by those who oppose a serious discussion about what to do, that is a stumbleblock. Showing that a consensus amongst experts exists would enable society to more swiftly move on to the important conversation on what to do about it. I agree with Hulme that on this deeply ethical question there is, and ought to be, a multitude of opinions.

As Stephan Lewandowsky and John Cook write in a response to Mike Hulme:

The public’s perception of that scientific consensus is necessary to stimulate political debate about solutions.

Another element that’s missing from this discussion is that scientific and ideological arguments  should be clearly distinguished from each other (“is” vs “ought”).

Unfortunately, ideological arguments are often dressed in a sciency-looking cloak. From that perspective, I appreciate the honesty in Lindzen stating blunty “we’ll all be dead by then”, the obvious implication being: so why care. That’s indeed what a lot comes down to: How do you value the future compared the present?