Consensus on consensus: a synthesis of consensus estimates on human-caused global warming

Most scientists agree that current climate change is mainly caused by human activity. That has been repeatedly demonstrated on the basis of surveys of the scientific opinion as well as surveys of the scientific literature. In an article published today in the journal Environmental Research Letters (ERL) we provide a review of these different studies, which all arrive at a very similar conclusion using different methods. This shows the robustness of the scientific consensus on climate change.

This meta-study also shows that the level of agreement that the current warming is caused by human activity is greatest among researchers with the most expertise and/or the most publications in climate science. That explains why literature surveys generally find higher levels of consensus than opinion surveys. After all, experienced scientists who have published a lot about climate change have, generally speaking, a good understanding of the anthropogenic causes of global warming, and they often have more peer-reviewed publications than their contrarian colleagues.

Figure: Level of consensus on human-induced climate change versus expertise in climate science. Black circles are data based on studies of the past 10 years. Green line is a fit through the data.

The initial reason for this review article was a specific comment by Richard Tol on John Cook’s literature survey as published in ERL in 2013. Cook found a 97% consensus on anthropogenic global warming in the scientific literature on climate change. This article has both been vilified and praised. Tol argued that Cook’s study is an outlier, but he did so by misrepresenting most other consensus studies, including the survey I undertook while at the Netherlands Environmental Assessment Agency (PBL). To get a gist of the discussion with Tol see e.g. this storify I made based on my twitter exchanges with him (warning: for climate nerds only). Suffice to say the authors of these other consensus studies were likewise not impressed by Tol’s caricature of their work. This is how the broad author team for the current meta-analysis arose, which shows that Cook’s literature survey fits well within the spectrum of other studies.

The video below provides a great overview of the context and conclusions of this study:

Surveys show that among the broad group of scientists who work on the topic of climate change the level of consensus is roughly between 83 and 97% (e.g. Doran, Anderegg, Verheggen, Rosenberg, Carlton, Bray, Stenhouse, Pew, Lichter, Vision Prize). If you zoom in on the subset of most actively publishing climate scientists you find a consensus of 97% (Doran, Anderegg). Analyses of the literature also indicate a level of consensus of 97% (Cook) or even 100% (Oreskes). The strength of literature surveys lies in the fact that they sample the prime locus of scientific evidence and thus they provide the most direct measure of the consilience of evidence. On the other hand, opinion surveys can achieve much more specificity about what exactly is agreed upon. The latter aspect – what exactly is agreed upon and how does that compare to the IPCC report- is something we investigated in detail in our ES&T article based on the PBL survey.

As evidenced by the many –unfounded- criticisms on consensus studies, this is still a hot topic in the public debate, despite the fact that study after study has confirmed that there is broad agreement among scientists about the big picture: our planet is getting warmer and that is (largely) due to human activity, primarily the burning of fossil fuels. A substantial fraction of the general public however is still confused even about the big picture. In politics, schools and media climate change is often not communicated in accordance with the current scientific understanding, even though the situation here in the Netherlands is not as extreme as e.g. in the US.

Whereas the presence of widespread agreement is obviously not proof of a theory being correct, it can’t be dismissed as irrelevant either: As the evidence accumulates and keeps pointing in the same general direction, the experts’ opinion will logically converge to reflect that, i.e. a consensus emerges. Typically, a theory either rises to the level of consensus or it is abandoned, though it may take considerable time for the scientific community to accept a theory, and even longer for the public at large.

Although science can never provide absolute certainty, it is the best method we have to understand complex systems and risks, such as climate change. If you value science it is wise not to brush aside broadly accepted scientific insights too easily, lest you have very good arguments for doing so (“extraordinary claims require extraordinary evidence”). I think it is important for proper democratic decision making that the public is well informed about what is scientifically known about important issues such as climate change.

More info/context/reflections:

Dutch version at sister-blog “klimaatverandering”

Column by first author John Cook in Bulletin of the Atomic Scientists

Stephan Lewandowsky on the psychology of consensus

Collin Maessen tells the backstory starting with Richard Tol’s nonsensus

Ken Rice at …And Then There’s Physics

Dana Nuccitelli in the Guardian

Sou at HotWhopper

Amsterdam University College (AUC) news item

 

Advertisement

Richard Tol misrepresents consensus studies in order to falsely paint John Cook’s 97% as an outlier

John Cook warned me: if you attempt to quantify the level of scientific consensus on climate change, you will be fiercely criticized. Most of the counterarguments don’t stand up to scrutiny however. And so it happened.

The latest in this saga is a comment that Richard Tol submitted to ERL, as a response to John Cook’s study in which they found 97% agreement in the scientific literature that global warming is human caused. Tol tries to paint Cook’s 97% as an outlier, but in doing so misrepresents many other studies, including the survey that I undertook with colleagues in 2012. In his comment and his blogpost he shows the following graph:

Richard Tol comes to very different conclusions regarding the level of scientific consensus than the authors of the respective articles themselves (Oreskes, 2004; Anderegg et al., 2010; Doran and Kendall Zimmerman, 2009; Stenhouse et al., 2013; Verheggen et al., 2014). On the one hand, he is using what he calls “complete sample” results, which in many cases are close to meaningless as an estimate of the actual level of agreement in the relevant scientific community (that counts most strongly for Oreskes and Anderegg et al). On the other hand he is using “subsample” results, which in some cases are even more meaningless (the most egregious example of which is the subsample of outspoken contrarians in Verheggen et al).

The type of reanalysis Tol has done, if applied to e.g. evolution, would look somewhat like this:

• Of all evolutionary biology papers in the sample 75% explicitly or implicitly accept the consensus view on evolution. 25% did not take positon on whether evolution is accepted or not. None rejected evolution. Tol would conclude from this that the consensus on evolution is 75%. This number could easily be brought down to 0.5% if you sample all biology papers and count those that take an affirmative position in evolution as a fraction of the whole. This is analogous to how Tol misrepresented Oreskes (2004).

• Let’s ask biologists what they think of evolution, but to get an idea of dissenting views let’s also ask some prominent creationists, e.g. from the Discovery Institute. Never mind that half of them aren’t actually biologists. Surprise, surprise, the level of agreement with evolution in this latter group is very low (the real surprise is that it’s not zero). Now let’s pretend that this is somehow representative of the scientific consensus on evolution, alongside subsamples of actual evolutionary biologists. That would be analogous to how Tol misrepresented the “unconvinced” subsample of Verheggen et al (2014).

No @RichardTol, a survey of vocal opponents of the mainstream is not a valid estimate of scientific consensus https://t.co/G8b2oK6Ojw (1/2)

— Bart Verheggen (@BVerheggen) September 15, 2015

Especially not as that sub-sample includes non-scientists who publicly oppose mainstream science: meaningless to base estimate on. (2/2)

— Bart Verheggen (@BVerheggen) September 15, 2015

Collin Maessen provide an detailed take-down of Richard Tol on his blog, quoting extensively from the scientists whose work was misrepresented by Tol (myself included). The only surveys which are not misrepresented are those by Bray and von Storch (2007; 2010). This is how I am quoted at Collin’s blog RealSkeptic:

Tol selectively quotes results from our survey. We provided results for different subsamples, based on different questions, and based on different types of calculating the level of agreement, in the Supporting Information with our article in ES&T. Because we cast a very wide net with our survey, we argued in our paper that subgroups based on a proxy for expertise (the number of climate related peer reviewed publications) provide the best estimate of the level of scientific consensus. Tol on the other hand presents all subsamples as representative of the scientific consensus, including those respondents who were tagged as “unconvinced”. This group consists to a large extent of signatories of public statements disapproving of mainstream climate science, many of whom are not publishing scientists. For example, some Heartland Institute staffers were also included. It is actually surprising that the level of consensus in this group is larger than 0%. To claim, as Richard Tol does, that the outcome for this subsample is somehow representative of the scientific consensus is entirely nonsensical.

Another issue is that Richard Tol bases the numbers he uses on just one of the two survey questions about the causes of recent climate change, i.e. a form of cherry picking. Moreover, we quantified the consensus as a fraction of those who actually answered the question by providing an estimate of the human greenhouse gas contribution. Tol on the other hand quantifies the consensus as a fraction of all those who were asked the question, including those who didn’t provide such an estimate. We provided a detailed argument for our interpretation in both the ES&T paper and in a recent blogpost.

Tol’s line of reasoning here is similar to his misrepresentation of Oreskes’ results, by taking the number of acceptance papers not just as a fraction of papers that take position, but rather as a fraction of all papers, including those that take no position on current anthropogenic climate change. Obviously, the latter should be excluded from the ratio, unless one is interested in producing an artificially low, but meaningless number.

Some quotes from the other scientists:

Oreskes:

Obviously he is taking the 75% number below and misusing it. The point, which the original article made clear, is that we found no scientific dissent in the published literature.

Anderegg:

This is by no means a correct or valid interpretation of our results.

Neil Stenhouse:

Tol’s description omits information in a way that seems designed to suggest—inaccurately—that the consensus among relevant experts is low.

Doran:

To pull out a few of the less expert groups and give them the same weight as our most expert group is a completely irresponsible use of our data.

You can read their complete quotes at RealSkeptic.

See also this storify of my twitter discussion with Richard Tol.

Rick Santorum misrepresents our climate survey results on Bill Maher show

In our survey of more than 1800 scientists we found that the large majority agree that recent climate change is predominantly human induced. The article where we discuss our results is publicly available and a brief rundown of the main conclusions is provided in this blogpost.

We were quite surprised to hear a US Presidential candidate, Rick Santorum, make the opposite claim based on our survey. On the Bill Maher show he said:

The most recent survey of climate scientists said about 57 percent don’t agree with the idea that 95 percent of the change in the climate is caused by CO2. (…) There was a survey done of 1,800 scientists, and 57 percent said they don’t buy off on the idea that CO2 is the knob that’s turning the climate. There’s hundreds of reasons the climate’s changed.

What did we actually find in our survey?

In our survey of 1868 scientists studying various aspects of climate change, we asked two questions about the causes of recent global warming. Of all scientists who provided an estimate ~85% think that the influence of human greenhouse gases is dominant, i.e. responsible for more than half of the observed warming. ~15% think greenhouse gases are responsible for less than half of the observed warming. If you zoom in to those respondents with arguably more expertise, the percentage agreeing with human dominated warming becomes 90% or larger.

The existence of a strong scientific consensus about climate change is also clear from previous surveys of scientists and of the scientific literature and from statements of scientific societies. A scientific consensus is a logical consequence of the evidence for a certain position becoming stronger over time.

Rick Santorum’s claim is misleading and wrong because:

1) it is based on a wrong interpretation of just one of the two survey questions about the causes of recent climate change.

2) it is based on the argument that respondents who didn’t provide a specific estimate for the contribution of greenhouse gases (22% of the total number) think that this contribution is small. That is a wrong inference.

3) it is based on the argument that respondents who think it is “very likely” or “likely” or “more likely than not” that greenhouse gases are the dominant cause of recent warming disagree with this dominant influence. That is a wrong inference.

Politifact also did a fact-check of Santorum’s claim and found it to be false. It gives a very good overview of what’s wrong with Santorum’s claim. Several people are quoted in their article, myself included as well as the blogger Fabius Maximus who came up with the 43% consensus figure used by Santorum to claim a 57% dissensus:

PBL survey shows strong scientific consensus that global warming is largely driven by greenhouse gases

Updates:

(5 Sep 2015): US Presidential candidate Rick Santorum used an erroneous interpretation of our survey results on the Bill Maher show. My  detailed response to Santorum’s claim is in a newer blogpost. Politifact and Factcheck also chimed in and found Santorum’s claims to be false. The blogpost below goes into detail about how different interpretations could lead to different conclusions and how some interpretations are better supported than others.

As Michael Tobis rightly points out, the level of scientific consensus that you find “depends crucially on who you include as a scientist, what question you are asking, and how you go about asking it”. And on how you interpret the data. We argued that our survey results show a strong scientific consensus that global warming is predominantly caused by anthropogenic greenhouse gases. Others beg to differ. Recent differences of opinion are rooted in different interpretations of the data. Our interpretation is based on how we went about asking certain questions and what the responses indicate.

To quantify the level of agreement with a certain position, it makes most sense to look at the number of people as a fraction of those who answered the question. We asked respondents two questions about attribution of global warming (Q1 asking for a quantitative estimate and Q3 asking for a qualitative estimate; the complete set of survey questions is available here). However, as we wrote in the ES&T paper:

Undetermined responses (unknown, I do not know, other) were much more prevalent for Q1 (22%) than for Q3 (4%); presumably because the quantitative question (Q1) was considered more difficult to answer. This explanation was confirmed by the open comments under Q1 given by those with an undetermined answer: 100 out of 129 comments (78%) mentioned that this was a difficult question.

There are two ways of expressing the level of consensus, based on these data: as a fraction of the total number of respondents (including undetermined responses), or as a fraction of the number of respondents who gave a quantitative or qualitative judgment (excluding undetermined answers). The former estimate cannot exceed 78% based on Q1, since 22% of respondents gave an undetermined answer. A ratio expressed this way gives the appearance of a lower level of agreement. However, this is a consequence of the question being difficult to answer, due to the level of precision in the answer options, rather than it being a sign of less agreement.

Moreover, the results in terms of level of agreement based on Q1 and Q3 are mutually consistent with each other if the undetermined responses are omitted in calculating the ratio; they differ markedly when undetermined responses are included. In the supporting information we provided a table (reproduced below) with results for the level of agreement calculated either as a fraction of the total (i.e., including the undetermined answers) or as a fraction of those who expressed an opinion (i.e., excluding the undetermined answers), specified for different subgroups.

For the reasons outlined above we consider the results excluding the undetermined responses the most meaningful estimate of the actual level of agreement among our respondents. Indeed, in our abstract we wrote:

90% of respondents with more than 10 climate-related peer-reviewed publications (about half of all respondents), explicitly agreed with anthropogenic greenhouse gases (GHGs) being the dominant driver of recent global warming.

This is the average of the two subgroups with the highest number of self-reported publications for both Q1 and Q3. In our paper we discussed both ways of quantifying the level of consensus, including the 66% number as advocated by Tom Fuller (despite his claims that we didn’t).

Fabius Maximus goes further down still, claiming that the level of agreement with IPCC AR5 based on our survey results is only 43-47%. This result is based on the number of respondents who answered Q1b, asking for the confidence level associated with warming being predominantly greenhouse gas-driven, as a fraction of the total number of respondents who filled out Q1a (whether with a quantitative or an undetermined answer). As Tom Curtis notes, Fab Max erroneously compared our statement to the “extremely likely” statement in AR5, whereas in terms of greenhouse gases AR5 in Chapter 10 considered it “very likely” that they are responsible for more than half the warming. Moreover, our survey was undertaken in 2012, long before AR5 was available, so if respondents had IPCC in mind as a reference, it would have been AR4. If anything, the survey respondents were by and large more confident than IPCC that warming had been predominantly greenhouse gas driven, with over half assigning a higher likelihood than IPCC did in both AR4 and AR5.

Let me expand on the point of including or excluding the undetermined answers with a thought experiment. Imagine that we had asked whether respondents agreed with the AR4 statement on attribution, yes or no. I am confident that the resulting fraction of yes-responses would (far) exceed 66%. We chose instead to ask a more detailed question, and add other answer options for those who felt unwilling or unable to provide a quantitative answer. On the other hand, imagine if we had respondents choose whether the greenhouse gas contribution was -200, -199, …-2, -1, 0, 1, 2, … 99, 100, 101, …200% of the observed warming. The question would have been very difficult to answer to that level of precision. Perhaps only a handful would have ventured a guess and the vast majority would have picked one of the undetermined answer options (“I don’t know”, “unknown”, “other”). Should we in that case have concluded that the level of consensus is only a meagre few percentage points? I think not, since the result would be a direct consequence of the answer options being perceived as too difficult to meaningfully choose from.

Calculating the level of agreement in the way we suggest, i.e. excluding undetermined responses, provides a more robust measure as it’s relatively independent of the perceived difficulty of having to choose between specific answer options. And, as is omitted by the various critics, it is consistent with the responses to the qualitative attribution question, which also provides a clear indication of a strong consensus. If you were to insist on including undetermined responses in calculating the level of agreement, then it’s best to only use results from Q3. Tom Fuller’s 66% becomes 83% in that case (the level of consensus for all respondents), showing the lack of robustness in this approach when applied to Q1.

Some other issues that came up in recent discussions:

• We cast a very wide net of respondents, including scientists who study various parts of climate change including impacts and mitigation.
• We made special efforts to include people with skeptical points of view, not all of whom are publishing climate scientists. As such, we probably slightly underestimated the strength of the scientific consensus.
• Our results are in good agreement with other opinion surveys, including e.g. Doran and Kendall-Zimmermann. Literature surveys such as by Cook et al generally find higher levels of consensus, since -as we also found, see figure just above- more published scientists are generally more convinced of human causation of global warming.
• See my comment at CCNF for a short discussion on the relevance of scientific consensus and pros/cons of evaluating it be means of literature survey or opinion survey.
• The background report with all survey questions and aggregated responses is basically an accompaniment to the article in ES&T and is based on the same data.

See also the basic summary of our survey findings and the accompanying FAQ.

 

Video-interview about climate science survey paper

Fellow Dutchman Collin Maessen interviewed me via skype about our recent paper “Scientists’ views about attribution of global warming“:

Collin wrote a blogpost about it as well which is well worth a read, giving a bit of context from other opinion and literature surveys.

The interview starts off with the general findings regarding the level of consensus, then focusing on how this compares with previous studies, how the media coverage is slanted towards contrarian views, and he gives me a chance to talk about my favorite part, how aerosol cooling masks the greenhouse warming and how this makes the phrasing of the IPCC AR4 attribution statement, by focusing only on the greenhouse warming part, prone to being misinterpreted. These aspects were also discussed in my blogpost from last month.

FAQ for the article “Scientists’ Views about Attribution of Global Warming”

published in Environmental Science and Technology (open access), DOI: 10.1021/es501998e, Supporting Information here.

A formal version of the FAQ is also available at the website of the PBL Netherlands Environmental Assessment Agency. A blog post with a brief description of the main conclusions is here.

 

General

1. What are the objectives of this survey?

The PBL aimed to characterize the spectrum of scientific opinion about physical climate science issues. The research was focused on issues that are a frequent topic of public debate, and explored questions such as:

• On which issues is there widespread agreement amongst scientists?
• On which issues do scientists hold varied opinions?
• How does the spectrum of scientific opinion compare to IPCC assessments?
• How do scientists view skeptical arguments and viewpoints?

2. What is the relevance of an opinion survey or of measurement of consensus in trying to assess the science?

Science is based on the critical evaluation of available evidence in the context of existing knowledge. It is not “just an opinion.” With this survey, we tried to identify how scientists assess the different viewpoints that exist in public discussions of climate science. If the evidence for a certain viewpoint has become sufficiently strong and stable over time, the scientists’ aggregated opinion could be expected to reflect that.

3. Are the survey results publicly available?

The full survey results are not publicly available, because the PBL intends to use the data for further analyses.
Update:
Graphs with the ‘straight counts’ for every question (i.e. the number of responses for each answer option) are available in a background report. These results are not segregated in different sub-groups.

 

Conclusions

4. How does this study compare to the often-quoted 97% consensus?

Our results are consistent with similar studies, which all find high levels of consensus among scientists, especially among scientists who publish more often in the peer-reviewed climate literature.

Cook et al. (2013) found that 97% of papers that characterized the cause of recent warming indicated that it is due to human activities. (John Cook, the lead author of that analysis, is co-author on this current article.) Similarly, a randomized literature review found zero papers that called human-induced climate change into question (Oreskes, 2004).

Other studies surveyed scientists themselves. For instance, Doran and Kendall-Zimmermann (2009) found lower levels of consensus for a wider group of earth scientists (82% consensus) as compared to actively publishing climatologists (97% consensus) on the question of whether or not human activity is a “significant contributor” to climate change. Our results are also in line with those of e.g. Bray and von Storch (2008) and Lichter (2007).

In our study, among respondents with more than 10 peer-reviewed publications (half of total respondents), 90% agree that greenhouse gases are the largest – or tied for largest – contributor to recent warming. The level of agreement is ~85% for all respondents.

While these findings are consistent with other surveys, several factors could explain the slight differences we found:

• Surveys like ours focus on opinions of individual scientists, whereas in a literature analyses the statements in individual abstracts are tallied. Literature analyses have generally found higher levels of consensus than opinion surveys, since the consensus is stronger amongst more heavily published scientists.
• This study sets a more specific and arguably higher standard for what constitutes the consensus position than other studies. For instance, Doran and Kendall-Zimmermann (2009) asked about human activity being a “significant contributor” to global warming, and Anderegg et al. (2010) investigated signatories of public statements, while we asked specifically about the degree to which greenhouse gases are contributing to climate change in comparison with other potential factors.
• Contrarian viewpoints are somewhat overrepresented in our survey and they may have overestimated their self-declared level of expertise (see question 9).

5. How is the consensus or agreement position defined?

The consensus position was defined in two ways:

• Greenhouse gases contributed more than 50% to global warming since the mid-20^th (Question 1). This is analogous to what was written in IPCC AR4.
• Greenhouse gases have caused strong or moderate warming since pre-industrial times (Question 3). “Moderate” warming was only interpreted as the consensus position if no other factor was deemed to have caused “strong” warming. This response means that greenhouse gases were considered the strongest –or tied for strongest- contributor to global warming.

The former definition exactly mirrors the main attribution statement in IPCC AR4 and served as a ‘calibration’ for the latter.

6. What does “relative response” mean on the y-axis of many Figures?

This gives the percentage of the respondents (often within a certain sub-group) for the specific answer option. We opted to show the relative response rather than the absolute response to enable comparing the responses of different sub-groups (with differing group sizes as denoted by N=…) within one graph.

7. What are “undetermined” answers?

Those are the sum of responses “I don’t know”, “unknown” and “other”.

8. Why do IPCC AR4 authors show a higher consensus than the other respondents?

AR4 authors are generally domain experts, whereas the survey respondents at large comprise a very broad group of scholars, including for example scientists studying climate impacts or mitigation. Hence we consider this to be an extension of the observation -in this study and in e.g. Anderegg et al. (2010) and Doran and Kendall-Zimmermann (2009) – that the more expert scientists report stronger agreement with the IPCC position. Moreover, on the question of how likely the greenhouse contribution exceeded 50%, many respondents provided a stronger statement than was made in AR4. Using a smaller sample of scientists, Bray (2010) found no difference in level of consensus between IPCC authors and non-authors.

9. How reliable are the responses regarding the respondent’s area of expertise and number of peer-reviewed publications?

Respondents were tagged with expertise fields, though these were in many cases limited and not meant to be exhaustive. These tags were mainly used to ensure that the group of respondents was representative of the group that the survey was sent to. A subset of respondents was also tagged with a Google Scholar metric. Those who were tagged as “unconvinced” reported more expertise fields than the total group of respondents and also a higher number of publications compared to their Google Scholar metrics, if available (see Supplemental Information).

10. Since most scientists agree with the mainstream and therefore most media coverage is mainstream, what is the problem with “false balance”?

Scientists with dissenting opinions report receiving more media attention than those with mainstream opinions. This results in a skewed picture of the spectrum of scientific opinion. Whether that is problematic is in the eye of the beholder, but it may partly explain why public understanding lags behind scientific discourse (e.g. the “consensus gap”).

 

Survey Respondents

11. How many responses did you get to the survey?

Out of 6550 people contacted, 1868 filled out the survey (either in part or in full).

12. How did you compile the list of people to be surveyed?

Respondents were selected based on

• keyword search in peer-reviewed publications (“global climate change” and “global warming”)
• recent climate literature (various sources)
• highly cited climate scientists (as listed by Jim Prall)
• public criticisms of mainstream climate science (as listed by Jim Prall)

13. Are all of the survey invitees climate scientists?

The vast majority of invitees are scientists who published peer-reviewed articles about some aspect of climate change (this could be climate science, climate impacts, mitigation, etc.). Not all of them necessarily see themselves as climate scientists.

14. Why did you invite non-scientist skeptics to take part in the survey?

They were included in the survey to ensure that the main criticisms of climate science would be included. They constitute less then 5% of the survey respondents. Viewpoints that run counter to the prevailing consensus are therefore somewhat magnified in our results.

15. How representative are the survey responses of the “scientific opinion”?

It’s difficult to ascertain the extent to which our sample is representative, especially because the target group is heterogeneous and hard to define. We have chosen to survey the wider scientific field that works on climate change issues. Due to the criteria we used and the number of people invited we are confident that our results are indeed representative of this wider scientific field studying various aspects of global warming. We checked that those who responded to the survey were representative of the larger group of invitees by using various pieces of meta-information.

16. Did you take into account varying levels of expertise of respondents?

Respondent were asked to list their area(s) of expertise and their number of peer-reviewed publications. These and other attributes were used to interpret differences in responses.

17. How did you prevent respondents from manipulating the survey results, e.g. by answering multiple times?

An automatically generated, user specific token ensured that respondents could only respond once.

18. How did you ensure respondent anonymity?

Survey responses were analyzed by reference to a random identification number.

 

Survey Questions

19. Are the survey questions public?

Yes, survey questions and answer options are available on the PBL website and as Supporting Information (part 2) to the article.

20. How did you decide on the questions to ask?

The survey questions are related to physical science issues which are a frequent topic of public debate about climate change.

21. Was the survey reviewed before it was sent to respondents?

Yes, before executing the survey it has been extensively tested and commented on by various climate scientists, social scientists and science communicators with varying opinions, to ensure that questions were both clear and unbiased. Respondents were not steered to certain answers.

——-

Reference: Bart Verheggen, Bart Strengers, John Cook, Rob van Dorland, Kees Vringer, Jeroen Peters, Hans Visser, and Leo Meyer, Scientists’ Views about Attribution of Global Warming, Environmental Science and Technology, 2014. DOI: 10.1021/es501998e. Supporting Information available here.

Please keep discussions on this thread limited to what is mentioned in this FAQ and to other questions you may have about the survey or the article. Discussion of the survey results should be directed at the more generic blog post.

Survey confirms scientific consensus on human-caused global warming

• A survey among more than 1800 climate scientists confirms that there is widespread agreement that global warming is predominantly caused by human greenhouse gases.
• This consensus strengthens with increased expertise, as defined by the number of self-reported articles in the peer-reviewed literature.
• The main attribution statement in IPCC AR4 may lead to an underestimate of the greenhouse gas contribution to warming, because it implicitly includes the lesser known masking effect of cooling aerosols.
• Self-reported media exposure is higher for those who are skeptical of a significant human influence on climate.

In 2012, while temporarily based at the Netherlands Environmental Assessment Agency (PBL), my colleagues and I conducted a detailed survey about climate science. More than 1800 international scientists studying various aspects of climate change, including e.g. climate physics, climate impacts and mitigation, responded to the questionnaire. The main results of the survey have now been published in Environmental Science and Technology (doi: 10.1021/es501998e).

Level of consensus regarding attribution

The answers to the survey showed a wide variety of opinions, but it was clear that a large majority of climate scientists agree that anthropogenic greenhouse gases are the dominant cause of global warming. Consistent with other research, we found that the consensus is strongest for scientists with more relevant expertise and for scientists with more peer-reviewed publications. 90% of respondents with more than 10 climate-related peer-reviewed publications (about half of all respondents), agreed that anthropogenic greenhouse gases (GHG) are the dominant driver of recent global warming. This is based on two different questions, of which one was phrased in similar terms as the quintessential attribution statement in IPCC AR4 (stating that more than half of the observed warming since the 1950s is very likely caused by GHG).

Figure 1. The more publications the respondents report to have written, the more important they consider the contribution of greenhouse gases to global warming. Responses are shown as a percentage of the number of respondents (N) in each subgroup, segregated according to self-reported number of peer-reviewed publications.

Literature analyses (e.g. Cook et al., 2013; Oreskes et al., 2004) generally find a stronger consensus than opinion surveys such as ours. This is related to the stronger consensus among highly published – and arguably the most expert – climate scientists. The strength of literature surveys lies in the fact that they sample the prime locus of scientific evidence and thus they provide the most direct measure of the consilience of evidence. On the other hand, opinion surveys such as ours can achieve much more specificity about what exactly is agreed upon and where the disagreement lies. As such, these two methods for quantifying scientific consensus are complementary. Our questions possibly set a higher bar for what’s considered the consensus position than some other studies. Furthermore, contrarian viewpoints were likely overrepresented in our study compared with others.

No matter how you slice it, scientists overwhelmingly agree that recent global warming is to a great extent human caused.

The role of scientific consensus in moving the public debate forward

Mike Hulme had an interesting essay at The Conversation, the main message of which was

In the end, the only question that matters [for the public debate about climate change] is, what are we going to do about it?

Hulme correctly argues that the basic science is clear enough so that for society the important issues to discuss are not science related, but policy related. I argued much the same here. He writes:

What matters is not whether the climate is changing (it is); nor whether human actions are to blame (they are, at the very least partly and, quite likely, largely); nor whether future climate change brings additional risks to human or non-human interests (it does).

Let’s leave the minor quibble aside that AR5 puts the anthropogenic contribution at ‘extremely likely’ having caused more than half of the recent global warming.

The part where I disagree with Hulme is where he argues that showing the existence of a scientific consensus on the above (it is warming; it’s due to us; it’s bad news) somehow stands in the way of  getting society to discuss that most important question. I think the opposite is true. It is the continuous doubt about the science, sowed by those who oppose a serious discussion about what to do, that is a stumbleblock. Showing that a consensus amongst experts exists would enable society to more swiftly move on to the important conversation on what to do about it. I agree with Hulme that on this deeply ethical question there is, and ought to be, a multitude of opinions.

As Stephan Lewandowsky and John Cook write in a response to Mike Hulme:

The public’s perception of that scientific consensus is necessary to stimulate political debate about solutions.

Another element that’s missing from this discussion is that scientific and ideological arguments  should be clearly distinguished from each other (“is” vs “ought”).

Unfortunately, ideological arguments are often dressed in a sciency-looking cloak. From that perspective, I appreciate the honesty in Lindzen stating blunty “we’ll all be dead by then”, the obvious implication being: so why care. That’s indeed what a lot comes down to: How do you value the future compared the present?

Scott Denning’s smashing presentation at Heartland climate conference ICCC6

Listen to Scott Denning’s sharp and to-the-point presentation, which he gave at Heartland’s climate conference, here. It’s worth the full 16 minutes of it. He rocks. Alternatively, read this little recap:

Denning attended the Heartland conference for the second year in a row and it seems like he’s outdone himself by giving an even better and sharper presentation than last years (which was excellent as well).

He emphasized some very important things:

– The big picture is what matters; details do not (at least in terms of policy relevance; for science nerds of course it’s different)

– Part of that big picture is that, whatever the sensitivity, a 400% increase in CO2 is going to make a big difference to the climate, because of the simple fact that adding heat warms things up.

– He offered a big challenge to the (strongly contrarian and libertarian) audience: Propose and advocate for effective solutions, otherwise others will. Policy will be enacted anyway. His challenge got particularly strong when he said “do you want Greenpeace to dictate the policy? (…) Are you cowards?”

Eric Wolff on areas of agreement and on the public debate about climate science

Dr. Eric Wolff is spot on (see also further below): 

as an outsider to the blogosphere, it surprises me that so many people, presumably mostly with even less knowledge and training than me, seem absolutely convinced they have mastered every area of climate science.

A peculiar line-up of speakers assembled recently at the Conference on the Science and Economics of Climate Change in Cambridge: Phil Jones, Andrew Watson, John Mitchell, Michael Lockwood, Henrik Svensmark, Nils-Axel Morner, Ian Plimer, Vaclav Klaus and Nigel Lawson. Bishop Hill reports that

Dr Eric Wolff of the British Antarctic Survey tried valiantly to find a measure of agreement between the two sides.

This proved an interesting exercise and resulted in a useful list (also reproduced by Judith Curry) on what we can all agree on (perhaps excluding those too far out on the fringes; links added by me):

1. CO2 does absorb infrared radiation
2. The greenhouse effect (however badly named) does occur in practice: our planet and the others with an atmosphere are warmer than they would be because of the presence of water vapour and CO2.
3. The greenhouse effect does not saturate with increasing CO2
4. The CO2 concentration in the atmosphere has risen significantly over the last 200 years
5. This is because of anthropogenic emissions (fossil fuels, cement production, forest clearance). [Ian Plimer disagreed, but as Curry stated: “the anthropogenic contribution is (should be) undisputed”]
6. If we agree all these statements above, we must expect at least some warming. [Bishop Hill noted “broad agreement” with this point]
7. The climate has warmed over the last 50 years, [as is evident from] land atmospheric temperature, marine atmospheric temperature, sea surface temperature, and (from Prof Svensmark) ocean heat content, all with a rising trend.
8. We probably don’t agree on what has caused the warming up to now, but it seemed that Prof Lockwood and Svensmark actually agreed it was not due to solar changes, because although they disagreed on how much of the variability in the climate records is solar, they both showed solar records without a rising trend in the late 20th century. [Excellent point and ironically hitting Svensmark with a stick of his own making. Note the distinction between variability and trend.]
9. On sea level, I said that I had a problem in the context of the day, because this was the first time I had ever been in a room where someone had claimed (as Prof Morner did) that sea level has not been rising in recent decades at all.  I therefore can’t claim we agreed, only that this was a very unusual room.  However, I suggested that we can agree that, IF it warms, sea level will rise, since ice definitely melts on warming, and the density of seawater definitely drops as you warm it.
10. Finally we come to where the real uncertainties between scientists lie, about the strength of the feedbacks on warming induced by CO2 [i.e. climate sensitivity]

Eric Wolff also chimed in with a substantial comment over at Bishop Hill’s, rebutting some commonly heard arguments and making some very spot-on remarks:

(…)

I should first state the rationale for the summary I made at the Downing event. The meeting was about the science and economics of climate change, and I was asked to lead a discussion that came between the science talks and the two economics talks. I therefore felt the most useful thing I could do was to try to summarise what we had heard, as a basis for the discussion of whether society should do anything in response to that, and if so what. In particular I did hear a surprising number of things on which almost everyone in the room could agree, and it seemed worth emphasising that, rather than rehearsing old arguments.

I notice in the thread here several comments about who sets the “terms of the debate”, and about the “context of the debate”. While such phrasings may make sense in discussing energy policy, it is a strange way to discuss the science. Our context is the laws of physics and our observations of the Earth in action; our aim as scientists is to find out how the Earth works: this is not a matter of debate but of evidence. I think some of the comments on this blog come dangerously close to suggesting that we should first decide our energy policy, and then tell the Earth how to behave in response to it.

Regarding Plimer’s proposal that volcanic emissions were more important than we thought, (…) if volcanoes were “causing” the recent increase, then around 1800 their emissions would have had to rise above their stable long-term rate, and then stayed high. This is however a rather hypothetical discussion because the change in the isotopic composition of CO2 in the atmosphere over the industrial period is not consistent with an increased volcanic source anyway.

Regarding the idea that the “temperature increase stopped in 2000”: my point is that we know there are natural variations (due eg to El Nino) that cause runs of a few years of temperature colder than the average or a few years warmer. Look on the record at the decade around 1910 for example when there was a long run of cold years on a flat background.

Such a period superimposed on a trend would look like the last decade (but more so). The point of my analogy is that you can’t determine the trend over several months by measuring the gradient in a run of a few days. Similarly, you simply can’t determine a multidecadal trend by measuring the gradient over a few years because all you get is the “noise” of natural variability.

Professor Morner claims that globally sea level has not risen at all; he dismisses the evidence, from both satellites and the global tide gauge network, that it has. (…) There are numerous reasons why a single site can show a sea level signal, either real or apparent, that departs from the global mean.

I am not the best person to discuss models and feedbacks in detail (see comment on expertise below). However, I could not let two issues pass. Firstly, when models are run out for a century into the future, they do indeed show runs of years with flat temperatures amidst a trend (…) (because of El Nino and other natural factors). I am therefore not clear why this is evidence that something is missing. Regarding positive feedbacks: a positive feedback implies amplification, but not a system out of control; this is only the case if the sum of the gain factors is greater than 1.

Finally, a few specific issues that interested or worried me.

(…) CO2 emits infrared as well as absorbing it. (…) indeed, this property is precisely why its effects do not saturate (but fall logarithmically), because it allows the height from which the emitted radiation finally escapes to rise into regions with less and less air.

Geckko accidentally made an important point. S/he did not like the statement: “We can agree that if it warms the sea level will rise”, because it was too simplistic. Well, as a scientist I always like to boil things down to a statement that my brain can grasp, but that contains the essential explanation of an observation or process. And this one does, for example being demonstrably what was observed in going from a cold ice age world, with sea level 120 metres below the present level, to the present. However, you are right: there are factors that could make this statement false, such as increased snowfall when it warms, adding more ice into ice sheets. As soon as several such competing processes have to be taken into account, our brains cannot predict the outcome, and so we have to resort to putting all the “millions of assumptions” into a numerical model and seeing which of them “win”. An argument for models?

Coldish made a good point about expertise, and this is where I am going to go into a slightly more challenging area. I freely admit that I am not an expert on all, or even most, aspects of climate. When I reach a topic that I have not previously studied, I go to those who are experts, either in person or by reading their work. I maintain scepticism about some of their conclusions, but my working assumption is that they are intelligent and that they have probably thought of most of the issues that I will come up with. Can I observe as an outsider to the blogosphere, that it surprises me that so many people, presumably mostly with even less knowledge and training than me, seem absolutely convinced they have mastered every area of climate science.

and more so, convinced that they are right and almost all of the experts are wrong. That must be the height of hubris.

However, Coldish specifically mentioned IPCC, and I think there is also an interesting point to make about that. At the Downing event, there seemed to be two IPCCs in the room. To some it was a huge plot, masterminded by some mysterious power that manipulates troublesome scientists. To me and the scientists in the room, it is (at least in WG1) simply a set of well-researched review papers, describing the present state of the peer-reviewed literature. I mention this only because I think the former view is a type of groupthink where, because people form an extreme opinion in their private space, they think it is widely held, or even true.

Alas, Wolff’s forray into the blogosphere was short, as is evident from a short comment a while later:

Just in case anyone thinks they are addressing me with their remarks:

I thought this might indeed be a chance for a civilised discussion, and some of the respondents seem happy to have that. However there are also a lot of remarks on here that are frankly rude and aggressive, and I won’t be returning. Now I remember why I hate the blogosphere.