Archive for the ‘Survey’ Category

Rick Santorum misrepresents our climate survey results on Bill Maher show

September 2, 2015

In our survey of more than 1800 scientists we found that the large majority agree that recent climate change is predominantly human induced. The article where we discuss our results is publicly available and a brief rundown of the main conclusions is provided in this blogpost.

We were quite surprised to hear a US Presidential candidate, Rick Santorum, make the opposite claim based on our survey. On the Bill Maher show he said:

The most recent survey of climate scientists said about 57 percent don’t agree with the idea that 95 percent of the change in the climate is caused by CO2. (…) There was a survey done of 1,800 scientists, and 57 percent said they don’t buy off on the idea that CO2 is the knob that’s turning the climate. There’s hundreds of reasons the climate’s changed.

What did we actually find in our survey?

In our survey of 1868 scientists studying various aspects of climate change, we asked two questions about the causes of recent global warming. Of all scientists who provided an estimate ~85% think that the influence of human greenhouse gases is dominant, i.e. responsible for more than half of the observed warming. ~15% think greenhouse gases are responsible for less than half of the observed warming. If you zoom in to those respondents with arguably more expertise, the percentage agreeing with human dominated warming becomes 90% or larger.

The existence of a strong scientific consensus about climate change is also clear from previous surveys of scientists and of the scientific literature and from statements of scientific societies. A scientific consensus is a logical consequence of the evidence for a certain position becoming stronger over time.

Rick Santorum’s claim is misleading and wrong because:

1) it is based on a wrong interpretation of just one of the two survey questions about the causes of recent climate change.

2) it is based on the argument that respondents who didn’t provide a specific estimate for the contribution of greenhouse gases (22% of the total number) think that this contribution is small. That is a wrong inference.

3) it is based on the argument that respondents who think it is “very likely” or “likely” or “more likely than not” that greenhouse gases are the dominant cause of recent warming disagree with this dominant influence. That is a wrong inference.

Politifact also did a fact-check of Santorum’s claim and found it to be false. It gives a very good overview of what’s wrong with Santorum’s claim. Several people are quoted in their article, myself included as well as the blogger Fabius Maximus who came up with the 43% consensus figure used by Santorum to claim a 57% dissensus:


PBL survey shows strong scientific consensus that global warming is largely driven by greenhouse gases

August 4, 2015


(5 Sep 2015): US Presidential candidate Rick Santorum used an erroneous interpretation of our survey results on the Bill Maher show. My  detailed response to Santorum’s claim is in a newer blogpost. Politifact and Factcheck also chimed in and found Santorum’s claims to be false. The blogpost below goes into detail about how different interpretations could lead to different conclusions and how some interpretations are better supported than others.

As Michael Tobis rightly points out, the level of scientific consensus that you find “depends crucially on who you include as a scientist, what question you are asking, and how you go about asking it”. And on how you interpret the data. We argued that our survey results show a strong scientific consensus that global warming is predominantly caused by anthropogenic greenhouse gases. Others beg to differ. Recent differences of opinion are rooted in different interpretations of the data. Our interpretation is based on how we went about asking certain questions and what the responses indicate.

To quantify the level of agreement with a certain position, it makes most sense to look at the number of people as a fraction of those who answered the question. We asked respondents two questions about attribution of global warming (Q1 asking for a quantitative estimate and Q3 asking for a qualitative estimate; the complete set of survey questions is available here). However, as we wrote in the ES&T paper:

Undetermined responses (unknown, I do not know, other) were much more prevalent for Q1 (22%) than for Q3 (4%); presumably because the quantitative question (Q1) was considered more difficult to answer. This explanation was confirmed by the open comments under Q1 given by those with an undetermined answer: 100 out of 129 comments (78%) mentioned that this was a difficult question.

There are two ways of expressing the level of consensus, based on these data: as a fraction of the total number of respondents (including undetermined responses), or as a fraction of the number of respondents who gave a quantitative or qualitative judgment (excluding undetermined answers). The former estimate cannot exceed 78% based on Q1, since 22% of respondents gave an undetermined answer. A ratio expressed this way gives the appearance of a lower level of agreement. However, this is a consequence of the question being difficult to answer, due to the level of precision in the answer options, rather than it being a sign of less agreement.

Moreover, the results in terms of level of agreement based on Q1 and Q3 are mutually consistent with each other if the undetermined responses are omitted in calculating the ratio; they differ markedly when undetermined responses are included. In the supporting information we provided a table (reproduced below) with results for the level of agreement calculated either as a fraction of the total (i.e., including the undetermined answers) or as a fraction of those who expressed an opinion (i.e., excluding the undetermined answers), specified for different subgroups.

Verheggen et al - EST 2014 - Table S3

For the reasons outlined above we consider the results excluding the undetermined responses the most meaningful estimate of the actual level of agreement among our respondents. Indeed, in our abstract we wrote:

90% of respondents with more than 10 climate-related peer-reviewed publications (about half of all respondents), explicitly agreed with anthropogenic greenhouse gases (GHGs) being the dominant driver of recent global warming.

This is the average of the two subgroups with the highest number of self-reported publications for both Q1 and Q3. In our paper we discussed both ways of quantifying the level of consensus, including the 66% number as advocated by Tom Fuller (despite his claims that we didn’t).

Fabius Maximus goes further down still, claiming that the level of agreement with IPCC AR5 based on our survey results is only 43-47%. This result is based on the number of respondents who answered Q1b, asking for the confidence level associated with warming being predominantly greenhouse gas-driven, as a fraction of the total number of respondents who filled out Q1a (whether with a quantitative or an undetermined answer). As Tom Curtis notes, Fab Max erroneously compared our statement to the “extremely likely” statement in AR5, whereas in terms of greenhouse gases AR5 in Chapter 10 considered it “very likely” that they are responsible for more than half the warming. Moreover, our survey was undertaken in 2012, long before AR5 was available, so if respondents had IPCC in mind as a reference, it would have been AR4. If anything, the survey respondents were by and large more confident than IPCC that warming had been predominantly greenhouse gas driven, with over half assigning a higher likelihood than IPCC did in both AR4 and AR5.

PBL background report - Q1b

Let me expand on the point of including or excluding the undetermined answers with a thought experiment. Imagine that we had asked whether respondents agreed with the AR4 statement on attribution, yes or no. I am confident that the resulting fraction of yes-responses would (far) exceed 66%. We chose instead to ask a more detailed question, and add other answer options for those who felt unwilling or unable to provide a quantitative answer. On the other hand, imagine if we had respondents choose whether the greenhouse gas contribution was -200, -199, …-2, -1, 0, 1, 2, … 99, 100, 101, …200% of the observed warming. The question would have been very difficult to answer to that level of precision. Perhaps only a handful would have ventured a guess and the vast majority would have picked one of the undetermined answer options (“I don’t know”, “unknown”, “other”). Should we in that case have concluded that the level of consensus is only a meagre few percentage points? I think not, since the result would be a direct consequence of the answer options being perceived as too difficult to meaningfully choose from.

Calculating the level of agreement in the way we suggest, i.e. excluding undetermined responses, provides a more robust measure as it’s relatively independent of the perceived difficulty of having to choose between specific answer options. And, as is omitted by the various critics, it is consistent with the responses to the qualitative attribution question, which also provides a clear indication of a strong consensus. If you were to insist on including undetermined responses in calculating the level of agreement, then it’s best to only use results from Q3. Tom Fuller’s 66% becomes 83% in that case (the level of consensus for all respondents), showing the lack of robustness in this approach when applied to Q1.

Verheggen et al - Figure 1 - GHG contribution to global warming

Some other issues that came up in recent discussions:

See also the basic summary of our survey findings and the accompanying FAQ.


Responses to the Climate Science Survey

April 12, 2015

Appeared in similar form on the PBL website

In the Spring of 2012, the Netherlands Environmental Assessment Agency PBL held a survey among 1868 scientists studying various aspects of climate change, including physical climate, climate impacts, and mitigation. The main results of the survey were published in an article in Environmental Science and Technology (ES&T) in August 2014: “Scientists’ views about attribution of global warming”. It showed that there is widespread agreement regarding a dominant influence of anthropogenic greenhouse gases on recent global warming. This agreement is stronger among respondents with more peer-reviewed publications.

A background report with the results for all 31 questions has now been made available. The total number of responses for each answer option is provided and a subdivision into seven groups for five questions. The background report contains previously unpublished data. Some highlights are provided below.

Climate sensitivity

Respondents were asked for their opinion regarding the best estimate and likely range for equilibrium climate sensitivity (ECS). This is an important quantity for projections of global warming, as it gives the expected warming that would follow from a doubling in atmospheric CO2 concentration after the climate system has equilibrated to the new conditions. Thus, expected warming in the future depends on the combination of total emissions and climate sensitivity.

The figure below gives the average estimates of ECS from 7 groups of respondents, including authors of the Working Group I report of the fourth IPCC Assesment Report (AR4), respondents who signed public declarations critical of mainstream climate science as embodied by IPCC (‘unconvinced’), and four different subgroups distinguished according to their self-declared number of climate related peer-reviewed publications (0–3; 4–10; 11–30; more than 30). Results from most groups were very close to the IPCC range (1.5-4.5 °C) mentioned in the fifth assessment report (AR5) – except those tagged as ‘unconvinced’ which strongly deviated from the other groups, and to a lesser extent the group of respondents with three or less publications. For all subgroups the ‘best estimate’ was slightly lower than the ‘best estimate’ reported in AR4 (i.e. 3 °C). AR5 provided no best estimate.

Scientists views on climate sensitivity - PBL

Role of climate science in society

Respondents were also asked their opinion about seven statements regarding the role of climate science in society and how the science should be communicated. There was a strong consensus that scientists themselves should communicate with both policymakers and the general public about climate change and that communication with the general public should focus on solid knowledge. To a lesser extent there was agreement that risks and uncertainties should be emphasised during such communication. Responses varied more strongly about whether or not existing uncertainties in climate science strengthen the case for mitigation (i.e. to avoid potential low probability, high impact events). There was (strong) disagreement with the statement that climate science would be too uncertain to be useful for policymaking on climate change.

Scientists views on role of science in society - PBL

The role of the sun in global warming

In the public debate about climate change the role of the sun is often put forward as an alternative explanation for global warming. Question 17 asked what fraction of recent global warming could be attributed to the sun. Those tagged as ‘unconvinced’ had the lowest fraction of respondents that indicated that they don’t know (together with AR4 authors) and the highest fraction that said that the role of the sun is unknown. As expected they also had by far the highest fraction (27%) that believed that the sun caused more than half of recent global warming.

As with the attribution questions (see the ES&T article), there appears to be a trend in responses going from the group with fewest publications to those with most. The more publications about climate change respondents report to have written, the larger fraction of them agree with the IPCC position that the sun hardly played a role in recent global warming, since the solar output decreased slightly over that period.

Scientists views on the role of the sun in global warming - PBL

More information:

PS: I’ll have a poster presentation about the survey at the EGU conference this week, in session EOS6 “Communication and Education in Geoscience” on Thursday evening.

Climate Science Survey – the questions

October 8, 2012

In the spring of 2012, a large scale climate science survey was held amongst 6500 scientists studying various aspects of global warming. The survey was spearheaded by the Netherlands Environmental Assessment Agency (PBL), where I was responsible for the execution and analysis during the first half of 2012.

The objective of this study is to gain insight into how climate scientists perceive the public debate on the physical scientific aspects of climate change. More info about the survey was posted on the PBL website at the time, which has recently been updated to include a link to the survey questionnaire. Please note that the survey is no longer active.

Some confusion has arisen over the status of this survey. I responded at WUWT in an attempt to clarify:

We undertook a survey in March/April of this year (which, as Hans Labohm mentioned in a comment on WUWT, had been previewed by a variety of people with different viewpoints). Some respondents, e.g. Timothy Ball, asked to see the questions again. After internal consultation, we decided to publish the survey questions on the institute’s website, so that they are viewable to all. We contacted the survey respondents to inform them of the questions being available to view. I informed Dr Ball of this as well, to follow-up on my earlier email to him.

Our email to all respondents, informing them of the fact that the survey questions are available on the web, was apparently misunderstood to mean that we were again soliciting responses to a survey; this is however not the case. Roger Pielke Sr had already put a notice about the survey on his blog, which he has since updated after an email clarifying that this is an inactive survey, to which he had previously responded.

Below we (Bart Verheggen and Bart Strengers) reply to some of the more substantive questions regarding the survey questions raised on WUWT. However, we will not discuss results or the survey sample at this point in time. We will do so when our manuscript has been accepted.



Get every new post delivered to your Inbox.

Join 148 other followers