In our survey of more than 1800 scientists we found that the large majority agree that recent climate change is predominantly human induced. The article where we discuss our results is publicly available and a brief rundown of the main conclusions is provided in this blogpost.
We were quite surprised to hear a US Presidential candidate, Rick Santorum, make the opposite claim based on our survey. On the Bill Maher show he said:
The most recent survey of climate scientists said about 57 percent don’t agree with the idea that 95 percent of the change in the climate is caused by CO2. (…) There was a survey done of 1,800 scientists, and 57 percent said they don’t buy off on the idea that CO2 is the knob that’s turning the climate. There’s hundreds of reasons the climate’s changed.
What did we actually find in our survey?
In our survey of 1868 scientists studying various aspects of climate change, we asked two questions about the causes of recent global warming. Of all scientists who provided an estimate ~85% think that the influence of human greenhouse gases is dominant, i.e. responsible for more than half of the observed warming. ~15% think greenhouse gases are responsible for less than half of the observed warming. If you zoom in to those respondents with arguably more expertise, the percentage agreeing with human dominated warming becomes 90% or larger.
The existence of a strong scientific consensus about climate change is also clear from previous surveys of scientists and of the scientific literature and from statements of scientific societies. A scientific consensus is a logical consequence of the evidence for a certain position becoming stronger over time.
Rick Santorum’s claim is misleading and wrong because:
1) it is based on a wrong interpretation of just one of the two survey questions about the causes of recent climate change.
2) it is based on the argument that respondents who didn’t provide a specific estimate for the contribution of greenhouse gases (22% of the total number) think that this contribution is small. That is a wrong inference.
3) it is based on the argument that respondents who think it is “very likely” or “likely” or “more likely than not” that greenhouse gases are the dominant cause of recent warming disagree with this dominant influence. That is a wrong inference.
Politifact also did a fact-check of Santorum’s claim and found it to be false. It gives a very good overview of what’s wrong with Santorum’s claim. Several people are quoted in their article, myself included as well as the blogger Fabius Maximus who came up with the 43% consensus figure used by Santorum to claim a 57% dissensus:
In short, Santorum’s claim commits “two orders of mischaracterization,” said Anthony Leiserowitz, who studies climate change public opinion at Yale University. He not only uses a flawed statistic, but also misstates what it’s allegedly disapproving. (…) The IPCC has said it is “extremely likely” (meaning a 95 percent confidence level) that humans are causing climate change. The IPCC also said it is “very likely” (meaning a 90 percent confidence level) that greenhouses gases are the driver.
Santorum restated IPCC points wrong. They never said 95 percent of the change in the climate is caused by CO2.
(…)
According to Fabius Maximus editor Larry Kummer, the survey’s findings disprove the IPCC’s confidence level finding. He walked us through how he got the 57 percent:
- About 65.9 percent of scientists said they agreed that greenhouse gases were the main driver of climate change.
- Of those scientists, 65.2 percent reported “virtually certain” or it was “extremely likely” that their estimate was correct (corresponding to a 95 percent or higher confidence level).
- That means, according to Kummer, about 43 percent (65.2 percent of 65.9 percent) were extremely confident that greenhouse gases were the main driver of climate change. In other words, 57 percent weren’t 95 percent confident.
(…)
“This is like something out of that book, How to Lie With Statistics,” said Stephen Farnsworth, who studies climate change and political communication at the University of Mary Washington. “What we’re talking about here is extraordinary cherry-picking. You’re only counting one question in one survey, and you’re talking about a very high (confidence level). Once you start stacking up numbers like this, you’re really distorting the real finding of this research.”
(…)
“You don’t get anywhere near 57 percent when surveying the broad earth science community, and you get very close to full consensus when you ask the experts in climate science,” said Peter Doran, a professor of earth science at Louisiana State University.
(…)
And multiple independent studies have “asked scientists directly” and found consensus levels of around 97 percent, said William Anderegg, who studies climate change at Princeton University. “Those studies were rigorously peer-reviewed and thus should be considered more credible than a blog post that misreads an institute report,” he said.
(…)
Essentially, Santorum garbles a blog’s textbook misinterpretation of a survey. In reality, the survey actually supports the idea of scientific consensus on climate change, the lead author told us.
We rate Santorum’s claim False.
Earlier, the scientist-blogger And Then There’s Physics also criticized the interpretation of Fabius Maximus (maybe I should come up with a nifty pseudonym for myself as well?):
Essentially, Fabius is arguing that only 43% of those in the survey agreed that more than 50% of the warming is anthropogenic AND that this was extremely likely (> 95%). One problem is that the question that his claim was based on had a very high fraction of respondents (22%) who didn’t actually respond with what they thought the human GHG contribution was. Verheggen et al discuss this at length in their paper, and in the blogpost, and conclude that including them skews the result to – probably – a lower value than is reasonable. This conclusion is based on what respondents mentioned themselves, and on comparing results to another survey question about the causes of global warming. This is in addition to Verheggen et al. going out of their way to include contrarians, which also probably produces a slight under-estimate of the actual level of consensus.
More fundamentally, though, Fabius’ analysis illustrates a lack of understanding of attribution studies. An attribution study is really a null hypothesis test. In this case, the null hypothesis is that more than 50% of the warming could be non-anthropogenic. This is rejected at the 95% confidence level, resulting in the conclusion that it is extremely likely that more than 50% of the warming is anthropogenic. Asking individual scientists what confidence they have is not the same as a formal attribution study. Also, the consensus is simply that more than 50% of the warming is anthropogenic. The extremely likely is the confidence we have in this consensus position. What’s relevant is the level of agreement with this consensus position, not how confident individual scientists are in this position. Or, more correctly, arguing that only 43% of scientists personally think that it is extremely likely that more than 50% of the warming is anthropogenic, is not the same as only 43% agreeing with the consensus that more than 50% of the warming is anthropogenic. It is clear from the Verheggen et al. study that a large majority of those surveyed agree that more than 50% of the warming is anthropogenic.
Indeed. And they assign various levels of confidence to the warming being predominantly caused by anthropogenic greenhouse gases. Most of the respondents were more confident of this being the case than IPCC concluded on the basis of formal attribution studies; some were less confident than IPCC. Not only did Fabius use the wrong confidence level as a reference (“extremely likely” instead of “very likely”); he’s also raising the bar which scientists have to reach in order to be counted as agreeing with human caused warming (i.e. a form of ‘impossible expectations’ or ‘moving the goalposts’). And he’s basing his claim on only one question, whereas another question that also asks about causes of climate change would lead to very different results, following his interpretation (i.e. a form of ’cherry picking’).
That last point, two similar questions leasing to very different results, signals a potential problem. At the very least it should make one wonder what the causes of that difference might be. ATTP alludes to this issue in the quotation above and I discuss this issue in detail in a previous blogpost as well as in the ES&T article. The first of these two questions (the one used by Fabius Maximus) had a very high fraction of respondents (22%) who didn’t actually respond with what they thought the human greenhouse gas contribution was, but who responded “I don’t know”, “unknown”, or “other”. Note that we didn’t ask whether the respondent agrees with the IPCC statement on attribution; rather we asked them to choose between 6 different percentage ranges of the greenhouse gas (GHG) contribution. Three of these corresponded with GHG having caused more than half of the observed warming (51-75%; 76-100%; >100%), and three with less than half (26-50%; 0-25%; <0%). All questions had space for respondents to leave comments. With this question, many respondents commented that it was difficult to pick such a specific percentage range. Hence, the 22% of respondents who didn’t pick a specific percentage range for the GHG contribution can’t be assumed to all be completely agnostic regarding their agreement or not with a dominant human influence on climate. And even less so can they be assumed to disagree with such. We argue that we don’t really know how many of this 22% agree, disagree, or truly have no opinion about it (i.e. agnostic). If we leave them out of the equation, and thus calculate the level of consensus based only on those respondents who actually provided an estimate of the GHG contribution, the results for both questions agree very well with each other. Another indication that our interpretation is likely correct.
Inadvertently, Judith Curry confirmed our interpretation with a comment at her blog:
curryja | August 27, 2015 at 2:44 pm | Reply
I have a problem with the categories, since for attribution I am around 50%, +/- about 25%. So i don’t fit comfortable in the categories as they are articulated in the survey. In terms of confidence, i would go with ‘very likely’ for my range, between 25% and 75%.
So I responded:
Bart Verheggen | August 30, 2015 at 6:34 am |
And that’s probably how many more respondents felt, that they weren’t comfortable with the categories as specified. So they would have to make a choice between a category that came somewhat close to their personal estimate, or other, dunno, or unknown. Hence the large fraction of responses in those latter three categories.
This is not just mindreading, but based on comments from respondents and based on comparing the results of this question to those from Q3.
And that is why we thought it better to leave those latter three categories out, because its number is in all likelihood much higher than the number of respondents who are truly agnostic on the question of human GHG dominated warming.
Judith again:
However, if you agreed with the IPCC consensus, should be no problem clicking the box 76-100%.
Me:
Not necessarily. If the question had just been whether the GHG contribution is smaller or larger than 50% you’d have a point, but we had 6 ranges in total. If a respondent felt that it’s probably more than half but didn’t have a narrow range in mind they’d have a very similar dilemma in choosing a particular range. Many of them did choose a range (mostly in the middle of the three categories, as we also discuss in the paper), but many others didn’t want to pin themselves down with a range that was too narrow to their mind and chose one of the undetermined options (don’t know, unknown, other). I’ve explained why we are confident that this explanation holds for a large number of the undetermined responses: The open comments and the comparison with the other attribution question (Q3) – studiously ignored by everyone including you.Nobody of my critics has directly addressed this reasoning. Neither has anybody offered an alternative reason why there was such a high fraction of undetermined responses for this question (and not for the others). Neither has anyone answered my question regarding a hypothetical alternative set of possible responses and what their consensus estimate would be in that case.
I rest my case.
Relevant background sources:
Published paper in Environmental Science and Technology about the main survey results.
Background report with all survey questions and aggregated responses.
FAQ with the study.
My Blogpost outlining the main results of the study.
My Blogpost focusing on how responses to the two questions about causes of climate change relate to each other.
Other rebuttals:
Politifact factcheck, rating Santorum’s claim false
FactCheck, rating Santorum’s claim false
Rebuttal by scientist-blogger ATTP
Interviews with me and with John Cook regarding Santorum’s erroneous remarks.
John Cook, about how Santorum uses our research to cast doubt on his other research
Tags: 1800, 43%, 57%, Bill Maher, Climate Science Survey, Consensus, Fabius Maximus, GWPF, Jo Nova, Politifact, richard lindzen, Rick Santorum, Sur-Rebuttal, survey, testimony
My view on climate change 
September 2, 2015 at 22:06
> maybe I should come up with a nifty pseudonym for myself as well
I call him the Editor, since it’s a shortcut of his own nick on his website, i.e. Editor of the Fabius Maximus website.
We need more editors anyway.
September 3, 2015 at 03:29
Hi Bart
Your survey is both good and useful. As I tried to warn you in advance, your reporting has left the door open for deliberate misinterpretation.
There is a strong consensus among climate scientists that human emissions of CO2 have caused half or more of recent warming.
66% specifically attributed half or more to emissions. It is possible that more may have done so had the questions been worded differently.
Those scientists with more numerous publications were more likely to make that attribution and were more confident in their judgment.
However, it is clear that the consensus among climate scientists in this survey does not approach the near unanimous agreement found in recent literature searches (Oreskes, Anderegg, Prall et al, Cook et al). Instead, the broad category of climate scientists in agreement on human contributions matches almost exactly that found in Bray, von Storch et al.
The findings of your paper suggest that more research is appropriate, both in attribution and confidence, but also in discovering why scientists with fewer publications do not share the same confidence as their colleagues with more publications.
September 3, 2015 at 05:05
==> “As I tried to warn you in advance, your reporting has left the door open for deliberate misinterpretation.”
Moving aside from the somewhat amusing notion of someone deliberately misinterpreting something….any kind of reporting leaves the door open for deliberate “misinterpretation.” That’s the thing about deliberate “misinterpretation.” Perhaps rather than expecting Bart to avoid the inevitable, it would be better to hold those who deliberately “misinterpret” accountable for their deliberate “misinterpretation.”
September 3, 2015 at 05:13
Maybe “misconscrew” might be a better term than deliberate misinterpretation.
http://www.urbandictionary.com/define.php?term=Misconscrew
September 3, 2015 at 06:48
Hi Tom,
That’s already a much better and more complete description of our findings than what you had previously written, e.g because it tacitly acknowledges the problem with taking these numbers at face value and it acknowledges an impotant difference between subgroups. Even better would be if you’d also include the results from Q3, which according to your interpretation would be 83% consensus for all respondents. Bonus points if you explain the difference in result for both questions.
September 3, 2015 at 07:10
Hiya Bart! How are you doing?
I’d love to earn those bonus points. Send me the data and I’ll write it up!
September 3, 2015 at 07:19
Joshua, perhaps you haven’t followed closely the correspondence between Bart and myself regarding this. Or perhaps motivated reasoning leads you to an indifference to fact.
Survey results are easy to misinterpret, especially by those unfamiliar with the chain of events involved in getting the results in front of you.
There is a protocol for presentation of survey findings. It’s not an iron-clad rule and nobody has to follow it. But if you follow the protocol and people misinterpret the results, that’s on them. If you don’t follow the protocol and people misinterpret the results, that’s quite possibly on you.
This really was a good survey. It has the potential to help our understanding of what scientists believe. The reporting so far, both on the internet and in the paper published recently could be greatly improved upon.
But as I recently wrote here, it won’t make any difference to what you write.
September 3, 2015 at 19:05
> your reporting has left the door open for deliberate misinterpretation.
Speaking of which:
How to mind probe and beg the question in one sentence.
Since all we have for argument in that post is “I do this for a living,” may we presume that Groundskeeper misprepresents for a living?
September 3, 2015 at 19:11
willard, I stand by that and have said this to Bart directly. You are as usual not adding to the conversation. But that never seems to bother you.
September 3, 2015 at 19:30
> I stand by that and have said this to Bart directly.
How to add to a conversation in one single step. For more steps, see also:
Arguing by assertion is bad reporting. Groundskeeper has yet to link to that post on this thread. Bad reporting. Quoting and citing this post is a contribution. This indicates that he’s misreconscrewing about me, yet again.
Yet he does this for a living.
***
Clicking on his handle leads to another post, where we learn that Humans will use 3,000 Quads by 2075. If they all come from coal we’re ruined. Interestingly, Groundskeeper rips off his shirt in just about every thread using his favorite meme, i.e. “but CAGW.”
You just can’t make this up.
September 3, 2015 at 19:50
Oh, and speaking of conversation, I’ll note that BartV’s point:
has yet to be addressed. Since the hurricane has now passed and that another post where GW basically repeats himself without adding anything has been published, it might be time for him to respond to it besides the usual “but SkS” and some more hammering on the table instead of adding to the conversation with AT on that earlier thread.
***
I also have a question regarding GW’s “report”:
The number adds up to 97%. Where has the 3% gone?
Something’s amiss in this “report.”
September 3, 2015 at 21:33
Tom,
On your blog you keep accusing me and my co-authors of burying the bad news because we were somehow not happy with what you consider the one and only true result. That is plain not true and I’ve told you so before. We differ in opinion on how to distill a meaningful number on the level of consensus from our results. I’ve given our reasons, based on evidence, multiple times (incl in the blogpost above). You’ve given yours, based on your presumed authority, as well multiple times.
It’s a bit disappointing to see you repeating unsupported accusations about me and my co-authors.
Since you’re such a staunch defender of your approach, perhaps you would care to answer my hypothetical alternative question:
But all these issues aside, we asked another question about causes of climate change! And that question didn’t have the issue of a surprisingly large fraction of undetermined responses. The consensus based on that question for all respondents is 86% or 83%, depending on whether you include or exclude these undetermined responses. Unless you have a good alternative explanation of why so many respondents didn’t provide an estimate for Q1, I suggest you used Q3 instead of Q1 if you insist on including undetermined answers. That way don’t have to make assumptions about these undetermined responses.
September 4, 2015 at 03:48
Bart, I’d be happy to go through specifics on your questionnaire if you’ll provide me with the data.
I’m not ‘accusing’ you of anything. I’m stating as fact that you mention Q1 one time, by number not by name and lump it in with the totals of another question. You mention the global percentage one time in a compound sentence with the results from another question.
The purpose of your survey was to measure the strength of consensus opinion among published climate scientists (with a few skeptics tossed in for balance). Your first question attempts to measure the strength of the consensus and you received enough responses to provide you with a result.
But you mention it once obliquely and spend the rest of your time talking about other issues, mostly issues that provide inferred explanations on why the answer to your question is 31 percentage points below the results published by your co-author. You infer difficulty of response, you write pages on the effects of publication success.
But. you. don’t. discuss. the. principal. finding. of. your. study.
As it happens, I agree that the actual consensus is probably higher than the 66% that you found. But you have to address it head on.
The first sentence of your paper and all of your blog posts should read, “We asked 1,868 published climate scientists what percentage of recent warming can be attributed to anthropogenic emissions of greenhouse gases. 66% said half or more.”
That’s why you conducted the survey. That’s why you asked the question. That’s why they provided you with their responses.
September 4, 2015 at 11:42
Here’s something to ponder. You do a survery of people you regard as experts. You might be looking for their opinion, but it’s their expert opinion, not simply some personal opinion. Some fraction respond with “don’t know” to a question for which there is an actual answer that an expert should be able to give. Do you leave the people who said “don’t know” in the survey or not?
The problem I can see with Tom’s view is that his result will probably depend more strongly on who was included in the survey than Bart’s result. A more inclusive survey would give a smaller value than a survey targeted at recognised experts. By excluding those who said “don’t know” (and hence, presumably, those who are insufficiently expert to give an actual response) you probably make the result less sensitive to the survey choices.
September 4, 2015 at 13:27
ATTP has it exactly right:
Though it’s probably not only those who consider themselves insufficiently expert who didn’t give an estimate. but also scientists who didn’t feel their opinion would be properly reflected in one of the narrow answer options. In any case, our interpretation is more robust in light of the large amount of undetermined responses.
Tom, we had multiple objectives with the survey, first and foremost to characterize the spectrum of scientific opinion about issues that ‘climate skeptics’ often bring up and that as a result play an important role in the public debate about climate change. In our first analysis we chose to focus on attribution (about which there were two questions by the way; that hasn’t registered yet judged from your responses) and as part of that on the issue of agreement with human induced warming (again, based on those two questions); on how the IPCC statement may be misunderstood partly due to the interplay between aerosol cooling and greenhouse warming; on media exposure of scientists with various opinions. That’s a lot broader than you make it out to be. It’s fine to focus on one aspect, sure, but don’t pretend that that’s all there’s to it and ignore other relevant questions and issues, especially as they may influence the conclusions you might reasonably draw.
September 4, 2015 at 13:28
ATTP, a lot of people have misunderstood my comments about this survey.
I have no objection to Bart’s focus on expertise and bringing the comments about the difficulty of providing an answer to Q1 to the table. That’s what analysis is all about.
Nor do I object to Fabius Maximus making his own inferences with regards to level of confidence. He’s free to make his own interpretation of the data.
But those are essentially second order results. They need to be put into context with the primary findings.
Remember the Bray von Storch survey–they came up with a 66% consensus as well, even though close to 85% were very concerned about future warming and a similar percentage expressed strong support for the IPCC.
There’s a lot in the survey to analyze. But by focusing primarily on level of expertise and leaning heavily on comments about the difficulty of answering the question, the various web posts and even the paper do a disservice to the audience and the people who took the survey.
First things first.
September 4, 2015 at 13:58
Tom,
The 66% result is not hidden. Yes, you might have to actually read the paper to find it, but it’s not difficult. You seem to be arguing that they didn’t highlight it enough, or focused on things that are secondary. Fine, but that’s simply a disagreement with what they decided to focus on when writing the paper and is – largely – a pointless discussion.
September 4, 2015 at 14:45
ATTP, as I wrote, it is mentioned once in a compound sentence far down the paper as ‘Q1’ and lumped together with ‘Q3.’ It’s why they did the survey. You don’t do that.
September 4, 2015 at 15:01
Tom,
Seriously? People who make statements like this typically don’t know what they’re talking about. I’ve seen nothing to convince me that you’re bucking that trend.
September 4, 2015 at 16:28
Okay, ATTP. Back to the yah-boo games. Considering what you know about opinion polling probably fits in a thimble, go right ahead on with your preaching.
September 4, 2015 at 17:57
Tom,
I’m not preaching. I’m simply pointing out that people who make absolute statements about how things should – or should not – be done, typically don’t know what they’re talking about. You could stop doing that, and I would stop suggesting that you don’t know what you’re saying. Not that complicated an idea. You could always give it a try.
September 4, 2015 at 20:17
> I’m not ‘accusing’ you of anything.
As long as it remainds in Groundskeeper’s mind, it can’t be an accusation per se:
Guilt trips by association ain’t accusation anyway:
Op. Cit..
INTEGRITY ™ — Not ‘Accusing’
September 5, 2015 at 01:24
==> “Considering what you know about opinion polling probably fits in a thimble, go right ahead on with your preaching.”
Interesting. So Tom elf-references his self-authority to validate his own opinion, with no apparent hint of irony.
September 5, 2015 at 02:18
…and the regular crowd shuffles in…
as far as elf referencing, Joshua, no apparent hint of irony there. Coming from a master of another form of Middle Earth behaviour that’s a bit rich.
A reactionary conservative with a two-digit IQ is using Bart’s survey to advance his candidacy for the Republican nomination for president. If you want to insist that there is nothing wrong with how the results are presented, fine. Except what Santorum is using is not what Bart wants used. Hence pretty much proving that inadequate reporting invalidated his results on that instance.
willard, ATTP and Joshua say I’m wrong. All we need are sod and BBD and it will be unanimous.
Bart has a tag on this blog called ‘Communication.’ It has 36 posts. Probably should be a few more. Pity that the tag ‘Skeptics’ received far more attention–52 posts. Misplaced focus and all that…
September 5, 2015 at 05:34
Tom –
OK. Nice. Two-digit IQ. I see that you maintain your high standard of dialogue.
So Santorum is using Bart’s survey, (except he isn’t because he’s using probably Jo Nova’s distortion of Bart’s survey), to advance his candidacy. And the reason why is because of what’s wrong with how the results are presented.
And Trump is using Obama’s nationality and religion to advance his candidacy because of what’s wrong with how Obama has presented his faith and place of birth.
Thanks for the explanation.
And thanks god we have folks of your expertise to explain how Bart could have presented his results in such a way that presidential candidates couldn’t have distorted them to advance their candidacy. Now please get to work on helping Obama to benefit from your expertise also.
September 5, 2015 at 06:13
When we in future wish to find an example of false equivalence, we need to look no further.
September 5, 2015 at 15:49
Oh. OK. So if Santorum distorts Bart’s evidence to advance his candidacy, we should be holding Santorum accountable for the distortion ,and not Bart for providing un-distorted evidence?
I glad we finally agree on something.
September 5, 2015 at 16:00
Not quite. I suggested that anyone who makes an absolute claim about how things should – or should not – work typically doesn’t know what they’re talking about. That’s of course based on my experience and isn’t always true, but absolute statements typically mean that person doesn’t know how to convince people that they’re right, so they – instead – simply insist that they’re right. Remember that this is not an RCT.
September 5, 2015 at 17:02
That’s okay, ATTP. It’s not the first time we’ve seen you pronounce on subjects you know nothing about.
September 5, 2015 at 17:32
> A reactionary conservative with a two-digit IQ is using Bart’s survey to advance his candidacy for the Republican nomination for president.
http://www.kidsiqtestcenter.com/Rick-Santorum-IQ.html
***
> [W]hat Santorum is using is not what Bart wants used. Hence pretty much proving that inadequate reporting invalidated his results on that instance.
(P1) Rick exploited Bart’s study.
(P2) All exploited studies are invalidly reported.
(C) Bart’s reporting of his study is invalid.
P2 is far from being obvious. It also presumes that Bart would have the power to prevent P1. From false premises one can infer just about anything.
***
How inadequate reporting invalidates study results deserves due diligence.
September 5, 2015 at 18:05
No, I think I do actually have some experience of situations where people would rather appeal to their own authority than make an actual argument.
September 5, 2015 at 19:48
Tom –
I have to believe that in your heart of hearts, you well know that the argument that Santorum misrepresents the prevalence of expert opinion because of how Bart reported his results, is nonsense.
I’ll be content to know that you know that you’re wrong, and that you just feel a reflexive need to pretend otherwise as a face-saving measure.
I won’t think any lower of you for it. Promoting absurd face-saving arguments rather than admitting to bias and/or error is a very human trait; you’re far from alone.
September 5, 2015 at 21:47
Guys,
Let’s stop the verbal ping-pong of telling each other in yet another way how much you disagree with them; that’s not conducive to having a constructive conversation.
September 6, 2015 at 01:50
Bart, next tune you do this you might try a different way of capturing the ranges you asked about.
Ask them about the minimum first.
“What is the minimum percentage of recent global warming attributable to human emissions of greenhouse gases?”
Present answers in 5 degree gradients. They tick a box to proceed.
Then ask for the maximum:
“What is the maximum percentage…”
Present in 5 degree gradients.
Pipe their responses (most survey software programs can do this) into a question that verifies their previous answers:
“Okay. You told us you think that human emissions of greenhouse gases account for between X% and Y% of recent warming. Does that capture your opinion accurately?” Yes or no. If no, they are led back to the first question.
This allows you to interrogate on the range. Or, if you want to keep the survey short, you can just analyze the range between min and max. It would be interesting to see who felt the range was narrow and who thought it was wide.
Anyhow, this might have avoided the problem you encountered with some respondents.
September 6, 2015 at 05:55
ATTP, although the first ‘symptom’ of the Dunning Kruger effect gets all the attention, the second is perhaps more relevant to you.
Fail to recognize genuine skill in others, much?
September 6, 2015 at 09:45
Tom,
Firstly, did you not read Bart’s comment. Secondly, I think I’ve avoided making inferences about your mental state here. Any chance you could do the same? Thirdly, my point is that it’s hard to see genuine skill in others when they simply state that they have it. Seriously, I really am trying to get you to recognise that “I’m right, you’re wrong”/”noone ever does that” is particularly unconvincing. You really could try to make an actual argument, rather than simply stating something as true. I’ll even give you a start.
This is not a randomised control trial. The data in Verheggen et al. is much more like the kind of data a physicist is used to dealing with. You collect data and then you do some kind of analysis to extract information. In such a scenario, few people would ever state – with certainty – what you can and can not do with that data. As long as you are clear about what you’ve done and what result you get, then it’s your choice what you choose to publish. However, you can – of course – disagree with the conclusions that are drawn.
So, for example, your claim seems to be that the core result is that 66% of those surveyed agree with the consensus. Fine, that indeed seems true. That, however, does not mean that that is the most interesting or relevant result. Why? Because as I’ve said in an earlier comment, that result is quite dependent on the survey setup. A more inclusive survey would probably produce a smaller value. A less inclusive survey would produce a larger value. Hence, since we are probably interested in the expert opinion, we remove those who claim that they “don’t know” and report that result as the headline result. Not only is it probably less sensitive to the survey setup than the basic result, it’s also more relevant to what’s of interest (what is the expert opinion).
You can of course choose to disagree with that and construct an argument as to why something else should be presented. Simply stating “this is never done” is not a good one. Also, if you actually disagree you typically publish something with a different analysis, not just state on blogs that the original work is wrong.
September 6, 2015 at 10:40
Gee, ATTP, thanks so much for explaining to me what a survey is. Real data? Analysis? Wow…
ATTP, people do offer guidelines on what to do with the data, including for analysis and reporting.
They do it in the hard sciences–everybody from the EU to your university provides research guidelines and may have sections devoted to analysis and reporting. Same with guidelines on research ethics.
They also do it for market research. In your neck of the woods, the MRS has a specific section for online research. Out here in Asia we tend to rely on ESOMAR guidelines. Those in the States have choices! They like choices in the States–CASRO, the MRA, even the AMA provide guidelines.
In many of those guidelines you see phrases like, “Authors have a duty to make publicly available the results of their research on human subjects and are accountable for the completeness and accuracy of their reports. They should adhere to accepted guidelines for ethical reporting. Negative and inconclusive as well as positive results should be published or otherwise made publicly available”
and… “Several examples of potentially
misleading data reporting are illustrated, including
using inappropriate statistical tests, neglecting negative
results, omitting missing data points, failing to
report actual numbers of eligible subjects, using inappropriate
graph labels or terminology, data dredging,
and others.”
I have a lot of respect for Bart Verheggen and I am not saying that any of the actions described in quotes were done by him or his research team. But you seem to think that publicly funded research results can be treated just in any old way the researcher desires.
Well, Lewandowsky, so f**k me. But real researchers face constraints.
You’re a strange man, ATTP. It would not occur to me to say you know nothing of physics, even though you rarely write about it and when you do you are often forced to correct what you write.
We’re all dogs on the internet, but why would you say I know nothing about surveys? You’re a strange man.
September 6, 2015 at 10:47
Tom,
I did no such thing. I constructed an argument as to why one might present the result that Bart did, rather than the result you seem to prefer.
Possibly because I’ve never said that. Why would you make that up?
As it stands, I’ve yet to see you actually construct an argument against what I’ve said and what was actually presented in the paper. Are you actually going to do so?
September 6, 2015 at 10:56
To steal my co-author’s favorite line, if you think I haven’t constructed an argument against what you’ve said and what was actually presented in the paper, ATTP, ‘read harder.’
Here is the abstract for the paper ‘Scientists’ Views About Attribution of Global Warming.’ Find the answer to Q1.”What fraction of global warming since the mid-20th century can be attributed to human-induced increases in atmospheric GHG concentrations?”
“Results are presented from a survey held among 1868 scientists studying various aspects of climate change, including physical climate, climate impacts, and mitigation. The survey was unique in its size, broadness and level of detail. Consistent with other research, we found that, as the level of expertise in climate science grew, so too did the level of agreement on anthropogenic causation. 90% of respondents with more than 10 climate-related peer-reviewed publications (about half of all respondents), explicitly agreed with anthropogenic greenhouse gases (GHGs) being the dominant driver of recent global warming. The respondents’ quantitative estimate of the GHG contribution appeared to strongly depend on their judgment or knowledge of the cooling effect of aerosols. The phrasing of the IPCC attribution statement in its fourth assessment report (AR4)—providing a lower limit for the isolated GHG contribution—may have led to an underestimation of the GHG influence on recent warming. The phrasing was improved in AR5. We also report on the respondents’ views on other factors contributing to global warming; of these Land Use and Land Cover Change (LULCC) was considered the most important. Respondents who characterized human influence on climate as insignificant, reported having had the most frequent media coverage regarding their views on climate change.”
See next comment.
September 6, 2015 at 11:05
Here are results as reported in Bart’s paper.
Section 1 is Aggregation of Results. Section 2 is Attribution.
Section 3 is titled Consensus.In the fifth paragraph of this section is the only mention of the headline finding of the report: “As a fraction of the total, the level of agreement based on Q1 and Q3 was 66% and 83%, respectively, for all respondents, and 77% and 89%, respectively, for the quartile with the highest number of self-declared publications.”
The question is not identified by wording, only by number. It is combined in a compound sentence with Q3. The percentage is separated from the question and combined with the percentage for Q3. The sentence continues to express percentages for the quartile with the highest number of self-declared publications.
That is the only mention of the global response to the question in the report.
As I said, I respect Bart Verheggen. As I have also said here and elsewhere, this is not best practice in reporting. It has the effect of hiding the global response to the question of most interest to both the scientific community and the general public. The question is of obvious importance to the researchers as well, given the amount of time they devote to analyzing responses to it by subgroup.
There is no excuse for not making the response rate clear.
September 6, 2015 at 11:10
Tom,
You’re still doing it. Where is your argument that this is the case. You’re simply telling me. My argument is simple. This is a dataset. As long as you’re clear as to your analysis, there aren’t strict rules as to what you can and cannot do. Telling me otherwise without actually constructing an argument is not very convincing. Also, you haven’t answered my question as to why you would claim I had said something that I have not.
September 6, 2015 at 11:15
ATTP, no, it is not a dataset. It is a paper based on analysis of a dataset. There are not only rules about how information should be presented, there are also conventions and traditions on where information should be reported and how it should be reported.
You are once again talking about something of which you are completely ignorant.
September 6, 2015 at 11:17
Fuller: “The question is not identified by wording, only by number. It is combined in a compound sentence with Q3. The percentage is separated from the question and combined with the percentage for Q3. The sentence continues to express percentages for the quartile with the highest number of self-declared publications.
That is the only mention of the global response to the question in the report.”
ATTP: “Where is your argument that this is the case.”
The infinite loop of the climate cult. Did not. Did too. Did not. Did too.
September 6, 2015 at 11:25
Tom,
Yes, I know. That’s what I’m saying. Repeating what I’m saying to me as if I hadn’t said it is silly.
I’ll let other people be the judge of that Tom.
September 6, 2015 at 11:27
ATTP: “My argument is simple. This is a dataset.”
Fuller: “it is not a dataset. It is a paper based on analysis of a dataset.”
ATTP: “Yes, I know. That’s what I’m saying. Repeating what I’m saying to me as if I hadn’t said it is silly.”
September 6, 2015 at 11:28
Totally ignored by ATTP: ” It is a paper based on analysis of a dataset. There are not only rules about how information should be presented, there are also conventions and traditions on where information should be reported and how it should be reported.”
September 6, 2015 at 11:29
Similarly unaddressed by ATTP: “The question is not identified by wording, only by number. It is combined in a compound sentence with Q3. The percentage is separated from the question and combined with the percentage for Q3. The sentence continues to express percentages for the quartile with the highest number of self-declared publications.
That is the only mention of the global response to the question in the report.”
September 6, 2015 at 11:34
Tom,
You still haven’t explained why you claimed I had said something that I had not. However, you want me to address this, so here goes. It is mentioned. That you have to actually read and digest the paper does not make this not true. You might think it should have been highlighted more. That you think this does not make you right. Here’s the bottom line. Verheggen et al. is a published paper. It went through peer-review and is published in a reputable journal. Your claims that some things are never done seem to be refuted by the simple fact that they clearly are.
September 6, 2015 at 19:28
If you have new and interesting things to say, please do so. But repeating the same thing over and over is neither new nor interesting, so please don’t.
And even more importantly: no accusations of wrongdoing -however implicit- without solid evidence of such. I removed the last string of comments for this latter reason.
September 6, 2015 at 20:14
AT, quote mined by GW:
GW’s response, quote mining himself:
***
Compare and contrast:
AT, in full:
GW’s response in full:
Citing these rules, conventions or traditions might be nice.
October 5, 2015 at 11:42
In the New York Times article “Enemies of the sun” ( 5 October 2015) economist Paul Krugman explains why the American Republicans are “fossilized” (i.e why they only support fossil energy) and why they reject wind and solar energy.
You can find this article on the internet.
June 4, 2016 at 08:18
You seem to be straining to get your results to fit into the “97 percent consensus”. The problem though is you are asking the correct question while those getting the 97% number are not.
The key tenet of AGW is that humans cause MOST climate change, and that is the question that should be asked as you did. But most surveys ask the question as simply “Do humans cause climate change?” This can be interpreted to mean “Do humans make ANY contribution to climate change?”
The problem with this then is almost all scientists even opponents of AGW agree that humans have some influence on climate change. The question therefore has to be phrased properly to accurately represent what the theory of AGW is really saying that humans cause MOST climate change.
So the fact in your survey you got a large number of “I don’t know” responses is an important fact and the large number of those responses should not be disregarded. It is representing a real fact that on the question of do humans cause MOST climate change there is still a great deal of uncertainty on that question.
Support for the idea that your survey deviates from the “97%” number because your survey asked the correct question “Do humans cause MOST climate change” is provided by another survey:
https://www.purdue.edu/newsroom/releases/2014/Q4/study-farmers-and-scientists-divided-over-climate-change.html
This survey also found when the question is asked “do humans cause MOST climate change” there is a large deviating from the “97%” number. It found the breakdown to that question was about 60% Yes and 40% No or “I don’t know”. Again the fact that a large proportion answered “I don’t know” is representing a real fact that should not be disregarded. It’s showing there is still a great deal of uncertainty on the question do humans cause MOST climate change.
Bob Clark
June 13, 2016 at 11:11
Hi Robert,
I’m not “straining” anything; I’m comparing the results from our survey to other somewhat comparable surveys and trying to explain the similarities and differences. As you rightly point out, one key difference is that we asked a much more precise question regarding attribution than most other surveys did, and in doing so we arguably set a higher bar for what we consider the ‘consensus position’ to be. There are more differences, e.g. in the make-up of the survey sample (we surveyed the broader scientific field who studied various aspects of climate change, so not only physical climatologists; also, we included a small but notable fraction of outspoken contrarians in our survey sample. Both these aspects lead to a lower level of consensus in our study.) See also the FAQ about our climate science survey (specifically item 4).
There’s a downside to the level of precision in our attribution question though: we asked respondents to choose between 6 percentage ranges of GHG contribution. That is rather tricky, and probably goes a long way to explaining the large fraction of unknown/don’t know responses for that question (see a more detailed argument for that here). If we had just asked respondents to choose between most warming vs less than that, the fraction of unknown/don’t know responses would in all likelihood have been smaller. We have evidence (see link above as well as the ES&T article) that the large fraction of unknown/don’t know responses is due to the difficulty that the question posed, by forcing respondents to choose a very precise range of GHG contribution. Many of these respondents are thus not necessarily agnostic about the question, but chose rather not to fill out one of the prescribed ranges. This is corroborated by comparing the responses to the other attribution question and by inspecting respondents’ open comments.
So I’m not disregarding anything; rather I’m trying to get to the most plausible explanation of our findings, considering all responses.
Thanks for the link the the agricultural survey. I don’t think it’s very comparable to surveys of large numbers of climate scientists, since in that survey only 19 state climatologists were surveyd (a very small number and a particular -and I think on balance more contrarian- subset of climatologists). What stood out in that survey was that the group labeled as “extension educators” was so much more skeptical of human induced global warming than the scientific community is. That is particularly striking considering the description of that group: “Extension educators are a unique set of agricultural advisors who serve to connect and translate research from universities to farmers in order to decrease risk to the farm enterprise and increase productive capacity and resilience.” From these survey results, looks like a at least half of them have an opion about climate science that is strongly at odds with their stated job description of translating research to stakeholders.