On the previous post a discussion about climate surveys erupted. Please continue here. I’ll chime in when I habe more time.
I’m curious to hear what you find good or not so good about a particular survey and why. If you have contributed to the discussion on the previous thread, perhaps you want to condense your main arguments in one comment so that we can start a more structured discussion. If succesful, I’ll follow up with a post including the various contributions. Call it crowdsourced blogging if you wish.
A noteworthy survey that has not been discussed much is the one by Brown, Pielke Sr and Annan that I wrote about before:
A recent survey of scientists having authored a recent journal article on climate change found that the majority concurred with the IPCC position. A sizeable minority was of the opinion that the IPCC reports overstate the importance of and/or certainty regarding CO2 compared to other forcings (both natural and anthropogenic!). Only very few respondents were of the “skeptical” opinion that warming is predominantly natural. Nobody denied that the globe is warming. A large minority found the IPCC too cautious, understating the human influence on climate and/or the seriousness of the problem. Note that the authors do not claim that their survey is representative; less then 10% of the 1800 scientists contacted replied. Moreover, the positions that respondents had to choose between were sometimes a little ambiguous.
Others that have been discussed a lot include those by Bray and von Storch and the recent one by Doran and Kendal-Zimmerman.
Tags: climate change, survey
My view on climate change 
January 3, 2012 at 22:49
I won’t have time to get into this for a day or so, but I’m happy to start the ball rolling by stating what is not at all controversial–that almost all scientists (97%) agree with a very generally worded statement along the lines of ‘that human addition of CO2 into the atmosphere is an important component of the climate system and has contributed to some extent in recent observed global average warming’. (from Pielke’s survey).’ Further, no climate scientists believe that global warming is a fabrication or that human activity is not having any significant effect on climate.’ (Ibid).
If that settles the story, we can all relax. Sadly, I highly doubt that it does. I agree with the scientists completely on these two statements. I think that many of the people who are loathed by your regular readers do, too. I sometimes wonder why…
The two question internet survey that I object to so vehemently more or less stopped there (or even a little before, as it was so anodyne as to be agreeable to Morano and Monckton, IMO). Pielke and von Storch went considerably further, which is where it starts to get interesting.
I’ll be back with some analysis when time permits. Thanks for posting specifically on this, Bart.
I’ll close by saying the public in the US and elsewhere largely agrees with the scientists–as Pielke’s son is fond of pointing out. But the devil’s in the details, as always.
January 3, 2012 at 23:54
Bart, I just looked at your request a little more carefully, so very quickly:
The first rule of a good survey is sample. Getting the right audience to respond. For a survey such as those being discussed here, that would mean getting climate scientists as opposed to those in other fields by mistake. It really is the key to success of a project. It’s very easy to pad the results of a study by getting lots of answers from a sympathetic group that falls outside the sample parameters, but that doesn’t contribute to a better understanding. (However, if you prepare for it, you can compensate. A couple of years ago I did an Examiner survey and got some bloggers to post links on their blogs to it–you among them, if I recall. When Anthony Watts agreed to it, I had to scramble to make sure my analysis frame could cope with the preponderance of Watts’ readers without prejudicing the results.)
More later…
January 4, 2012 at 01:08
First rule ofthese surveys is not to let Tom Fuller near them:)
Seriously Tom, you didn’t bother addressing anyone’s concerns before, and we’ve seen you in action so we already understand how you use survey results and it isn’t pretty.
January 4, 2012 at 01:13
I , personally, don’t have an opinion about the survey’s in question – just the way the data was used by Tom.
January 4, 2012 at 01:31
Bart,
Thanks for this separate thread. Reading Nathan’s comments here provided the laugh of the day.
January 4, 2012 at 03:21
“Only very few respondents were of the “skeptical” opinion that warming is predominantly natural.”
Even I don’t fall into this category, although I can’t say the opinion would be wrong.
January 4, 2012 at 04:02
Second look at Pielke’s survey shows that it provides an excellent opportunity for scientists to express a wide variety of opinions, by marking two adjacent options to indicate when an opinion falls between stated options. However, all analysis of that would be manual, and would be really time consuming. My heart goes out to whoever compiled the responses.
Von Storch’s approach is more standardized, and slightly reduces the ability of respondents to offer gradations of opinion, but is far easier to analyze. It’s the method I would have chosen.
The two minute, two question disaster drill is similar to ROTR (Run of the River) surveys posted on popular websites for instant public opinion snapshots–not really worth anything at all.
More later (sorry, Nathan).
January 4, 2012 at 10:15
Paul, what’s so funny about Nathan pointing out that Tom Fuller consistently ignores concerns about the way he interprets surveys?
For example, I am still waiting for him to explain how he can claim that the Von Storch/Bray survey shows that statements on polar bears, Himalayan glacier melt, African drought, etc etc. are exaggerations and that most scientists will have none of that. There are no questions on these aspects in the survey *at all*. As in “none”.
And that’s just one concern. He also consistently ignored my request for him to explain how he interprets the word “adequate”, how he knows the climate scientists that were surveyed interpreted it the same way, and why that means it should concern me.
Really not difficult questions to answer, and rather important for the discussion. Especially when he then only attacks me on a strawman (and asks me a question I already answered…).
January 4, 2012 at 11:04
You and your readers might like to look at http://en.wikipedia.org/wiki/Scientific_opinion_on_climate_change and the talk-page discussion http://en.wikipedia.org/wiki/Talk:Scientific_opinion_on_climate_change#Putting_uninteresting_and_old_surveys_first and http://en.wikipedia.org/wiki/Talk:Scientific_opinion_on_climate_change#Sub_article
My own opinion of surveys is there, but to save you the trouble:
“With the surveys, there are two internal problems: selection [or responders], and the questions asked; the Bray and von S ones suffer particularly from this. The Oreskes work doesn’t suffer from those problems, though, because it isn’t a survey of people but the literature. I think that is good; what matters in science isn’t what people think – that isn’t, so to speak science – but what gets into the literature. Or rather, what is the conversation between scientists; that used to be the literature, arguably nowadays blogs and preprints and stuff count for more; but still, opinion is over rated. Inverted, that is why I’d rate statements by organisations higher…”
January 4, 2012 at 12:23
Tom,
to be honest I don’t have much interest in what you have to say about the survey. You said a lot, and it was not very enlightening. I don’t see how you can improve what you’ve already said. And if you actually had any argument of substance you would have said it already. Giving you ‘think time’ is not going to improve anything.
Paul Kelly
I thought you meant you enjoyed my joke about keeping the surveys away from Tom – are you laughing with me or at me?
January 4, 2012 at 15:52
Looking at the surveys makes it clear that the majority of scientists agree with the basic proposition that has driven examination of the issue over the past few decades.
What I hope to show if and when I get time is that there is considerably more diversity of opinion about a few key and many minor elements of the discussion among active and publishing scientists in the field. These differences are ignored in popular discussion, and I and others have been criticized for pointing them out–or, in forums such as Stoat or the portion of Wikipedia that Connelly patrolled, actively shut out (or edited out) of the discussion.
What some seem to find disturbing is any evidence at all of cracks in the public facing narrative regarding scientific solidarity. As has often been the case, attempts to paper this over have caused more problems than the differences warrant. Nobody is saying there is less than a majority of scientists backing the IPCC and its world view. Nobody is saying that the majority of papers don’t back it either. But when Oreskes artificially structured her literature search so as to be able to avoid any or all skeptical publications, it hurt her findings.
The majority of scientific opinion (for whatever that’s worth) sides with you and yours, Bart. But it’s possibly closer to 81% than 97%, if the votes are counted properly.
More on that later
January 4, 2012 at 20:29
“And that’s just one concern. He also consistently ignored my request for him to explain how he interprets the word “adequate”, how he knows the climate scientists that were surveyed interpreted it the same way, and why that means it should concern me.”
Which brings me to my initial impression. The “other” Dunning-Kruger effect. How does one allow for the responses on such a survey if you’re dealing with people who are extremely knowledgeable and competent on the issue?
When such people say they ‘don’t know enough’ or ‘we need better data’ or similar comments, do they mean exactly what the questioner had in mind? Being part of the highly expert system in question often means that people set much higher standards for ‘adequate’, good enough’ or ‘excellent’ than others might.
( Reminds me of my daughter’s comment when working at a place that had an enormous variety of acts, often in the same week. She observed that popular or rock musicians came off stage all hyped up and pleased as punch that everything went so well. The classical musicians came off stage and promptly went through all the tedious details of what had to be improved for a better performance next time. The audience never noticed anything needing ‘improvement’.)
January 4, 2012 at 21:55
“I’ll be back with some analysis when time permits. “
“More later…”
“More later (sorry, Nathan).”
“More on that later”
This reminds me of when my phone starts blinking the notification LED to let me know the battery is low, stop using up the battery power to tell me about the battery!
Going back to the original discussion that led to the survey topic:
Tom: “Remember–the images trotted before us are specifically designed by the marketing gurus at NGOs to scare us, not educate us. Polar bears, Himalayan glaciers, flooded cities, hurricane-like tornados–these were commissioned and produced by our friends at Greenpeace and the WWF to stimulate concern, political support and donations.”
Me:“I believe the same concept applies more generally – some types of arguments and conclusions have a “bad smell” associated with them. In biology arguments about the complexity of the eye and whether Charles Darwin believed this or that have a bad smell. In medicine arguments about trace elements in vaccines have a bad smell.
Of course examples in the climate blogosphere abound but they include arguments about Michael Mann, the hockey stick, polar bear populations and Himalayan glaciers. They have a bad smell because they reveal the sources of information the claimant utilises and trusts and those sources are not scientific in nature.”
Tom:“Sharper00, happy new year to you, too. Funnily enough we agree on the examples you bring to bear as odoriferous. Sadly, I doubt if we agree about the origins of the odor.”
Me: “No doubt Tom but as above people that find mainstream scientific positions odorous are coming from a perspective that is incompatible with mine.”
Tom’s next response brought up the Von Storch survey.
Now what I was talking about was what experts on Himalayan glaciers think about Himalayan glaciers, what experts on polar bear populations think about polar bear populations and so on. It’s my belief that the views Tom Fuller expressed run contrary to theirs. In terms of support I’d expect someone to reference scientific literature rather than an implied claim by an anonymous journalist.
Tom’s response was to start talking about what climate scientists think in general about the climate science debate and the politics/media portrayal of it. He appears uninterested in Himalayan glaciers or polar bears but rather in exaggerated claims of which he believes they are examples. Furthermore he seems almost entirely focussed on specific personalities he believes make those exaggerated claims and his desire to marginalise those personalities is strong enough that when he sees “exaggeration” in the Von Storch survey he can’t imagine anything but those particular people making those types of claims. Attempts by multiple people to break that link and explain why it doesn’t follow were unsuccessful.
As noted by William M. Connolley the only thing that’s truly important about what a scientist thinks is the specific field in which they have expertise, of that the only important part is what they’ve had published. That is what represents our state of knowledge on Himalayan glaciers, polar bears and everything else.
Factually oriented people cannot accept claims on factual matters without facts. I cannot accept claims about Himalayan glaciers are exaggerated unless such claims are compared to what the experts believe. I cannot accept that someone answering a survey about exaggeration believes Himalayan glaciers have been exaggerated unless the survey includes this information.
Tom Fuller is not factually oriented, he is personality oriented. Every time he writes he writes about people. He views climate sensitivity in terms of Morano and Romm not 0 degrees to 6 degrees. When someone argues with him he invokes Monckton and Morano. He insists people should trust not science or scientific principles but individuals like Curry and McIntyre.
People regularly become infuriated with Tom because they want to argue facts, he wants to argue personalities.
Unfortunately Tom’s trust network smells bad.
January 4, 2012 at 22:39
Can we aree that in the Bray and von Storch survey
(http://coast.gkss.de/staff/storch/pdf/GKSS_2010_9.CLISCI.pdf – Bart, can I suggest that you add links to all the surveys being discussed?) section 11 (questions 13-16), which asks how “adequate” data and theoretical understanding are, is too ambiguous to be useful? The term “adequate” is not expanded on. The data may be considered adequate to draw broad conclusions that warrant strong action, but totally inadequate given the seriousness of the threat we face and the need to understand in detail how we should respond.
The next section, which asks questions about the adequacy of models, is also similarly problematic and most interesting in that it shows how the responders view the comparative strengths of different aspects of the models.
Further down there are some less ambiguous, though still under-defined, questions. In section 22 question 58 is “How convinced are you that climate change poses a very serious and dangerous threat to humanity?” The response to this has a mean of 5.8, with 1 being “not at all” and 7 being “very much” . Question 109 “Over the issue of climate change, the general public should be told to be” has a mean response of 5.4, with 7 being “very worried”.
I will be using question 101 “Some scientists present extreme accounts of catastrophic impacts related to climate change in a popular format with the claim that it is their task to alert the public. How much do you agree with this practice?” when discussing biased and leading questions in questionnaires with my Year 7s (11-12 year olds).
January 4, 2012 at 22:49
sharper00, you failed your audition as a mentalist. PatrickO, not in my opinion. I think the survey’s valid. Adeladay, I don’t see anything I can or should respond to.
All, sorry I don’t have time to get to this. Sorrier still that it allows sharper00 the opportunity to try and muddy the issue. I’ll return to the earlier thread to further discuss polar bears, Himalayan glaciers, etc., and the weird idea that I care more about personalities than I do about science. But again, that will be later.
Have fun!
January 4, 2012 at 23:11
Nathan,
Laughter is the result of the simultaneous involuntary twitching of seventeen specific facial muscles, generally produced by the juxtaposition of two or more separate ‘truthful’ realities. In this case it was the passionate, but errant, denouncement of Fuller and the claim of a lack of opinion on the meaning of the survey. So, the laugh was neither with you or at you. It was because of the juxtaposition of the words. Note that the laugh required my accepting your lack of opinion to occur.
I was predisposed to laugh because you are such a shining example of having the wrong focus. One more anonymous poster so intent on winning a mostly irrelevant argument.
January 5, 2012 at 01:03
Paul
“In this case it was the passionate, but errant, denouncement of Fuller and the claim of a lack of opinion on the meaning of the survey.”
Passionate? No. Errant? No – Tom is quite clearly using this data poorly, to support his personal claims when he can do no such thing as he doesn’t know what the scientists actually thought.
“I was predisposed to laugh because you are such a shining example of having the wrong focus. One more anonymous poster so intent on winning a mostly irrelevant argument.”
Thanks. I get that you completely missed my point. Do tell, what is the ‘right’ focus? Please enlighten me.
And you too, Paul Kelly, are ‘anonymous’ – how many other ‘Paul Kelly’ users are out there? How do I know that’s your real name?
Tom,
You are still trying to present yourself as an impartial, objective, observer trying to shine a light on the ‘unspoken’. You are no such thing. You have an agenda and you are using this data to promote your agenda: that’s why you are intellectually dishonest.
January 5, 2012 at 01:30
At what point have I ever claimed to be a dispassionate observer? I am not and have never said so. I am a lukewarmer, and I want to be a fierce advocate of that position. Where do you get that stuff, Nathan?
I also have been working with surveys of this sort for a very long time. A very long time.
January 5, 2012 at 01:50
Nathan,
The proposition (which you call Fuller’s personal claim) that exaggeration of climate science harms the message is held by many.across the spectrum of political and climate viewpoints. He cites a survey that shows that a percentage of scientists are aware of exaggerations, some of which Fuller then names. For this to be a misuse of the data, one must believe that none of the scientists surveyed were are aware of or influenced in their opinion by any or all of the exaggerations Fuller alleges. That defies logic.
The right focus is on immediate, bottom up, socially based, pro-active deployment of fossil fuel replacing technologies and efficiencies.
January 5, 2012 at 02:08
Tom,
Ok, so why should we accept what you say, when you colour these studies with your opinion. Nowhere in the study does it speak of Lambert, Tobis or Romm.
“I am a lukewarmer, and I want to be a fierce advocate of that position.”
It’s wonderful Tom, but I don’t think you really know what being a ‘Lukewarmer’ means. Apparently something about climate sensitivity being lower? What you have done is just chosen a position, and have no real understanding of what it means. Where are you studies on the effects of global warming in a ‘lukewarmer’ scenario? What does it ACTUALLY mean? Simply claiming a low sensitivty is not enough, you need to explain what that means. What actions are different under Lukewarmer Scenarios than under the IPCC scenarios?
Paul
You have still missed my point. Tom madde claims about specific individuals. The data was misused because he made claims about specfic people. He cannot use the data to support these claims because he does not know what the scientists think. Nowhere in the survey does it define what ‘exaggeration’ is. Nowhere does Tom define it. It’s just a bunch of armwaving to promote an agenda (Tom admits above he wants to promote an agenda).
Neither of you actually read what I wrote previously did you?
“The right focus is on immediate, bottom up, socially based, pro-active deployment of fossil fuel replacing technologies and efficiencies.”
That’s wonderful and I whole-heartedly agree. However, this has nothing to do with what we are talking about – you have presented a non-sequitur.
January 5, 2012 at 02:52
Nathan, if I can be so bold, I might venture to suggest you do a bit more reading before you do a lot more writing…
January 5, 2012 at 11:21
Tom
What a pointless comment. I have repeated the same claims perhaps 3 times, and only now you claim I need to do more reading?
January 5, 2012 at 11:23
Tom,
sorry my mistake. I understand you’re tlkaing about being a Lukewarmer.
May I be so bold as to ask you to define it… Or at least show me where you have? Or show me how it differs from the IPCC?
January 5, 2012 at 12:37
Folks, please remain on topic (i.e. surveys) and refrain from personal insults.
January 5, 2012 at 13:10
Sure thing Bart.
January 5, 2012 at 13:41
It’s useful to know that there is broad agreement amongst scientists on the question of whether AGW is real and is a serious threat. It’s also useful to know where there are differences of opinion on the many individual issues which are relevant to the argument about AGW. So there are merits in having both a simple two question survey and a more detailed and nuanced one such as von Storch’s.
The key, as has been pointed out, in any survey is having questions which are clear and unambiguous. I haven’t read the whole von Storch survey, most of the exerpts I’ve seen are fine, but Q101 is clearly problematic – some people are asuming “extreme accounts of catastrophic impacts” to mean “exaggerated accounts….” but some possible outcomes would certainly be considered “extreme”. Exaggerating the likelihood or impact of such outcomes may be problematic, if this is what the question meant then it should have made this clear, but they should not be minimised either.
January 5, 2012 at 14:17
On the subject of polar bears and glaciers, any reasonable reading of the research on polar bear polulations would see that they are certainly under threat, nor is there any doubt that they live in the part of the world which will experience the most warming, so I don’t understand why some people seem to object so violently to this being pointed out. I guess one could make an argument that in the overall scheme of things polar bears are not that important and using them as a symbol of climate change is playing on people’s emotions. Well so what? There is no contradiction between communicating a message honestly and doing it in a way that has the maximum impact on the intended audience. Maybe the fact that this kind of communication is effective is precisely why people object to it.
As for glaciers, yes the IPCC made a cock up and handled it badly but that doesn’t alter the fact that rising temperatures unboubtedly pose a threat to glaciers, that this is already observable in some areas and that many millions of people rely on glaciers for their water supplies, which makes it a very real cause for concern.
January 5, 2012 at 18:04
Hi Andrew–happy New Year! Nathan, please don’t worry about it–no problem at all. We all arrive at message threads a bit unprepared for who we’re going to meet every now and then.
Sadly, still snowed under–hope to contribute something meaningful this weekend.
January 5, 2012 at 18:40
Hi Tom,
Same to you!
January 5, 2012 at 21:18
Lunch break digression (with mouthful of cracker and cheese). Andrew, would you agree that population of polar bears declined due to the normal stressors that have contributed to declines of major mammals elsewhere (i.e., habitat loss, hunting, etc.) that did not include climate change?
(I know you see what’s coming, but I’ll press on…) Would you then agree that the robust recovery of populations since the 1960s coincided with the dramatic temperature rises seen in the Arctic that has caused much of our concern about climate change?
If so, can you see why my discussion of polar bear populations tries to balance all the stresses placed on their success as a species, rather than focusing just on climate change?
January 6, 2012 at 00:04
Tom,
Sure, polar bear numbers did fall in the past due to the kind of factors you mention, but then concerted conservation measures were brought in and numbers increased over the latter part of the last century. And yes, this did coincide with rapid warming, but it’s taken the sea ice a while to fall to a level where it is a serious threat to the polar bears’ habitat. That doesn’t mean that climate change is the only threat to polar bears – I thing we would find in very many cases all over the world that the environmental threat posed by AGW is additional to stresses already being placed on the environment from other human activities.
So it’s fair to point out when discussing preservation of polar bears that we should consider all factors and not just AGW, but it’s also fair when discussing the threat posed by AGW to point out the threat it poses to the habitat of polar bears.
January 6, 2012 at 00:08
I agree, Andrew. I don’t want to minimize the impact AGW may have on all northern species, including polar bears. I just want to make sure we all realize it’s not the only, and may not prove to be the dominant, factor influencing their health as a species.
January 6, 2012 at 18:43
Tom: That sounds reasonable. Right? but…
I find it weird that you don’t want to minimize the impact of AGW, but your framing in your last sentence seeks to do just that.
How reasonable would…I just want to make sure we realize it’s not the only, but still may prove to be the dominant, factor influencing their health as a species…be? It is not really any different in content to your statement but leaves a different image in the readers mind, wouldn’t you say?
I would say not as reasonable as removing the framing altogether…I just want to make sure we realize it’s not the only factor influencing their health as a species. But leaves us with a fairly trivial statement of the obvious. Oh, what to do?!
January 6, 2012 at 19:44
Hi jr,
I personally view the effects of climate change on endangered/threatened species as a new and unwelcome additional stressor. I think it may well prove to be the straw that broke the camel’s back for some unlucky species.
I also think it pales in comparison to other stressors, and will for at least the next 30 years. We had a long back and forth on a thread about this either last year or the year before, and as usual I got called a denier for not classifying AGW as primus inter pares as a stressor.
The simple fact about polar bears is that we shoot 1,000 of them a year. Out of an estimated total population of 25,000. That’s 4% per year that we’re willing to sacrifice to human priorities. Get that out of the way and I think polar bears thrive, despite reduced ice and its effects on their food supply, which is very real.
January 6, 2012 at 23:25
Here’s a little survey.
1) Is there a benefit in greatly reducing our use of fossil fuels over the next 35 – 50 years. If no, go to question 5. If yes, continue.
2) Do you favor government imposed carbon pricing? If no, go to question 4. If yes, continue.
3) Are you willing to pay at least some of the cost of such pricing? If no, go to question 5. If yes, continue.
4) Can individuals, groups and institutions influence the market to accomplish energy transformation?
5) Thank you for taking this survey.
January 6, 2012 at 23:47
1. Yes.
2. Yes.
3. Yes.
4. Yes.
5. You’re welcome.
January 7, 2012 at 03:32
The Structure of Scientific Opinion on Climate Change, Farnsworth (2011). International journal of Public Opinion Research.
According to this one, 84% of the 489 respondents agree that AGW is happening, and that’s a GMU study with a higher figure than Oreskes’ paper.
Then there’s the STATS 2007 survey by Harris Interactive, also conducted at GMU, found at this useful summary page, which has similar results but apparently also this surprising (even to me) bit:
January 7, 2012 at 03:45
Journalist’s Resource has a piece on the Farnsworth (2011) paper. It concludes with a salient point which I think addresses much of Tom’s view of the disagreements he believes to be very important…
January 7, 2012 at 11:35
J Bowers, Thanks for the refs. The quote from your last comment hits the nail imho: At a meeting with climate sceptics as well as from internet interactions, I gather that those who are critical of the consensus view view consensus as meaning unanimity, whereas most others would view it as a broad, general agreement.
January 7, 2012 at 17:39
JBowers, I think that that figure is probably pretty close to the truth. I’ll have a look at that paper when I look at the other surveys.
I don’t think that there is actually much confusion on the term consensus, though…
January 7, 2012 at 19:30
Tom,
I can tell you first hand that a sceptic audience of around 100 was pretty unanimous in equating consensus with unanimity.
January 7, 2012 at 20:23
Might be true of skeptics, Bart, but remember skeptics are a distinct minority.
It might also be true that the audience was responding to the supposition that consensus, once achieved after considering all views, tends to speak as though it were one voice.
January 7, 2012 at 23:03
That’s twice in a week you’ve almost made me fall off my chair Tom. Must buy a cycling helmet ;)
January 8, 2012 at 01:19
Well, JBowers, if more people wrote that both public and scientific opinion broadly supported the thesis in question, and that support varied around the three-quarter mark, a lot of this back and forth could be directed to more interesting topics.
January 8, 2012 at 01:33
Tom,
I think there is much confusion on the term consensus. There’s certainly a disparity among those who purport to say what the consensus is. There are multiple meanings right here. To Bart’s skeptics it means unanimity. To me it’s whatever stoat says it is. To you it often means the IPCC tribe.
January 8, 2012 at 02:47
I actually think there’s more than one common usage of the term in climate discussions. The one I try and fight against is the one with the capital C–the Consensus says…
January 8, 2012 at 08:21
Let’s try it:
Scientific opinion broadly supported the thesis in question, and that support varied around the three-quarter mark.
January 9, 2012 at 00:20
See how easy that was?
January 12, 2012 at 22:11
Bart,
OT … I would have liked to join the discussion on your walking the talk post at planet3.0 but am so far unable to get out of moderation there. My comment, in part, says:
“Fuller’s third criterion is more about dissonance than hypocrisy. All of the criteria apply to communication in general. Stand up comics, who Fuller saw a lot of as a comedy club manager, build their acts using them. The criteria are critical for achieving the audience’s willful suspension of disbelief, the essential element of successful communication.”
What I’d like readers of planet3.0 to understand is that if a communicator or his message fail to meet these criteria, it does not mean that he or his message are wrong, or evil, or hypocritical or unworthy. It simply means the communication will be less effective.
January 13, 2012 at 00:00
Utility rather than intent… what a concept.
January 13, 2012 at 00:09
Paul,
I just saw and approved your comment over at P3.
January 13, 2012 at 00:23
Bart,
Thanks. I’m going to put the last paragraph of my above comment over there now.
January 13, 2012 at 02:32
I doubt if anyone’s overly concerned, but I just would like to apologize to anyone who’s waiting on my humble evaluation of the surveys. Life and work have gotten in the way. Maybe over the long weekend.
January 13, 2012 at 09:16
Great, Tom, I finally hope to see you explain how to interpret the word “adequate”, why it should be interpreted as such, and why I then should be worried about the answer.
Others may read the Von Storch & Bray survey again, and note that at the end they printed comments from those responding. One of those comments refers to the word “adequate”, too. Funnily enough I had not seen that comment until quite recently, well after I asked Tom Fuller to explain.
January 14, 2012 at 04:17
> pales in comparison to other stressors,
> and will for at least the next 30 years.
During the next 30 years we could avert most of the worst effects of increased fossil CO2, which will be stressors for the following
a) 30 years
b) century
c) millenium
d) all of the above
January 15, 2012 at 21:12
Reposted from other thread:
Okay, here goes. Don’t know if I will make it all the way through a meta-analysis, but I can at least start.
First: Marco, it is typical in surveys to permit a certain ambiguity for certain words in a question, as the word is expected to mean different things to different people. So, where you (quite correctly) see that adequate can mean different things to scientists working in the same or different fields, to the surveyor that is a feature, not a bug.
Remember that von Storch et al surveyed scientists working in many different countries, including a majority where English is not presumed to be the mother tongue. Remember also that the specialties cited by respondents varied widely, from physics of the climate system to impacts of climate change to policy analysis and more.
Each of these areas of focus should have a different definition of adequacy regarding data quality and availability. Using a general term such as adequacy allows them all to have their own interpretation in mind when selecting an answer. The broad agreement shown shows that, whatever definition of adequacy is attributed to respondents (and we understand that there is more than one), the proposition fails.
However, the analysis of von Storch et al would have greatly benefited from the use of cross tabulations, where different categories of respondents were analyzed separately. I would actually volunteer to do this analysis if offered the data.
While the survey looks competent and the description of both sampling and response rates is ‘adequate’ (sorry), it appears to me that the analysis of results was a bit superficial. Not that I disagree with the findings reported–indeed, the Pielke/Jepson/Annan survey more or less reinforces the findings of von Storch et al. It’s just that I don’t think von Storch et al took sufficient advantage of the data they collected.
Anybody here know von Storch? I will go to him and ask for the data if someone is willing to introduce me to him. And I’ll do the analysis here on Bart’s blog so you can all follow and correct any errors I make.
January 15, 2012 at 22:15
This comment is reposted from the previous thread and probably precedes Marcos’ comment which prompted this:
Marco, you’re kind of missing the point. What the survey is not testing is the adequacy of the data. What the survey is testing is the opinions of various stakeholders in the community about the data. Their own definition of adequacy is surely sufficient, isn’t it?
I’m going back to the other thread.
January 16, 2012 at 07:59
Ah, but Tom, their own definition may be “adequate” for them, but we, as users of the survey, don’t know how they interpreted the word. And thus you get people getting all excited because scientists consider something “inadequate”, without knowing in what way they consider it “inadequate”. Which means we’re getting excited about something we don’t know.
January 16, 2012 at 09:38
Well, Marco, there are many things we don’t know. But if you’re going to ask a Roger Pielke or Al Gore about the adequacy of data, you’ll get a very different definition of data as well as adequacy than if you ask Steve McIntyre or Keith Briffa. The reason people use ambiguity in this type of question is so they can ask more than one type of person. Forcing them to define it in their own mind and respond is a common surveying technique.
And I don’t know who’s excited besides yourself, actually. I guess it never happens in climate science–strangers to the field who walk in and are shocked and surprised by a word or phrase and don’t really know how it’s use in common practice by professionals… nothing like a ‘trick’ for example…
January 16, 2012 at 12:54
Tom
“The reason people use ambiguity in this type of question is so they can ask more than one type of person.”
Just one point, is that they were asking Climate scientists, so I am not sure this applies.
Also the principle disagreement wasn’t anything to do with the survey technique, but rather your use of the survey data.
January 16, 2012 at 18:26
Umm, Nathan, you’ll have to clarify. The survey was sent out to a wide variety of people in differing fields ranging from modellers to policy administrators.
As for your second point, you’ll really have to refresh my memory. My reply to Marco was in regards to his complaining about the term ‘adequate’. He thought it was ambiguous and that it could mean different things to different people–I imagine his greatest concern is that those not on board with the consensus view would seize upon the finding and trumpet the supposed ‘lack of adequacy’ for data as a talking point.
As the survey was conducted in 2008, I don’t really think that should be a major concern, but you never know. By me bringing it up now, it might revive the issue, so Marcos’ concern may not be displaced.
January 16, 2012 at 18:32
Tom, asking an ambiguous question results in ambiguous answers. Your claim that it is a common surveying technique is simply not true. In fact, all basic textbooks discuss the problem of asking ambiguous questions and using ambiguous words. All you get out is how people “feel”, which has very little use in surveys.
I would also like to point out, for the umptieth time, that YOU claimed *I* should be worried that climate scientists considered something not adequate enough. That is, YOU got excited about the answer. But, and to me no longer oddly, you continuously refused to explain to me how you interpreted “adequate”, why you believed that that was the proper interpretation of the word, and why that then meant I should be worried. You could just have answered the first and last part, as you now seem to admit that you may well be interpreting the meaning of the word in a different way than those surveyed.
January 16, 2012 at 18:56
Marco, I don’t know how to say you’re incorrect in a delicate manner–but you are. In telephone surveys and qualitative research, when someone asks for a definition of a term, they are given the verbal reply, “What does it mean to you? Just use your best definition.” It is completely normal to leave the definition to the eye of the beholder.
You should be concerned at the large percentages of the climate community (not just scientists) who think, feel or believe that the data is not adequate using whatever definition they have in their mind when they respond to the question. But that doesn’t mean you have to get so excited that you have to plant a stake in the ground and defend it until death. I’m not excited. I’m sitting in front of my computer drinking a cup of coffee. My pulse is normal and my heart rate is slow (well, the coffee is helping with that).
The fact that a policy advisor thinks the data she has available for her meetings with government policy makers is not adequate should be just as worrying as the fact that a modeller feels the same way about her data. It doesn’t matter very much that they’re considering different data sets. It doesn’t matter very much that they have different definitions of adequate. (It is a call for further research within each group to drill down on these issues–this survey calls it to our attention, which is its function–later work can clarify a lot of these issues, but first you have to identify the issues, Marco. Believe me, this is how it works.)
It should matter to all of us that across the entire spectrum of the climate community there are apparently serious concerns about data quality and adequacy.
But it’s no reason to get excited.
January 16, 2012 at 22:02
Tom, I have little intention to believe you. Too many instances where a check of your claims don’t hold up.
No surprise, therefore, that the professional surveyor I know calls your claim BS. And he didn’t mean “Bad Science”. Whenever a survey makes people ask for a definition, it either means the word used is too difficult (one of the rules for surveys: use words that the target population will understand), or it is ambiguous (another rule for surveys: the questions should not be ambiguous). You can read tons of literature discussing that.
Yes, you can then ask follow-up questions. To some extent Von Storch & Bray did that. I also pointed you to those questions. But rather than a response to that, you just repeated the mantra “you should be concerned”.
Why, Tom? Qualify and quantify why I should be concerned. I don’t respond well to the handwave approach.
January 16, 2012 at 23:32
Marco, citation please on claims not holding up?
As for your professional surveyor, name? Quote? Citation of previous work in this area? How did you phrase your communication with him/her? Talk about handwaving…
If he in fact said what you wrote here, he is quite literally incorrect on the assertion you post–that the word is too difficult. Tell me on what planet adequate is a difficult word? It’s not difficulty, it’s ambiguity. Which works in the surveyor’s favor, as it can legitimately mean different things to people in different lines of work. If he does not know that or disputes it, I will have trouble believing he has extensive experience in this field.
I have little intention of believing you, as you rush off with your claims and paranoid fantasies about anyone who doesn’t agree with you, and the quality of your English descends dramatically.
I’ll revise and extend my remarks about you being concerned. Thinking people should be concerned. The rabid reaction force can choose whether or not they fit that definition–although many have already formed an adequate opinion on the subject.
January 17, 2012 at 00:39
Tom, the term ‘adequate’ is not just ambiguous, it can be interpreted to mean almost opposite things.
If one person says the data is not adequate to draw any meaningful conclusions about climate change and another says the data is not adequate to give us the detailed regional analysis needed given the imminent threat we face from climate change then these people hold fundamentally different positions.
The question is all but worthless without any indication of what the data should be adequate for.
Indeed, given that that particular section of the survey starts with “In this section we would like to determine if there are areas in climate science that you perceive to be especially in need of increased research support and/or efforts” I would say that the second interpretation of ‘adequate’ is the more natural one.
January 17, 2012 at 00:58
Tom,
In the examples you gave earlier, it was you who decided what was meant by each ambiguous term; for example ‘alarmism’. You went on to claim that the surveyed group used the same definition as yourself and used that to further your claim.
This completely contradicts your most recent claim that ambiguous terms allow the survey respondent to decide what it means.
When you gave your analysis earlier you completely failed to allow the ‘variety’ of interpretations shine though.
“I imagine his greatest concern is that those not on board with the consensus view would seize upon the finding and trumpet the supposed ‘lack of adequacy’ for data as a talking point.”
This is exactly what you did with the response to the ‘alarmism’ claim. Nowhere did you give space for the variety of interpretations over the word ‘alarmism’. You never explored it. You just used your personal definition and assigned it to the respondents.
If you have done this kind of research for the length of time you have then I am thinking whomever paid you got a bad deal, as they simply would have been given your opinion and no understanding of what the respondents were actually saying,
January 17, 2012 at 00:58
A survey of this thread indicates those who were not already predisposed to believing Tom Fuller’s claims do not consider his argument adequate.
January 17, 2012 at 01:04
sharper00 it would be more accurate to say that those who had already condemned Fuller for not kowtowing to the preachings of the Elect will find no reason to abandon their rigid prejudices.
January 17, 2012 at 01:12
Question 1:
Who here thinks that Tom Fuller is showing an adequate understanding of this survey.
My Answer: No.
January 17, 2012 at 01:14
Tom,
“sharper00 it would be more accurate to say that those who had already condemned Fuller for not kowtowing to the preachings of the Elect will find no reason to abandon their rigid prejudices.”
This is remarkably paranoid. Why would our understanding of how you assess and use survey data have anything to do with ‘preachings of Elect’…
Someonethinks they’re a Martyr, don’t they Mr Fuller.
January 17, 2012 at 01:39
Martyr? How quickly the story changes.
Nathan 27/July/2010: Some things never change.”
“yeah, as usual we see Tom Fuller contradicting himself saying on this thread that he was once a skeptic, but then came round, but in other blogs we read that the sequence was reversed.”
Nathan 28/July/2010: lukewarmer over time
“I guess I doubted your position as I have never seen you blog that you doubted the basic science. In that case I’ll just have to take your word on it.”
So am I a skeptic, a lukewarmer or a martyr? How much time do you want to spend classifying me as opposed to trying to make sense out of the issue we are discussing?
You seem to be making an argument that you don’t understand.
January 17, 2012 at 01:59
Tom
You have taken quotes from 18 months ago…
Did you go googling for that?
You are hilarious! The story changed over 18 months, WOW that was quick!
I don’t even know if they’re my quotes.
AND as usual you haven’t adressed the substance of my claim.
January 17, 2012 at 02:57
I’m waiting for any claim of substance, Nathan. And who knows, maybe it wasn’t you who said that–maybe it was a Nathan impersonator…
Make a claim.
January 17, 2012 at 08:42
Tom, I asked him in person (he’s a friend), and I won’t give you his name or publications. He already has his own problems to deal with (patients can be pretty unhappy people if they don’t think they are treated properly, and unfortunately let the surveyor know in unhappy ways), he doesn’t need any denidiots going after him.
Also, you may want to learn how to read. I offered TWO possible reasons for people asking for a definition: the word is too difficult, option one, or the word is ambiguous, option two. Somehow you only saw option one and went into a state of frenzy. There, also proof that yet another claim of you did not hold water (I also remember a certain thread in which you claimed scientists deliberately misrepresented Arrhenius, where you, despite repeated requests from me, decided never to provide any proof of your claim; not surprising, because there isn’t any such proof).
What you are defending is one of the well-known examples of a bad question in surveys:
“How regularly do you go to the movies, never, sometimes, regularly, or often?”; with the exception of “never”, the options only survey somebody’s feelings, not how often they actually go to the movies, and as such the answer is useless.
January 17, 2012 at 11:22
Inadequate for a scientist means that there is more stuff to be done, not that the current knowledge is useless for practical purposes. If the previous state of knowledge was adequate there is little point in studying it further. All you get is a paper in a third rate journal.
S and vB were asking whether further work was needed, not that the previous studies were no good. Of course further work is needed.
Marco is right, Tom is wrong. Tautologies are simple.
January 17, 2012 at 14:26
I just spoke to my friend again, and funnily enough he had a good example of using the word “adequate” in a survey he had done, and the potential problems:
“Did you consider the number of daily visits by the nurses adequate?”
Answer was mostly no. What would the obvious solution be? Get the nurses to visit more often! Right? Wrong!
They also asked the patients to indicate how many times on a daily basis they noted the nurses visiting their room. Perhaps not surprisingly, that number was lower than reality. Through very simple means, which did not include increasing the number of visits, but rather make the nurses more ‘visible’ during their visits, they significantly improved patient satisfaction!
January 17, 2012 at 14:42
Tom,
it seems you just ignore what people write.
Here it is again:
“In the examples you gave earlier, it was you who decided what was meant by each ambiguous term; for example ‘alarmism’. You went on to claim that the surveyed group used the same definition as yourself and used that to further your claim.
This completely contradicts your most recent claim that ambiguous terms allow the survey respondent to decide what it means.
When you gave your analysis earlier you completely failed to allow the ‘variety’ of interpretations shine though.
“I imagine his greatest concern is that those not on board with the consensus view would seize upon the finding and trumpet the supposed ‘lack of adequacy’ for data as a talking point.”
This is exactly what you did with the response to the ‘alarmism’ claim. Nowhere did you give space for the variety of interpretations over the word ‘alarmism’. You never explored it. You just used your personal definition and assigned it to the respondents.
If you have done this kind of research for the length of time you have then I am thinking whomever paid you got a bad deal, as they simply would have been given your opinion and no understanding of what the respondents were actually saying,”
Sorry to everyone else.
January 17, 2012 at 14:46
Tom,
I am sure everyone here would think it was odd that you quote to posts from 2010 as ‘evidence’ of me changing my opinion ‘quickly’.
Also note, that calling you a martyr (which actually was tongue in cheek) is not contradicted by me calling you a skeptic. in fact the two are completely unrelated.
However, given that we discussed this about 2 weeks ago I would think you would be well aware that I know you are a ‘Lukewarmer’ – a delightfully meaningless marketing term. So the rationale for you quoting those 2010 posts is a mystery to me.
January 17, 2012 at 19:01
Hi everybody,
Well it’s the start of a busy work-week so this’ll be my last comment before next weekend. In any event, with the arrival of Rabett on the scene the chance of a meaningful discussion dwindles to approximately zero, so this may be a convenient stopping point.
On the substantive point in question, I repeat–surveys usually do not define adjectives or descriptors. If they are important they quantify them (On a scale from 1 to 5, where 1 means not at all adequate and 5 means completely adequate…). But they don’t define them because there’s no end to it and no way for survey respondents to act on the definitions in a survey of appropriate length. When Coca Cola does a customer satisfaction survey they don’t define satisfaction. When Gallup does a poll on voting intention they don’t define likely.
The respondents’ individual perceptions of what a term means is taken as adequate, because it is assumed that with a variety of responses the definition converges around a central meaning that holds for the group at large.
You may disagree with this. But it’s common usage.
Marco, your memory of our last conversation about Arrhenius is as flawed as everything else that comes off your keyboard when you engage with me. It’s almost clinical.
The fact that a broad spectrum of those actively working in the climate change arena believe that data quality and quantity is not adequate should spur further investigation. Your collective attacks on the survey that brought it to your attention is absolutely typical of the warmist response to inconvenient information over the past few years.
I, like most critics of the consensus, long ago abandoned any idea that you would engage on a topic in good faith, and this is another example. A bunch of dudes who show up on a blog who don’t know about market research want to rewrite the textbooks and invent new rules of the road for a research field over a century old. Cool. You dudes are the ones who keep trying to popularize the term Dunning Kruger–look in a mirror.
You are willing to engage in any amount of bizarre contortions to avoid facing disturbing facts. That does nothing to improve the adequacy of the data. It does nothing to change the perceptions of those working in the field. It does nothing to advance the discussions regarding climate change.
One of Halpern’s favorite whines is that I manage to turn every thread into a discussion about me. On this, as with everything else he pontificates about, he is mistaken. It is you who tries to turn every thread into a discussion about me. And it is because you cannot seem to deal with the arguments I advance.
The sad thing is everybody sees it except you.
January 17, 2012 at 22:16
Tom, that you have a thing about Eli is your problem. Your nonsense only works at Kloor’s, and even there you have worn out your welcome Declare victory and sulk off but you are still, as Marco and Nathan point out, talking through your hat. For example, on questions of belief, a five point scale is the gold standard, not yes/no, which in most cases is simply card forcing.
Now Eli, of course, is an absolute expert on surveys. Don’t believe him? Why he just told you so, just like Tom Fuller did. What, you don’t believe Tom. For shame. Everyone knows that Tom knows Tom knows it all.
January 17, 2012 at 22:52
Yer jes Joshin’.
January 17, 2012 at 23:07
Go back to work Tom. You promised us.
January 17, 2012 at 23:15
“sharper00 it would be more accurate to say that those who had already condemned Fuller for not kowtowing to the preachings of the Elect will find no reason to abandon their rigid prejudices.”
I just think your claims have a fairly crummy support. I guess on the topic of topic of Tom Fuller I’m lukewarm while you’re an alarmist.
I find it sort of odd that Marco and others have criticised the survey repeatedly for it’s vague and ambiguous results as well as their contrast with your highly specific claims yet in when you (eventually) got around to providing some type of explanation you effectively supported their view by pointing out that ambiguous terms are indeed ambiguous and open to the unknown interpretation of the individuals involved.
It would seem like the important issue on the “adequate data” result is the likelihood of shared concerns between an expert in the field and an outsider. Two contrasting examples might be:
#1 To nuclear safety engineers: How do you rate the safety fail safe procedures of active nuclear power plants operating in Eastern Europe today?
#2 To biologists: How do you rate the adequacy of data demonstrating dinosaur to bird development in evolutionary theory?
For #1 odds are that if nuclear safety engineers are concerned about nuclear power plants then I am too. Their opinion on which countries have unsafe plants might effect whom I want to live downwind from.
For #2 I am unlikely to share concerns except as an intellectual exercise in understanding a complex topic. Whether one explanation rules, or an alternative one does has no bearing on me personally or most people in fact. I can also be pretty certain that “adequate” is being used a very specific and professional context such that “inadequate” has meanings other than “useless” or “garbage”.
Now what if someone comes along and tells me survey result #2 shows that evolution is a theory in crisis? Even the experts say the data is inadequate and “We can’t teach this stuff to high school kids, even the scientists themselves say it’s not adequate!”.
Those with a keen eye would spot the context transference from the speciality to the common debate, where obviously it does not belong. It would be very reasonable to say “The survey does not support what you’re saying, you need a more specific one asking if they think it’s adequate enough to teach in high school!”.
Alas the not-at-all-interested-in-personalities claimant has run off, because someone he dislikes posted in proximity to him and because the only possible reason others disagree with him is because of those he disagrees with.
January 18, 2012 at 00:26
I would suggest that for anyone who is genuinely unsure about how much weight can be put on the ambiguous ‘adequate data’ question, looking at the contrast between the distribution of responses to this question (11a – Q13 – on page 22) and, for example, the question “Over the issue of climate change, the general public should be told to be…” (60 – Q109 – on page 76) ought to clarify the issue.
I don’t think it would be possible to honestly conclude that the same people who thought climate data was inadequate to draw strong conclusions would also believe that the public should be told to be quite worried about climate change. Perhaps what they actually believe is that given the urgency of the situation the amount of data collection and analysis we are doing is inadequate.
January 18, 2012 at 00:49
[edit. No namecalling. BV]
January 18, 2012 at 01:12
Tough? You guys are creampuffs. Tough is when people use logical and coherent arguments.
You aren’t tough.
January 18, 2012 at 02:01
Tom, you’re hilarious.
Go back and answer the criticisms… Let’s get some confirmation of your toughness.
January 18, 2012 at 02:20
Sorry dude–read what I wrote. Real busy, won’t have time to spend time on this til this weekend, if I’m lucky. Deal.
January 18, 2012 at 09:10
Tom, I point you again to a thread here at Bart’s
In it, you claim:
“But it has not stopped people who obviously know better from repeating what is a deception.” and
“From deliberate distortions of the work of Svante Arrhenius to red buttons blowing up skeptics, the Consensus Team has landed itself where it is today”
I repeatedly asked you to provide evidence for those claims. All of your arguments were handwaving, appeals to authority (in the logical fallacy sense of the meaning), contradictions (the 1906 paper was, according to you, both “well-established and often-cited” and “very rarely mentioned”), and hypocritical. Regarding the latter: you claimed ignoring the 1906 estimate was evidence of a deliberate distortion, but consistently ignored my question whether you’d ever tell a pseudoskeptic who cited the 1906 paper that it was a deliberate distortion, since his 1908 paper puts it at 4 degrees.
At NO point did you ever establish, or even tried to establish, that Arrhenius’ results were deliberately distorted. Heck, you could not even make a case for Arrhenius’ results being distorted.
January 18, 2012 at 09:25
Regarding the survey, you again make the already contradicted claim that we attacked the survey. As I and others noted, we attacked *your interpretation* of that survey. Of course, you construct that as an attack on the survey, since you apparently consider YOUR interpretation of the survey the only possible interpretation.
Again, I cannot do much with “concern” if I don’t know what exactly the problem is. If I want to open a cinema in a town, and need the population of that town to come to my cinema on average once every two weeks, should I be concerned if a survey finds the people say they “sometimes” want to go to the cinema? I have no idea, since that “sometimes” could be anything from once a week to once a year. If it is once a year, I’d better not even try to open a cinema. If it’s once a week, I’ll make a handsome profit!
And now that I have my cinema, a survey shows they do not find my cinema “adequate”. Darn. What to do? No idea! I have no idea what they do not consider “adequate”. At the very best, this would be a reasonable question for a very exploratory survey. One that shows I need to have another, more in-depth survey done. One that actually can be used.
Finally, your example on the voting likelihood shows how you clearly do not understand basic surveying techniques. “Likely” is NOT an ambiguous word! It means the same for everyone: more likely than not. It allows one to create estimations of voter turn-out. What you will NOT see Gallup ask is “is the information you received adequate?”
What Gallup *may* ask is “Is the amount of information you received about the viewpoints of the candidates adequate to decide which candidate you will vote for?” (well, probably in more appropriate English syntax), and even then they’d likely add a follow-up question, asking to indicate where information is missing.
You’re just focusing on those three-question surveys that are generally of little use.
January 18, 2012 at 10:14
Keep it civil, guys.
No time to keep up in detail at the moment, but some thoughts:
Asking the respondent about ‘adequacy’ is asking them about what feeling they have about the issue. That’s a different question than asking them a quantitative question e.g. like ‘How many datasets blah, blah, blah’.
Both are valid, but very different questions. It depends what you wnat to know. The former gives a qualitative picture about how the respondent perceives something (difficult to interpret though…), whereas the latter asks a quantitative question (which probably appeals more to the physical scientists among us).
January 18, 2012 at 10:49
What would Kevin Trenberth have answered to the question about how adequate climate data is?
http://www.skepticalscience.com/Understanding-Trenberths-travesty.html
January 18, 2012 at 11:59
As Eli said, all data is wrong, some data is useful. Interesting discussions of this with examples @ Rabett Run and Judy’s
January 20, 2012 at 21:29
If anyone would like to see one of the reasons I’ve been too busy to continue this, it’s here: http://3000quads.com/
January 23, 2012 at 18:26
A bit late to the party, I know. The Von Storch survey is interesting with some good information in there. Some questions could have been designed better. And some crosstab analysis would have been interesting, too. In fact, any analysis would have been useful (not the Tom Fuller kind from what I’ve read here though).
Would be interesting to ask industry the adaptation mitigation questions. Wonder if the charts would have been skewed the other way?
In the Brown, Pielke Sr and Annan survey, the survey questions are quite awful IMO. Far too many questions within each question.
I didn’t look at any other survey mentioned.
Thanks for the post, Bart – it’s given me some ideas.
January 27, 2012 at 15:35
Speaking of surveys, here is one by Peter Norvig:
http://norvig.com/oreskes.html
His final reaction:
> My final main reaction to reading these 59 abstracts is that an amazing amount of research went in to building up this consensus on global warming, but I hadn’t heard much about the specifics. This is partly my fault, but is also another failing of the press. Reporters think (with some good reason) that the public is not interested in hearing about Analysis of some direct and indirect methods for estimating root biomass and production of forests at an ecosystem level and so they never cover such things. But by failing to talk about the years of research and the building on the works of others that go into producing a paper like that, reporters give all ideas equal footing: a half-baked whim with no evidence gets equal footing with a proven theory with hundreds of confirming studies, because it is too complicated to talk about the confirming studies.
February 14, 2012 at 21:15
I think surveys are fine as long as the details are presented as to how the sample was acquired, how questions were structureed, margin of error, etc.
I’m a meteorologist and I find value in carefully conducted surverys – of any type.
February 19, 2012 at 13:50
had a good example of using the word “adequate” in a survey he had done, and the potential problems
March 1, 2012 at 03:35
Bart,
I hope your lack if posting is due to being busy with other things and not out of despair of making a difference.
March 1, 2012 at 14:19
That’s right Paul, it’s the former more so than the latter ;-)
March 1, 2012 at 15:00
Glad to hear it. You do make a difference. Participating here has certainly informed and improved my thinking on the issues discussed. I’m sure that is true for others as well.
March 1, 2012 at 17:59
Thanks Paul; that’s nice to hear.
March 2, 2012 at 00:19
Paul +1
March 2, 2012 at 15:28
Thanks Tom!
March 4, 2012 at 19:39
Let’s all start the chant: “What do we want?” “A new post from Bart!” “When do we want it?” “Well, whenever he gets around to it…”
April 20, 2012 at 22:16
Willard, Justin Gillis of the NY Times does a good job on that sort of thing given the space. However, you have to realize that journalists always think they need a personal interest hook for such stories
April 24, 2012 at 07:31
Yet again Fuller directly contradicts himself (aside from all the direct lies).