Robert Grumbine (a scientist-blogger well worth reading) explains how the scientific process works and how scientists communicate, and how it differs from blog debates (which he describes as “having a beer”). The following is lifted from his comment at Chris Colose’s blog a while ago. He sets up his argument in response to another commenter (self-identifying as “a genuine skeptic”):
———————————-
As a skeptic, I share your frustration equally, and as a genuine skeptic, and someone who does care about the environment, I am ready any day of the week to have my opinion sway back to believing in / trusting the consensus / IPCC position. Further, I know exactly what would sway me: dialogue, and constructive debate with the skeptics, in particular those skeptics of the ilk of Lindzen, Christy, and, although he refuses the label “skeptic”, Pielke Sr. (…)
(…) I’ve disagreed with Pielke Sr., for instance, but in the scientific norm. Tenderhearted readers, unaccustomed to the scientific norm, might have thought I was awfully hard on Roger. (One did say so.) But his own comment was that he appreciated my constructive discussion. This is a cultural issue that I think the general population does not understand. Normal exchanges, for science, about what’s going on, what’s good, or not, are fairly rough and tumble. It may not be the best thing that science is conducted this way, but it is what it is.
The scientific norm issue is a different matter. The scientific norm is the professional literature, not blog commentary. If you look in to Lindzen and the response in the scientific literature, you’ll find that he’s been met properly (by the standards of science, that is). Namely, he suggested his ‘adaptive iris’ idea. This was based on there being a certain relationship (it had to have a particular sign, and large magnitude) between surface temperatures in the tropics and cloudiness (and, for that matter, particular types of cloud). One paper in the scientific literature doesn’t buy you much. It is the start of the conversation to publish in the literature, not blessing as holy writ. He published, and then got the best possible response — other people used other (better) data sets and observing methods to see if they could get the same answer as he had gotten in his first cut. Unfortunately for his hypothesis, the better data sets erased his effect. Indeed, not only was the magnitude much smaller than he thought, the sign was the opposite of what he thought.
As far as scientific norms go, he got extremely good treatment. a) he did publish his idea (no ‘conspiracy to suppress’) and b) other people took a serious look at it. It is significant work to take a look at somebody else’s new idea. As a scientist, if you can get others to look at your idea, you have done extremely well. As happens in science, perfectly normally, the initial proposition got rejected by more detailed analysis. Since what was at hand was deriving a relationship between observational quantities, and Lindzen is a theoretician, it’s no great surprise or shame that he didn’t get all the niceties on his data sets right. As usual, devils lay in the details, and the responses were from groups familiar with all the devils laying in those details.
Where things went problematic was that contrary to proper scientific practice, Lindzen didn’t drop his disproven idea. A bit of ‘is so’ publishing (sorry, it was painful to read his response article and this is all I can say of it) in response to the objections was it. And then much complaining outside the scientific literature about conspiracy, scam, censorship, … To be honest, even his original Iris publication was an example of lenient reviewing. There were problems in his data management in the original paper that even I saw (correctly) would be a problem for his idea — and I’m not a tropical person (polar regions mostly), nor, then, sea surface temperature, nor then or now satellite sensing of clouds. The later publications — in the scientific literature — about his errors confirmed my suspicions, and, unsurprisingly, added a number of problems to what I suspected. But that’s not what you see out on the blog universe.
You can make some headway over at scholar.google.com. A fair amount of the non-scientific world shows up there, but a fair amount of the scientific world is present.
…
always see an ad hominem attack for what it is (I refer to the attacks by commenters at RealClimate, which were not removed by the editors). In most cases, straw men arguments can also be seen for what they are. And then an argument, “we don’t have to answer that ’cause it wasn’t peer-reviewed” always also increases the lay public’s skepticism. No one takes that response seriously, and again, skepticism can only increase.
I agree that outside the scientific community nobody takes seriously that something didn’t appear in the scientific literature.
That is a problem with outside the scientific community.
Doing science is difficult. Over the past 400 years, the modern scientific method has accumulated a lot of knowledge and understanding. Doing science means changing that body of knowledge and understanding. Sometimes that means saying that even though we used to think that something was the case, it really isn’t. Making that argument successfully is hard work. ‘Even’ the easier argument of making an addition is hard work. It’s hard work because other people have to be able to rely very strongly on everything you say in your paper. (…)
There are two parts to the scientific publication process important for your comment here. One is, to publish in the professional literature about your idea, you have to examine and explain your idea thoroughly. ‘thoroughly’ turns out to be a lot of work. Second is, you have to research all the relevant aspects of your problem and honestly discuss them. The up side of this is, once you’ve finished a proper scientific paper, it can stand for some time. You might turn out to be wrong about something — because there were different data than you used, or a better technique than you used, or … several things. Being shown wrong by later and much more labor-intensive examinations is fine. But being shown wrong because you failed to do your homework is disaster.
In contrast is blog posts and comments. The standard there is what I’ll call ‘chatting over a beer’. If you and I sit down and start talking, both of us with something we like to drink, having a relaxed conversation, that’s wildly different than scientific literature. Both of us will say what we think, but there’s no concern about tomorrow you trying to write a paper on which our professional reputations will hang based on what I say. We’re just chatting. I’ll give you my best answer at the time, but if I’ve forgotten something, or the answer is 25.0 instead of 2.50, eh. Just chatting over a beer. Just a blog comment. If someone started writing a scientific paper based on comments in blogs … all kinds of wild things could show up. The earth is flat, hollow, expanding, 6000 years old, and so on. Somebody, somewhere, in the blogosphere has said all such things.
To do science, we need something much more reliable than ‘anything anybody ever says anywhere’. We even need something better than “well, he’s normally pretty good so even tough he’s never worked on this kind of problem before and doesn’t know how the satellites detect what he’s working with, he _must_ be right anyhow.” That something more is the professional scientific literature.
Within the world of science (all 20 or so of us), it is an extremely telling, and negative, thing that much of what the general public thinks is the case about science is actually based on things which are said _only_ outside the scientific literature. If the speaker had confidence in his statement, he’d try to publish it in the literature. And, if they were right about the ‘conspiracy’, they should at least have a rejection letter and comments from the editor and reviewers to show. Instead, they talk about the conspiracy, but have no rejection letters (_I’ve_ got rejection letters — they’re normal to trying to do science.)
But the public perception is quite different. Still, I have to think if someone won’t go in front of his professional peers and stand for what he thinks is scientifically correct, he doesn’t really believe it himself. If his only or major audience is people who don’t know the science thoroughly, I have to figure he thinks that’s the only audience who’ll let him get away with whatever it is he’s saying now. Fine for you and me over a beer. No fine for doing science.
——————————–
Just to add a qualifier: Not everything that appeared in the scientific literature is necessarily “good science”. Likewise, not everything that appeared on a blog is necessarily unscientific or wrong. Just as in a bar, the greatest ideas and insights can be heard. But amidst a helluva lot of chatter about the weather. Which is actually pretty nice right now. Enjoy your beer!