Richard Tol misrepresents consensus studies in order to falsely paint John Cook’s 97% as an outlier

by

John Cook warned me: if you attempt to quantify the level of scientific consensus on climate change, you will be fiercely criticized. Most of the counterarguments don’t stand up to scrutiny however. And so it happened.

The latest in this saga is a comment that Richard Tol submitted to ERL, as a response to John Cook’s study in which they found 97% agreement in the scientific literature that global warming is human caused. Tol tries to paint Cook’s 97% as an outlier, but in doing so misrepresents many other studies, including the survey that I undertook with colleagues in 2012. In his comment and his blogpost he shows the following graph:

Richard Tol misrepresenting existing consensus estimates

Richard Tol comes to very different conclusions regarding the level of scientific consensus than the authors of the respective articles themselves (Oreskes, 2004; Anderegg et al., 2010; Doran and Kendall Zimmerman, 2009; Stenhouse et al., 2013; Verheggen et al., 2014). On the one hand, he is using what he calls “complete sample” results, which in many cases are close to meaningless as an estimate of the actual level of agreement in the relevant scientific community (that counts most strongly for Oreskes and Anderegg et al). On the other hand he is using “subsample” results, which in some cases are even more meaningless (the most egregious example of which is the subsample of outspoken contrarians in Verheggen et al).

The type of reanalysis Tol has done, if applied to e.g. evolution, would look somewhat like this:

  • Of all evolutionary biology papers in the sample 75% explicitly or implicitly accept the consensus view on evolution. 25% did not take positon on whether evolution is accepted or not. None rejected evolution. Tol would conclude from this that the consensus on evolution is 75%. This number could easily be brought down to 0.5% if you sample all biology papers and count those that take an affirmative position in evolution as a fraction of the whole. This is analogous to how Tol misrepresented Oreskes (2004).
  • Let’s ask biologists what they think of evolution, but to get an idea of dissenting views let’s also ask some prominent creationists, e.g. from the Discovery Institute. Never mind that half of them aren’t actually biologists. Surprise, surprise, the level of agreement with evolution in this latter group is very low (the real surprise is that it’s not zero). Now let’s pretend that this is somehow representative of the scientific consensus on evolution, alongside subsamples of actual evolutionary biologists. That would be analogous to how Tol misrepresented the “unconvinced” subsample of Verheggen et al (2014).

Collin Maessen provide an detailed take-down of Richard Tol on his blog, quoting extensively from the scientists whose work was misrepresented by Tol (myself included). The only surveys which are not misrepresented are those by Bray and von Storch (2007; 2010). This is how I am quoted at Collin’s blog RealSkeptic:

Tol selectively quotes results from our survey. We provided results for different subsamples, based on different questions, and based on different types of calculating the level of agreement, in the Supporting Information with our article in ES&T. Because we cast a very wide net with our survey, we argued in our paper that subgroups based on a proxy for expertise (the number of climate related peer reviewed publications) provide the best estimate of the level of scientific consensus. Tol on the other hand presents all subsamples as representative of the scientific consensus, including those respondents who were tagged as “unconvinced”. This group consists to a large extent of signatories of public statements disapproving of mainstream climate science, many of whom are not publishing scientists. For example, some Heartland Institute staffers were also included. It is actually surprising that the level of consensus in this group is larger than 0%. To claim, as Richard Tol does, that the outcome for this subsample is somehow representative of the scientific consensus is entirely nonsensical.

Another issue is that Richard Tol bases the numbers he uses on just one of the two survey questions about the causes of recent climate change, i.e. a form of cherry picking. Moreover, we quantified the consensus as a fraction of those who actually answered the question by providing an estimate of the human greenhouse gas contribution. Tol on the other hand quantifies the consensus as a fraction of all those who were asked the question, including those who didn’t provide such an estimate. We provided a detailed argument for our interpretation in both the ES&T paper and in a recent blogpost.

Tol’s line of reasoning here is similar to his misrepresentation of Oreskes’ results, by taking the number of acceptance papers not just as a fraction of papers that take position, but rather as a fraction of all papers, including those that take no position on current anthropogenic climate change. Obviously, the latter should be excluded from the ratio, unless one is interested in producing an artificially low, but meaningless number.

Some quotes from the other scientists:

Oreskes:

Obviously he is taking the 75% number below and misusing it. The point, which the original article made clear, is that we found no scientific dissent in the published literature.

Anderegg:

This is by no means a correct or valid interpretation of our results.

Neil Stenhouse:

Tol’s description omits information in a way that seems designed to suggest—inaccurately—that the consensus among relevant experts is low.

Doran:

To pull out a few of the less expert groups and give them the same weight as our most expert group is a completely irresponsible use of our data.

You can read their complete quotes at RealSkeptic.

See also this storify of my twitter discussion with Richard Tol.

Tags: , , , , , , , , , , , ,

35 Responses to “Richard Tol misrepresents consensus studies in order to falsely paint John Cook’s 97% as an outlier”

  1. Collin Maessen Says:

    Tol showed up on Real Skeptic and repeated the same accusations. In his comment he strongly implies that you manipulated data to get the answers you wanted:

    Or take Verheggen’s study. In the sample selection phase, they decided that these are the people they want to ask. After having seen the answers, they decided that some of their interviewees are not worth listening to.

    That’s one hell of an accusation to make. Especially when he leaves out the detail that you were selection on the basis of expertise. Something you again explained in the above post…

  2. Willard Says:

    Wrong one, sorry:

  3. Marco Says:

    As I also pointed out at Collin’s place, in his supposed submitted comment he included for Oreskes the “no position” (consensus = 75%), for Cook et al he removed the “no position” (consensus = 97%). Had he treated Oreskes like he did Cook et al, it would be 100 vs 97%. Suddenly he has two supposed ‘outliers’.

    So it is quite rich that he accuses Bart of data manipulation to get the answers he wanted.

  4. thomaswfuller2 Says:

    Bart, I can’t any longer say I’m surprised by this kind of writing from you, but I am saddened. You are incorrect in your description of your own survey. You are incorrect in your description of others as well.

    Anderegg, Prall et al. is junk. Spencer Weart was correct when he rubbished the paper the day it came out.

    They miscounted papers, got scientists’ specialties wrong–they miscounted the number of papers published by one of the paper’s co-authors. They searched only in English and used Google Scholar.

    Worse, they cherry-picked victims of their attack from the documents used as referrers. They got 46 names from a paper called the Science and Environmental Policy Project.

    There are 4,000 signatories to the 1992 statement from the Science and Environmental Policy Project. Why did Anderegg, Prall et al limit themselves to just 46 names from this list? Is it perhaps because they didn’t want to include any of the 73 Nobel Prize winners in their list of ‘deniers?’ James Prall runs a weblog that has pictures of those he has labeled climate deniers. For some reason I can’t find pictures of many of those who signed the 1992 statement, including Jonas Salk, Rita Levi-Montalcino, Elie Wiesel, Norman Borlaug, Linus Pauling, etc.

    RomanM demolished their use of poor math in their analysis in a post over at Only In It For The Gold which I can’t find.

    I focused on the breach of ethics in social research caused by using the meta tag ‘denier’ on their paper and linking from their paper to a website that published the names and even pictures of the respondents.

    And, as with all the Konsensus literature searches, a mystifying inability to come up with numerous skeptic papers published during the period researched. Hundreds of them actually.

    Sample bias.
    Cherry picking respondents.
    Violations of research ethics.
    Literally hundreds of factual errors.
    Wrong math.

    Junk science.

  5. thomaswfuller2 Says:

    The last paragraph is mistakenly pasted in from another rant. Apologies. This is not meant for the Prall train wreck:

    “And, as with all the Konsensus literature searches, a mystifying inability to come up with numerous skeptic papers published during the period researched. Hundreds of them actually.”

    That’s meant for Oreskes.

  6. Marco Says:

    I see Tom Fuller demands Tol does not include Anderegg et al. Oh wait, he complains Bart points out Tol misrepresented Anderegg et al. No surprise there.

    Of course, Tom Fuller himself misrepresents the statement that many Nobel prize winners signed (The so-called Heidelberg Appeal). The latter does not in any way challenge climate science. It is perhaps therefore no surprise that more than half of the Nobel prize winners who signed the Heidelberg Appeal, also were signatories to the UCS’ “Warning to humanity”, which was a lot more explicit on ecological challenges.

    This may explain why you can see people like Tom Fuller point out Linus Pauling as a signatory to the Heidelberg Appeal, while ignoring what he wrote in 1991:

    (you’d almost think he promotes alternative energy sources in that 1991 letter…)

  7. thomaswfuller2 Says:

    Marco, Prall et al culled 46 of the people they labeled deniers from signatories to this letter.Why? Why only 46? Why not label Norman Borlaug a denier?

  8. Willard Says:

    > You are incorrect in your description of others as well.

    Which description? BartV simply shows that RichardT’s argument is invalid. There’s no need to describe others’ research to do that.

    We now return to GW’s “but Anderegg” squirrel program.

  9. Willard Says:

    Oh, and interested readers might not know how the GW does Anderegg ClimateBall episode was left the last time:

    Tom, first of all they are called UE, as in “unconvinced experts”. Which actually takes away some of the worst of the worst in several of the sources Anderegg et al used.

    Second, please provide evidence that they “won’t get jobs or funding because they are on that list”. This paranoia is completely out of whack with reality. Lindzen still gets funding. Bob Carter seems to have no problems getting money. It’s not like these two are known to be cheerleaders for the IPCC…

    Not also that all of them *put themselves on a list*. If they don’t want to be on a list that indicates they are ‘skeptical’ of the IPCC conclusions, DON’T SIGN UP FOR A LIST!

    http://initforthegold.blogspot.com/2010/06/fullers-complaints-w-anderegg-prall-et.html?showComment=1278135095204#c8211144474666574295

  10. Bart Verheggen Says:

    This post is about how Tol (mis)represented several consensus studies; the sort of criticisms of one of those studies that Tom brought up and that others responded to is not relevant to Tol’s portrayal of those consensus studies and thus not relevant to this post.

    Please abide by the comment policy: remain on topic and refrain from accusations and derogatory language.

  11. Marco Says:

    Tom, I checked, and Anderegg et al state they used the SEPP 1992 statement, NOT the Heidelberg Appeal. There were 47 signatories to the SEPP statement, and all of them were included in the analysis.

    It is the following list:
    http://www.sourcewatch.org/index.php/SEPP_and_the_Statement_by_Atmospheric_Scientists_on_Greenhouse_Warming
    (the direct link no longer seems to work).

    Conclusion: you falsely accused Anderegg et al.

  12. thomaswfuller2 Says:

    Marco, where did you check? Marvel Comics? Oh–Sourcewatch.

    Try checking the paper:

    http://www.pnas.org/content/suppl/2010/06/07/1003187107.DCSupplemental/pnas.201003187SI.pdf#nameddest=STXT

    The paper states, “We define UE researchers as those who have signed reputable statements strongly dissenting from the views of the IPCC. We compiled UE names comprehensively from the following 12 lists:

    1992 statement from the Science and Environmental Policy
    Project (46 names), 1995 Leipzig Declaration (80 names), 2002 letter to Canadian Prime Minister Jean Chretien (30 names), 2003 letter to Canadian Prime Minister Paul Martin (46 names), 2006 letter to Canadian Prime Minister Stephen Harper (61 names), 2007 letter to U.N. Secretary General Ban Ki-Moon (100 names), 2007 TV film The Great Global Warming Swindle interviewees (17 names), NIPCC: 2008 Heartland Institute document “Nature, Not Human Activity, Rules the Climate,” ed. S. Fred Singer (24 listed contributors), 2008 Manhattan Declaration from a conference in New York City (206 names listed as qualified experts), 2009 newspaper ad by the Cato Institute challenging President Obama’s stance on climate change (115 signers), 2009 Heartland Institute document “Climate Change Reconsidered: 2009 Report of the Nongovernmental Panel on Climate Change (NIPCC)” (36 authors), and 2009 letter to the American Physical Society (61 names).

    1. There are 4,000 signatories to the 1992 statement from the Science and Environmental Policy Project. Why did Anderegg, Prall et al limit themselves to just 46 names from this list? Is it perhaps because they didn’t want to include any of the 73 Nobel Prize winners in their list of ‘deniers?’ James Prall runs a weblog that has pictures of those he has labeled climate deniers. For some reason I can’t find pictures of many of those who signed the 1992 statement, including Jonas Salk, Rita Levi-Montalcino, Elie Wiesel, Norman Borlaug, Linus Pauling, etc.

    But what were 73 Nobel Prize winners doing signing a ‘denialist’ letter? Were they fooled, scammed, drugged into submission?

  13. thomaswfuller2 Says:

    willard, the paper was tagged ‘deniers.’

  14. thomaswfuller2 Says:

    Bart, what if Tol is right? I’m not saying he is–I haven’t read his comment.

    But I will say that the famous literature reviews–Oreskes ‘Beyond the Ivory Tower’, Anderegg, Prall et al PNAS 2010 and Cook et al, ‘Quantifying the Consensus’ ERL 2013 are so seriously flawed that you should not want your paper associated with them.

    Your survey is good. I have gone over my objections to your reporting on your survey elsewhere. AFAICT Tol’s objections regarding your team’s work are focused on the same area I object to–ignoring the headline figure and focusing solely on subgroups.

    (On a side note, if I were a young scientist who had been invited to participate in your research study, I am not sure how I would feel about your saying that I am not expert and that my opinions should be ignored. I guess going forward we should all ignore people like Tamsin Edwards and listen only to people like Richard Lindzen.)

    But it would not be a good thing for a valid survey to be lumped together with egregious examples of junk science.

    The website Skeptical Science provides links to 117 papers by skeptical scientists published during the time frame used by Oreskes in Beyond the Ivory Tower that she did not report finding in her paper.

    Quantifying the Consensus used amateur reviewers who violated the terms of their agreement with the research team regarding communicating with each other and not searching for the authors of abstracts they were reviewing. One reviewer classified 765 abstracts in 72 hours. Scientists subsequently contacted have repudiated the classification of their paper.

    Your research study is of much higher quality than Prall et al, Oreskes and Cook et al.

    They are junk science. Your work is not.

  15. thomaswfuller2 Says:

    Marco, did you contact the paper’s authors? I ask because I may be in error.

    Desmogblog reports SEPP’s actions as follows:

    June, 1992

    SEPP has promoted the The Heidelberg Appeal which was publicly released at the United Nations Earth Summit held in Rio de Janeiro from June 3-14 1992.

    Fred Singer helped to organize the conference where it was revealed, and may have been involved with the Appeal in other ways. [9]

    The document was promoted through Philip Morris’s public relations firm, APCO & Associates. They then organized a seminar promoting various aspects of the associated scientific claims which was “co-sponsored” by SEPP and the George Mason University’s International Institute (funded by Philip Morris). [10]

    The appeal states that “a Natural State, sometimes idealized by movements with a tendency to look toward the past, does not exist and has probably never existed since man’s first appearance in the biosphere, insofar as humanity has always progressed by increasingly harnessing Nature to its needs and not the reverse.” [11]

    February, 1992

    Published a “Statement by Atmospheric Scientists on Greenhouse Warming” which objected to the United Nations Conference on Environment and Development Earth Summit planned to take place in Rio de Janiero on June, 1992.

    The signatories to the letter complained that the Earth Summit “aims to impose a system of global environmental regulations, including onerous taxes on energy fuels, on the population of the United States and other industrialized nations. Such policy initiatives derive from highly uncertain scientific theories. They are based on the unsupported assumption that catastrophic global warming follows from the burning of fossil fuels and requires immediate action.”

    The paper I link to and have referenced is the June paper. If you have confirmation from Prall or any of the others involved in Anderegg, Prall et al PNAS 2010 that they in fact used the February paper, then please let me know so I can correct the errors on my weblog.

    I will do so somewhat grumpily as the citation from their paper reads as follows: “1992 statement from the Science and Environmental Policy Project (46 names)”.

    It will not affect the rest of my criticisms about the paper. They got basic data wrong, they search using Google Scholar and did not cross reference with easily available databases of academic publications. They got the math wrong in their analysis. Most egregiously, they labeled participants in the survey ‘deniers’ and included a link to Prall’s website where the participants could be identified.

    But if one of the authors can say that they referenced the other SEPP letter I will correct my misstatement.

  16. thomaswfuller2 Says:

    If I do correct my statement, I will include the observation that all the signatories to the second letter would fit Bart’s definition of ‘expert’ climate scientists due to the number of publications each of them has achieved.

  17. Bart Verheggen Says:

    Tom,

    Tol’s reanalysis of several survey’s numbers goes a lot further than just agreeing with you on those particular points (including vs excluding responses without an actual estimate; including vs excluding the other relevant attribution question; focusing on subgroups or not).

    Read my post again. Many of the numbers he puts in his graph are entirely meaningless as an estimate of the scientific consensus on climate change. That conclusion is independent of what you think of those other consensus studies.

    E.g. you can’t meaningfully judge scientific consensus by looking at a subsample which was constructed on the basis of their stated opinion on the issue at hand. That is a circular argument.

  18. thomaswfuller2 Says:

    Anderegg, Prall et al have a note at the bottom of their Supporting Information page: http://www.eecg.utoronto.ca/~prall/climate/list_sources.html

    It leads to this page: http://www.eecg.utoronto.ca/~prall/climate/list_sources.html

    They list this document: SEPP92: 1992 SEPP Statement by Atmospheric Scientists on Greenhouse Warming, 47 signers (all listed): SEPP 1992 statement on greenhouse warming.

    The link provided opens a page displaying the error message ‘404 – File or Directory not found.’

  19. thomaswfuller2 Says:

    Bart, when you invited climate scientists to participate in your research study, did you already know you were going to segment them by number of publications?

    If so, did you notify those with fewer publications that they would be considered ‘inexpert’?

  20. Bart Verheggen Says:

    Tom,

    No, we didn’t. By the way, the adjective “inexpert” is yours, not ours.

    We did ask the questions we asked in order to be able to relate answers to each other in a general sense, but no, we didn’t set out in advance to segregate responses regarding a consensus position specifically by number of publications.

  21. thomaswfuller2 Says:

    Well, if people with more publications are more expert, which you have said, what does that make people with fewer publications?

    If you decided to focus on number of publications as a report criteria after looking at the results it was even more incumbent upon you to clearly walk through them, starting with the headline number.

    Not best practice.

  22. Marco Says:

    Tom, I already pointed out the dead link, that’s why I used Sourcewatch as a link, as it lists the signatories. It is clear that they used the February 1992 statement from SEPP, simply because the Heidelberg Appeal is NOT a SEPP letter/statement. Even the one link you managed to find makes it clear that SEPP only *promoted* the Heidelberg Appeal and that Fred Singer may have been involved in drafting it. That still does not make a SEPP letter/statement. Jim Prall’s homepage that is linked in the SI makes it even more clear.

    You may also want to read the Heidelberg Appeal and see how that fits with any supposed skepticism about climate change. There is no mention of climate change. It really is that simple to show that it isn’t the SEPP statement that Anderegg et al used. I have no need for further confirmation, and you should not either.

    “I will include the observation that all the signatories to the second letter would fit Bart’s definition of ‘expert’ climate scientists due to the number of publications each of them has achieved.”

    If you do, you’d be lying. For Bart to include them as experts they would have to have publications related to climate change. Many don’t. I am not even sure all of them have any publications at all (does Roy Leep? Elliot Abrams? Brian Sussman? Robert E. Zabrecky? I could not find any).

  23. thomaswfuller2 Says:

    Marco, those guys were so inept that nothing is clear about what they have done. I’m inclined to agree with you but I’m going to wait and see. I’d hate to have to retract a correction.

  24. Marco Says:

    It is obvious what statement they used to anyone but Tom Fuller (who were inept again?). You can even go through Prall’s database with names and find those signatories of the SEPP92 statement.

  25. Willard Says:

    Groundskeeper Willie rips off his shirt and chases squirrels in yet another thread. While persistence may be a ClimateBall virtue, this fails to address BartV’s argument. Not the best lukewarm practice.

    Let’s repeat the argument:

    Tol’s line of reasoning here is similar to his misrepresentation of Oreskes’ results, by taking the number of acceptance papers not just as a fraction of papers that take position, but rather as a fraction of all papers, including those that take no position on current anthropogenic climate change. Obviously, the latter should be excluded from the ratio, unless one is interested in producing an artificially low, but meaningless number.

    Where’s the justification for Tol’s lowballing?

  26. thomaswfuller2 Says:

    I regret the error.

    https://thelukewarmersway.wordpress.com/2015/09/28/the-mistake-i-made-when-criticizing-anderegg-prall-et-al-pnas-2010/

  27. Nobodyknows Says:

    “John Cook’s study in which they found 97% agreement in the scientific literature that global warming is human caused.”
    What kind of Logic is this?
    I once made a study that found 97% agreement that dogs are black.

  28. thomaswfuller2 Says:

    Bart, you write, “E.g. you can’t meaningfully judge scientific consensus by looking at a subsample which was constructed on the basis of their stated opinion on the issue at hand. That is a circular argument.”

    Would that apply to a subsample of scientists selected because of number of publications?

  29. Marco Says:

    Odd question; of course that does not apply. They are not selected based on their opinion, but based on their expertise.

  30. thomaswfuller2 Says:

    No Marco, as Bart says, publication number is not equivalent to expertise. It is used as a proxy for it, but IMO probably should not be, for reasons discussed on the last thread. Bart has previously acknowledge the serious flaws in using publications as an equivalent for expertise. As none of you can find another metric, you use it. You probably shouldn’t.

    However, publication frequency can also be viewed as a proxy for alignment with the editorial opinion of journals, which in this case means the standard consensus view on anthropogenic contributions to climate change.

    If that is a valid point of view (and I have no doubt you at least will contest it), then it is a “subsample of scientists constructed on the basis of their stated opinion on the issue at hand.”

  31. Marco Says:

    Tom, it can be viewed as such, but you would do so without any evidence whatsoever to back it up. Just show me that the “editorial opinion” of the top-10 journals in climate science is “AGW > 50%”. Authors cannot know the editorial opinion, most journals do not publish any statements about there being a consensus. Moreover, most of those with a lot of publications have been publishing for more than 20 years. You’d have to show that this was the “editorial opinion” already more than 20 years ago. And after you have shown me what you believe to be evidence that this is the consensus “editorial opinion” already so long ago, you need to explain the publication records of e.g. Lindzen and Pielke Sr. Based on your argumentation, they should not have been able to get such a long publication list.

    It is starting to look a lot like you simply do not like the answer and therefore “MBW”.

  32. Bart Verheggen Says:

    Tom,

    Regarding using the number of publications as a proxy of expertise, I said e.g.

    Your caveats re number of publications are mostly valid. But even if imperfect, it can still be a useful indicator of a respondent’s expertise, esp because we cast such a wide net of respondents.

    You seem to argue that it could likewise, or even more so, be regarded as a proxy for somebody’s opinion. That to me is a strange and unsupported claim. Number of publications relates logically to how much scientific research somebody has been engaged in. There surely are many more variables influencing someone’s number of publications, that’s why I said it could be used as an imperfect proxy.

    Ideally, a publication is accepted or denied based on scientific validity; not based on the opinion of the authors. I know peer review isn’t perfect (I could give some examples based on my own experience), but there isn’t a conspiracy either where people with certain opinions, even though their work is perfectly legitimate scientifically, are prevented from publishing. I’d wager that every scientist has had the experience of their work not being accepted for publication for reasons that they disagree with. That doesn’t mean that they’re being censored.

  33. thomaswfuller2 Says:

    Hi Bart,

    Valid points all, but I’m not disputing them. I’m arguing that once you rely on a proxy you are subject to the weaknesses of the proxy as well as the advantages it brings. Mr. Mann might have interesting things to say about that…

    The weaknesses of publication records are:

    1. Very capable younger scientists have not had time to establish a record of publications. Dismissing their opinions leads to loss of useful information.

    2. As ‘alarmists’ like to point out whenever an older scientist expresses a skeptical viewpoint, at some point in the natural cycle of a person’s career, ongoing education becomes less important. One can make the case that someone reaching the end of their career actually knows less than a freshly minted scientist.

    3. The tools and techniques used in tertiary education are different than they were when many older scientists were educated. In addition, new knowledge is incorporated into texts available to younger scientists. This again may advantage the young at the expense of the old.

    4. Some scientists are co-authors of numerous papers for reasons other than their ability to contribute to the main body of the scientific arguments advanced in the paper. Their publication count may be more impressive than their actual command of the field.

    5. Some very good scientists work outside the academic world and publication may not be a priority for them. Using publications as a proxy for expertise again may devalue their opinions.

    I see no good reason to accept your statement that “Number of publications relates logically to how much scientific research somebody has been engaged in.” Are there publications supporting this point of view?

    It seems in the way you refer to it that it is an assumption on your part. I think the 5 points I list constitute an argument that the assumption, if that is what it is, is unwarranted.

    Because your analysis framework emerged after viewing the results of the survey, as you mention above, it seems that you are in the position of trying to compensate for unwanted results.

    And I know you better than that, Bart. I don’t think you operate that way. But you haven’t adequately (for me, at any rate) explained your analytical choices.

  34. thomaswfuller2 Says:

    You might find this of interest: http://higheredstrategy.com/wp-content/uploads/2012/06/2012-Bibliometrics-and-Publication-Culture-HESA.pdf

    This paper suggests that citation counts might have been more productive in determining expertise: http://link.springer.com/chapter/10.1007%2F978-3-658-03348-4_4#page-1

    Several publications that discipline normalization would have been valuable to your research efforts, eg. https://books.google.com.tw/books?id=olu3k6CiBuUC&pg=PA262&lpg=PA262&dq=problems+with+publication+counts&source=bl&ots=00hoPJRWoB&sig=ZTT3T0EEsWvYpeDABIx9FcRRdbg&hl=en&sa=X&ved=0CEQQ6AEwBmoVChMIzNb5g6mvyAIVQx2UCh1_mg2a#v=onepage&q=problems%20with%20publication%20counts&f=false

    In fact, a cursory overview of publications regarding publication counts shows that the problems pub counts face in providing valid information are formidable.

  35. hobamja Says:

    Myself being in a high school physics class find it interesting that other scientist manipulate good evidence for personal reasons or because “They wanna stand out”. You would think that everyone would want to be one the same page when it comes to climate change due to it be one of the most rebuttled scientific facts, this is why people are skeptical when looking at new data because of the people who manipulate.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: