cc: Phil Jones <p.jonesatXYZxyz.ac.uk>, Ray Bradley <rbradleyatXYZxyz.umass.edu>, Malcolm Hughes <mhughesatXYZxyzr.arizona.edu>, Steve Schneider <shsatXYZxyznford.edu>, tom crowley <tomatXYZxyzan.tamu.edu>, Tom Wigley <wigleyatXYZxyzker.UCAR.EDU>, Jonathan Overpeck <jtoatXYZxyzrizona.edu>, <asocciatXYZxyz.net>, Keith Briffa <k.briffaatXYZxyz.ac.uk>, Tim Osborn <t.osbornatXYZxyz.ac.uk>, <Tim_ProfetaatXYZxyzberman.senate.gov>, Ben Santer <santer1atXYZxyzl.gov>, Gabi Hegerl <hegerlatXYZxyze.edu>, Ellen Mosley-Thompson <thompson.4atXYZxyz.edu>, "Lonnie G. Thompson" <thompson.3atXYZxyz.edu>, Kevin Trenberth <trenbertatXYZxyz.ucar.edu>
date: Mon, 27 Oct 2003 11:18:31 -0500
from: Mike MacCracken <mmaccracatXYZxyzcast.net>
subject: Re: CONFIDENTIAL Fwd:
to: "Michael E. Mann" <mannatXYZxyzginia.edu>, Michael Oppenheimer <omichaelatXYZxyznceton.EDU>
Hi Mike--Back from a couple of weeks away and trying to catch up.
I thought it might be of interest that OMB has put out draft guidelines on what is supposed
to constitute peer review for agency reports or reports they rely on about major
issues--and just to note that there is a sense (in at least one agency and some CCSP
staff), supproted by some discussions with the author of the guidelines, that most or all
climate materials would need to live up to the guidelines.
Two points on what is there:
a. Frankly, there are a lot of problems with the proposed guidelines in that they seek to
have all reviewers essentially be so unconflicted that no one who knows anything is likely
to qualify or be willing to be a reviewer. The whole notion of the content of the review
comment mattering more than its source is totally lost (much less offering nay guidance on
how seriously agencies need to take any comments). I imagine those on journal review boards
or serving as editors (like Steve) might want to check out the proposal and see how their
guidelines compare--and how they think the OMB guidelines might work (or not work) for
b. However, the guidelines do presume that journal peer review provides a challengeable
qualification to the paper. Interestingly, there is no indication that the journal must be
of any given quality or follow any approved procedures, so what is sure, if these
guidelines go through, is that there will be a rash of new journals created, all of little
I have made these and a number of related points to the OMB in response to their
solicitation of comments. And now, the NRC is going to hold a meeting on them (see email
notice below--though without form), as apparently I have not been alone in objecting. I'll
be on travel but did send in my letter to OMB (copy available on request--since it is
several pages long, I won't burden everyone with the letter). By the way, comments deadline
has been extended to Dec 15 to accommodate NRC workshop, I presume.
In any case, this matter of what constitutes "peer review" is coming up for attention by
this Admin--so perhaps this effort of skeptics to get things into what they call
peer-reviewed journals is so they can be cited more directly by the Admin.
Subject: PEER REVIEW OF REGULATORY SCIENCE WORKSHOP-November 18, 2003
In light of expressions of interest and concern from within the research
community regarding the newly issued "Proposed OMB Bulletin and Supplemental
Information Quality Guidelines: Peer Review and Information Quality," and
with the encouragement of U.S. Office of Management and Budget (OMB), The
National Academies Science, Technology, and Law (STL) Program intends to hold
a one-day public workshop on Tuesday, November 18, 2003, in Washington, D.C.,
at which federal agencies subject to these new standards can share their
views and hear ideas and concerns from each other and from external
communities, including academic researchers, about the implications, merits,
and practicality of the proposed bulletin. The workshop is intended to
assist the agencies in developing their agency-specific comments on the
bulletin and ultimately in developing their peer review procedures.
Further details on the agenda will be sent out in late October. Please free
to forward this announcement to other interested parties.
If you would like to attend the workshop, please fill out the attached
registration form and fax to (202-334-2530).
For more information please contact:
Contact Name: Stacey Speer
Attachment Converted: "c:\eudora\attach\peer_review_andnfo_quality.pdf"