from: Phil Jones <p.jonesatXYZxyz.ac.uk>
subject: Re: Report
to: "Brohan, Philip" <philip.brohanatXYZxyzoffice.gov.uk>, Ian Harris <i.harrisatXYZxyz.ac.uk>
Harry and I are meeting at 2pm today and you'll get the answers and/or
additional material later today or tomorrow. As I'm away tomorrow, I'll
answer 3 now and give my thoughts on 6.
3. Each month the HC updates the 1991-2005 file with the latest CLIMAT messages.
I also do this and pick up earlier CLIMAT messages (for the last 18 months) or
so in case any late data get added in. I do this monthly and add in any months
of MCDW (Monthly Climatic Data for the World - a file I get from NCDC) that
have been received in the past month. MCDW is currently about 2-3 months behind
real time. About 100 stations get added through this - most are additional stations
in the US that don't appear in the GTS CLIMAT messages.
I also from MCDW add in their back data. This is approximately an additional 50-100
station months worth of data - for a random selection of countries. This often (but not
always) includes Pakistan, a few sites in India, Sudan, Brazil, a few sites in
Thailand plus others. These listed have been regulars in the last 12 months.
How NCDC gets this data isn't clear to me. It is likely to be back data received after
cut-off each month. It could be mailed in reports. If it is coming by the GTS your system
could be losing it. In assessments done by WMO every year, there is a +/- 2-5% difference
in CLIMAT reports received at the main monitoring centres around the world.
Each month I also manually add in monthly temperatures for some sites in the Antarctic.
These are for the South Pole and Russian sites on the continent (3 sites, Bellingshausen,
Vostok and Mirny). South Pole never comes in over the GTS and the Russian sites are
very sporadic. I get these by email from the SP and by email from a colleague in St.
Petersburg (Victor Lagun). Victor is on the attached paper which came out in IJC
All the Antarctic data in this paper are in the dataset. Every now and then I go to the
web site of Gareth Marshall and pick up and manually add in back Antarctic data (this is
mainly for the French site at Dumont D'Urville) and the NZ site (Scott Base) as reporting
other countries has been relatively good recently.
I don't keep any sort of record of any of this, but all the manually entered data is
by me for outliers. All the MCDW data and back CLIMATs entered automatically get their
anomalies listed out and I look for largish values.
I am happy to continue doing all this after HadCRUT3 starts. The Antarctic stuff is
as coverage is poor down there. NCDC have produced maps of recent trends for the AR4 from
their dataset and their Antarctic coverage is much poorer than HadCRUT2. We are currently
trying to figure out if this is because of missing data or the their plotting software. It
that the single 5 by 5 boxes get omitted when plotting.
6) What Harry has plotted is the annual average corrections in the histograms. The CRU
averages appear small, but the stations are all over the world, so averaging makes them
appear near zero - adjustments don't have a preferential sign (which might be a point
making, if you gridded with and without the adjustments, differences would be very small).
For Canada, although a big country, it is still only a small part of the world. The
they have applied tend to make sites warmer or colder by about 0.7 deg C. There is less
cancelling going on and the adjustments fall into two camps.
Maybe if we split the CRU adjustments into continents (even just NH and SH) we might
get the bimodal shapes that Canada shows. The different seasonal cycles around the world
partly produce the CRU result as for winter months adjustments tend to be larger. I'll
if Harry can try this for USA+Central America/Caribbean, S. America, Europe, Asia
Australasia and Africa (+NH and SH). Some of these may not have many samples though.
At 10:07 25/04/2005, Brohan, Philip wrote:
Thanks for the report, it is basically fine - clear and concise. It
describes well all the good work you've done on the data.
There are a few points that still need clearing up, mostly details
needed for the HadCRUT3 paper I am writing:
1) I am using revision 10 of the data files (data, normals and SDs). Is
this the final version? If not, please send me the latest revision.
2) Have you any references for the data changes you have made? We'll
need these for the paper.
3) Have any other station data changes or additions been made since the
production of HadCRUT2? (Antarctic data?) This isn't really part of your
report, but we'll need to put details in the paper.
4) Please send me a list of the station IDs you deleted. I can then
delete them from the recent data file as well.
5) You've done a good job cleaning up the normals, but I still need
estimates of the expected uncertainty of the station normals where they
are inferred from limited data or taken from WMO. You will know this
from the work you describe in section 3 on process improvement. Can you
add a couple of figures like those in section 4, showing the difference
between the WMO and new normals for your 617 stations, and the effect of
using only 15 of the 30 years in generating a climatology?
6 I'm a bit confused by section 4: Additional information. If I
interpret it correctly, for both the CRU and Canadian data, a bit under
half of the measurements have been corrected. But the CRU corrections
are most likely to be small (~.2C), while the Canadian corrections are
never small, and are most likely to be medium sized (~.6C). Is this
right? Can you shed any light on this discrepancy?
(At first glance it looks as if the uncorrected Canadian data has large
inhomogeneities which are approximately normally distributed, but only
those larger than a threshold have been identified and corrected; while
the CRU corrections are an attempt to correct for the remaining small
inhomogeneities. But this is pure speculation on my part.)
On Fri, 2005-04-22 at 18:56, Ian Harris wrote:
> Please find attached a final draft of the report, "ENHANCEMENT AND
> QUALITY CONTROL OF CRU MONTHLY TEMPERATURE STATION DATA SET'.
> All comments, etc will be dealt with as quickly as possible.
> Ian "Harry" Harris
> Climatic Research Unit
> University of East Anglia
> Norwich NR4 7TJ
> United Kingdom
Philip Brohan, Climate Scientist
Met Office Hadley Centre for Climate Prediction and Research
Tel: +44 (0)1392 884574 Fax: +44 (0)1392 885681
Global climate data sets are available from http://www.hadobs.org
Prof. Phil Jones
Climatic Research Unit Telephone +44 (0) 1603 592090
School of Environmental Sciences Fax +44 (0) 1603 507784
University of East Anglia
Norwich Email p.jonesatXYZxyz.ac.uk