Saturday, April 14, 2012


cc: "'Jenkins, Geoff'" <>, "'David Sexton'" <>, "Murphy, James" <>
date: Wed, 3 Dec 2008 10:36:08 +0000
from: C G Kilsby <>
subject: RE: The change factors that Ag sent to Newcastle recently -
to: "'Phil Jones'" <>, Colin Harpham <>, "'vassilis glenis'" <>

Phil (et al by cc)

yes, I knew the number were non-final, thought you did too.

However, I have concerns that though the final ones will change, they won't eliminate
these nonsense values.

Presumably the wacky numbers are the result of fitting pdfs to (small) samples of changes:
there is a finite prob of v large or small changes.
This is a long standing problem with using estimates of uncertainty, especially with a
fixed interval variable (such as sun hours, pdry etc.)
Its OK to put 95% or 67% intervals on the mean, but altogether different when you actually
generate and see the real numbers associated with (say) the 1 and 99 percentiles!

What to do?

1. Arbitrarily capping the changes is, as we discussed previously, not desirable, for
several reasons, including introducitn of bias if you simply truncate.

But, we have to do this in some form in order to stop the WG falling over!

There is every justificaiton to do something like this if you get negative sunshine hours
or DTR reducing more than DTR itself etc.

This could be done by MOHC (preferably), Ag, or least favourite, us by fixing at the WG

The most elegant alternative is (for David) to use a logit, as we do for pdry to avoid
going below zero. This would presumbaly mean re-running though....

2. I'm also worried about the big variations month to month, but can see how this comes out
of the emulator etc. I don't think any smoothing (a la UKCIP02) is respectable either, so I
guess we are stuck with this. It does mean though that the "wackiness" of any generated
series will be limited, becasue consecutive months will cancel their effects.

3. A little galling to spend time fixing the WG up to reproduce control to good level, and
then perturb to the climate of Venus, and as various people having criticised for
retaining wacky numbers (me included), difficult to see how we can justify this sort of
thing now!

If we are only generating one or so weird time series per thousand , we can justify them as
real outliers: the first thing is to make the changes physically (if not climatologically)

I therefore seek advice from David and James on how to limit these values? !



From: Phil Jones []
Sent: 03 December 2008 09:43
To: C G Kilsby; Colin Harpham
Subject: RE: The change factors that Ag sent to Newcastle recently - PLEASE LOOK!

So we're working with not-the-final numbers !
At 00:33 03/12/2008, Stephens, A (Ag) wrote:

Hi Phil,

I thought everyone was aware that this data was not the final data.

Before I doing anything we should see what David says regarding how many of these
problems should magically go away with the next dataset. It should be due for delivery
late next week.

However, nice analysis, very useful.



From: Phil Jones [[1]]
Sent: 02 December 2008 17:10
To: C G Kilsby; 'Colin Harpham'; 'vassilis glenis'; David Sexton; Murphy, James;
Jenkins, Geoff; Stephens, A (Ag)
Subject: The change factors that Ag sent to Newcastle recently - PLEASE LOOK!
Dear All,
CRU and Newcastle are working on the Change Factors that Ag has sent up.
My understanding is that these are the final set! I hope this is not the case, as
there are serious problems with them. According to Vaz at Newcastle, Ag sent him
1000 cfs for each of the 25km cells (and there are 439 25km cells). At the
moment we are working directly with these 25km cells, before going down to the
What we're taking from
for the WG is mean T and DTR directly - we do nothing. Ag then takes two
formulae that we gave him to calculate what we want (delta's for sunshine and vapour
pressure from cloudiness and RH - needing T as well to get VP).
The WG runs for most of these model variants with no problems, but crashes
on 47 of them. What Colin here has put together is an excel file of the
deltas for these 47 that crash the WG. The plot then shows these deltas by month for
4 variables, Tmean, DTR, Sunshine, Vapour Pressure.
First remember that the WG (the CRU part) is only failing on 47 (out of 439 times
1000 runs).
We should probably do a similar plot for a chunk of the ones that work OK.
The numbers of the 25 km grid boxes are in the excel file. They relate to Vaz's
numbering system from north to south. Almost all of the boxes where failures have
occurred have been in central southern England. You can see these in some in
Chris' ppt that Kathryn sent around earlier today. They are the gaps on the RH maps.
It only takes one very odd number to get the WG to crash. Many of these crashes are
caused by what happens for Vapour Pressure in July. Why July is beyond me (?), so
Ag - can you check the code that Colin sent to produce Vapour Pressure from RH
and T. It might be best of you could send us more of this code as this could be where
some of the other problems come from (eg. cloudiness to sunshine).
The other values that cause the WG to crash are some of the DTRs and the July
Sunshine. It is the reductions that cause the problem. We can put a trap in for
these but they make no sense. There are two examples of DTR reducing by >7 deg C
in November. The problem here is that this is larger than the actual DTR for November..
The drops in sunshine in July (on two occasions) are larger than the average sunshine,
which means the WG ends up with negative sunshine!
The increases in DTR in June and August by almost 15 deg C also seem quite out of
the ordinary. Also, there appears a tendency for some of these variants for
Tmean and DTR to be high one month, then low, then high again - a feature
which can be seen in the plots.
The oddest model variant is on line 27 in the xls (labelled 373). July T increases by
14.42 deg C, sunshine decreases by 5.06 hours, vapour pressure increases by 26.4mb!
The only climate I know like this is Singapore Airport!
Hopefully with a bit of work we will be able to solve these issues quite quickly.
I hope they are with you Ag (sorry) as it will be easier to correct for. If they are in
the original model variants from the HC then we have problems. As I understand it
the users will be able to get access to these numbers - which cause us lots of

Prof. Phil Jones
Climatic Research Unit Telephone +44 (0) 1603 592090
School of Environmental Sciences Fax +44 (0) 1603 507784
University of East Anglia
Norwich Email

Scanned by iCritical.

Prof. Phil Jones
Climatic Research Unit Telephone +44 (0) 1603 592090
School of Environmental Sciences Fax +44 (0) 1603 507784
University of East Anglia
Norwich Email

No comments:

Post a Comment