cc: "C G Kilsby" <c.g.kilsbyatXYZxyzcastle.ac.uk>, "Phil Jones" <p.jonesatXYZxyz.ac.uk>
date: Thu, 11 Dec 2008 11:14:00 +0000
from: "vassilis glenis" <vassilis.glenisatXYZxyzil.com>
subject: Re: Queries re WG falling over and wacky numbers ..
to: "Colin Harpham" <c.harphamatXYZxyz.ac.uk>
Just to let you know that the WG didn't crash this time. However, it
might be a good idea to use the previous version with the new set of
data to check if there are still silly numbers in.
On Wed, Dec 10, 2008 at 9:57 AM, Colin Harpham <c.harphamatXYZxyz.ac.uk> wrote:
> The while loops already had a >1000 where the normalised variable is set to
> the range -1...1, this was put in because the WG would occasionally hang
> with certain observed data. However the exit condition of the loop still
> held (between 0.0 and 2*mean - denormalised). With proper numbers this is
> fine. I have now put a loop exit condition in for perturbed runs (only)
> because the perturbed variable can it seems exceed 2*mean.
> Temperature range can also become negative so I have set the range to 0.0 if
> Apart from Trange this mod does not do any filtering - silly number in,
> silly number out.
> I have done some random checks with the 'ini' files Vas sent and it seems
> Vas, can you give the revised code a run and see if still hangs anywhere.
> -----Original Message-----
> From: C G Kilsby [mailto:email@example.com]
> Sent: 09 December 2008 15:58
> To: 'Phil Jones'
> Cc: 'vassilis glenis'; Colin Harpham
> Subject: Queries re WG falling over and wacky numbers ..
> Having a chat with Vas re issues etc. Any ideas on the following Q's:
> 1. Looking at the numbers/cfs when it falls over: these are not all actually
> extremes: more likely it is weird >combinations< of variables/cfs that
> casues the problem. If so, clipping won't help!!
> Also, if this is the case, it shows that the corss-correlations of eth pdfs
> of cfs are not high, and ill-conditioned vectors of cfs result. We would
> have been better off using the WG cross correlations (IVRs) to determine the
> VP, sunshine, wind etc rather than fixing the cfs for these with the
> much-vaunted "physically-based" relationships coming out of the RCMs !!!
> Is there any other "sanitising " strategy to spot these bad combinations?
> Can we write some rules for what we would expect, and report back/exclude
> the run if they are non-physical ??
> Or do we just accept this as part of the "statistical" rather than
> "physical/deterministic" approach?!
> 2. What is this "while loop" that your WG follows when it falls over (for
> VP?) ?
> What happens if you jump out after (say) 100 iterations as a fail safe?
> Could you set something up to log this if it happens, rather than the code
> just hanging?
> (Ideally won't need this, if the clipping and sanitising works, but I fear
> this won't happen 100% !!)