date: Tue, 14 Oct 2003 17:27:24 -0400

from: "Michael E. Mann" <mannatXYZxyzginia.edu>

subject: Fwd: Re: smoothing

to: Tom Wigley <wigleyatXYZxyzr.edu>, Kevin Trenberth <trenbertatXYZxyz.ucar.edu>, Keith Briffa <k.briffaatXYZxyz.ac.uk>, Phil Jones <p.jonesatXYZxyz.ac.uk>, ckfolland@meto.gov.uk, tkarl@ncdc.noaa.gov, jto@u.arizona.edu, mannatXYZxyzginia.edu

Sorry--one more error. The MSE values for "minimum norm" and "minimum roughness" are

switched in the figure legend. Obviously the former is a better fit...

mike

Date: Tue, 14 Oct 2003 17:08:49 -0400

To: Tom Wigley <wigleyatXYZxyzr.edu>, Kevin Trenberth <trenbertatXYZxyz.ucar.edu>, Keith Briffa

<k.briffaatXYZxyz.ac.uk>, Phil Jones <p.jonesatXYZxyz.ac.uk>, ckfolland@meto.gov.uk,

tkarl@ncdc.noaa.gov, jto@u.arizona.edu, mannatXYZxyzginia.edu

From: "Michael E. Mann" <mannatXYZxyzginia.edu>

Subject: Re: smoothing

Bcc: Scott Rutherford <srutherfordatXYZxyz.edu>

correction '1)' should read:

'1) minimum norm: sets padded values equal to mean of available data beyond the

available data (often the default constraint in smoothing routines)'

sorry for the confusion,

mike

At 05:05 PM 10/14/2003 -0400, Michael E. Mann wrote:

Dear All,

To those I thought might be interested, I've provided an example for discussion of

smoothing conventions. Its based on a simple matlab script which I've written (and

attached) that uses any one of 3 possible boundary constraints [minimum norm, minimum

slope, and minimum roughness] on the 'late' end of a time series (it uses the default

'minimum norm' constraint on the 'early' end of the series). Warming: you needs some

matlab toolboxes for this to run...

The routines uses a simple butterworth lowpass filter, and applies the 3 lowest order

constraints in the following way:

1) minimum norm: sets mean equal to zero beyond the available data (often the default

constraint in smoothing routines)

2) minimum slope: reflects the data in x (but not y) after the last available data

point. This tends to impose a local minimum or maximum at the edge of the data.

3) minimum roughness: reflects the data in both x and y (the latter w.r.t. to the y

value of the last available data point) after the last available data point. This tends

to impose a point of inflection at the edge of the data---this is most likely to

preserve a trend late in the series and is mathematically similar, though not identical,

to the more ad hoc approach of padding the series with a continuation of the trend over

the past 1/2 filter width.

The routine returns the mean square error of the smooth with respect to the raw data. It

is reasonable to argue that the minimum mse solution is the preferable one. In the

particular example I have chosen (attached), a 40 year lowpass filtering of the CRU NH

annual mean series 1856-2003, the preference is indicated for the "minimum roughness"

solution as indicated in the plot (though the minimum slope solution is a close 2nd)...

By the way, you may notice that the smooth is effected beyond a single filter width of

the boundary. That's because of spectral leakage, which is unavoidable (though minimized

by e.g. multiple-taper methods).

I'm hoping this provides some food for thought/discussion, esp. for purposes of IPCC...

mike

______________________________________________________________

Professor Michael E. Mann

Department of Environmental Sciences, Clark Hall

University of Virginia

Charlottesville, VA 22903

_______________________________________________________________________

e-mail: mannatXYZxyzginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137

[1]http://www.evsc.virginia.edu/faculty/people/mann.shtml

______________________________________________________________

Professor Michael E. Mann

Department of Environmental Sciences, Clark Hall

University of Virginia

Charlottesville, VA 22903

_______________________________________________________________________

e-mail: mannatXYZxyzginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137

[2]http://www.evsc.virginia.edu/faculty/people/mann.shtml

______________________________________________________________

Professor Michael E. Mann

Department of Environmental Sciences, Clark Hall

University of Virginia

Charlottesville, VA 22903

_______________________________________________________________________

e-mail: mannatXYZxyzginia.edu Phone: (434) 924-7770 FAX: (434) 982-2137

[3]http://www.evsc.virginia.edu/faculty/people/mann.shtml

## No comments:

## Post a Comment