[R] Smoothing Techniques - short stepwise functions with spikes

Ralf B ralf.bierig at gmail.com
Tue May 11 09:17:27 CEST 2010

R Friends,

I have data from which I would like to learn a more general
(smoothened) trend by applying data smoothing methods. Data points
follow a positive stepwise function.

|                                    x
|                      xxxxxxxx xxxxxxxx
|       x    x
|xxxx xxx xxxx
|                                                   xxxxxxxxxxxxxxxxx
          xxxxxxx xxxx

Data points from each step should not be interacting with any other
step. The outliers I want to to remove are spikes as shown in the
diagram. These spikes do not have more than one or two points. I
consider larger groups as relevant and want to keep them in. I
sometimes have less than 5 points for each step, and up to 50 at max.
Given these conditions would you suggest using one of the moving
averages (e.g. SMA, EMA, DEMA, ...) or the locally linear regression
(lowress) method. Are there any other options? Does anybody know a
good site that overviews all methods without going to much into
mathematical details but rather focusing on the requirements and
underlying assumptions of each method? Is there perhaps even a package
that runs and visualizes a comparison on the data similar to packages
like 'party' ? (with 1000s of active packages, one can always hope for

Thanks in advance!

More information about the R-help mailing list