18.1.2 Algorithms (Smooth)

1 Moving window in adjacent-averaging, Savitzky-Golay or percentile filter method
2 The adjacent-averaging method
3 The Savitzky-Golay method
4 The percentile filter method
5 The FFT Filter method
6 The Lowess and Loess method
7 The Binomial method
- 7.1 Cutoff frenquency

Moving window in adjacent-averaging, Savitzky-Golay or percentile filter method

When the smoothing method is adjacent-averaging, Savitzky-Golay or percentile filter, each smoothed data point is computed from data points within a moving window. Let $\left\{f_i| i = 1,2,...,N\right\}$ be the input data points and let $\left\{g_i|i = 1,2,...,N\right\}$ denote the output data points. Each $g_i$ is computed from $\left\{f_m|i - floor(npts/2) < m < i + floor(npts/2)\right\}$

where npts is the value of the Points of Window variable.

However, when the smoothing method is FFT filter, moving window is not used. Instead, the whole signal is processed.

The adjacent-averaging method

The adjacent-averaging method uses the simplest possible averaging procedure: each $g_i$ is the average of the data points within the moving window. If the Weighted average option is used, the average will be computed using weighted averaging. In this case, a parabolic weight is used, with the weight area normalized to 1. For a window whose center is in i (which means to calculate the ith averaged point), the weight which corresponds to the jth (j=0, 1, ... npts-1) point is:

$w_j=1-(\frac{(j-i)}{(N+1)/2})^2$

where N is the number of Points of Window.

The Savitzky-Golay method

The Savitzky-Golay method performs a polynomial regression to the data points in the moving window. Then $g_i$ will be computed as the value of the polynomial at position i.

The percentile filter method

For the percentile filter, the pth quantile of the points in the moving window is assigned as $g_i$ , where p is specified by the parameter, Percentile. The pth quantile (or 100 pth percentile) is computed from the empirical distribution function as follows:

Let $npts\cdot p/100=j+g$

where j is the integer part of $npts\cdot p/100$ , and g is the fractional part of it.

Then we can compute the pth quantile, which is denoted by y, with the following equations:

$y= \begin{cases} x_j, & \mbox{if }g=0 \\ x_{j+1}, & \mbox{if }g>0 \end{cases}$

where $x_j$ is the jth (j=0, 1, ... npts-1) point in the moving window.

The FFT Filter method

When the FFT Filter method is selected, Origin performs the following:

Calculate the mean of the first 1% data points and the mean of the last 1% data points.
Construct a straight line throught these two points and subtract the input data by this line.
Perform FFT on the dataset acquired in last step.
Apply filtering with the low-pass parabolic filter on the transformed acquired in last step. Fourier components with frequencies higher than a cutoff frequency are removed. The cutoff frequency is defined as:
$f_{cutoff} = \frac {1}{2n\Delta t}$
where n is the Points of Window specified, and $\Delta t$ is the time (or more generally the abscissa) spacing between two adjacent data points. Larger values of n result in lower cutoff frequencies, and thus a greater degree of smoothing. The transformed data is multiplied with a one-side window so the above formula is further divided by 2 to account for a two-sided window.
The function used to clip out the high-frequency components is a parabola with a maximum of 1 at zero frequency, and falling off to zero at the cutoff frequency.
Perform IFFT on the filtered spetrum.
Add the baseline to the dataset acquired in last step.

Note:

The low-pass parabolic filter used here is the same as the low-pass parabolic filter in the FFT Filters tool.
There are more variable available when this function is used in script. See the X-Function document for the smooth X-Function for details.

The Lowess and Loess method

Lowess and Loess are abbreviations for "locally weighted scatterplot smoothing" or "locally weighted least squares". We say "locally" because we calculate each smoothed value using neighboring points contained within a span of values. This method is classically performed by the following steps:

Calculate weights for a center point $x_i$ , and all neighboring points contained within the span, using the tri-cube weight function...
$w_i(x)=(1-(\frac{|x-x_i|}{d_i})^3)^3$
where $x$ is a neighbor point within the span associated to the current center point $x_i$ , and $d_i$ is the distance along the abscissa from $x_i$ to the most distant neighbor points within the span.
Perform the weighted least square regression.
- For Lowess, a weighted linear regression is used.
- For Loess, a second-order polynomial regression is used.
Obtain predicted value $(x_i,\hat{y}_i)$ given in the step2 for $x_i$ .
Move to next point $x_{i+1}$ , then perform step1-3 to get predicted value $(x_{i+1},\hat{y}_{i+1})$ . The calculation stops when all points are calculated.