1.

Smoothing of Data

1.1Simulation
(core)
Produce a discrete time series ) t ( y
i
,

=
t =
5
1 j
j j
) t f 2 cos( a ) t ( y . (1)

For your own choice of the amplitudes ) a (
j
and frequencies ) f (
j
, and add some Gaussian
noise to each digitised value. Experiment with amplitudes (including that of the noise term),
and frequencies, showing results graphically. Then smooth your noisy curves with at least
two different filters, e.g., a simple moving average smoother and an order two binomial filter.

=
+
=
p
p k
k i k i
y g x . (2)

Test the validity of the expression below, for your filters.

) y var( g ) x var(
p
p k
2
k
|
|
.
|

\
|
=

=
. (3)
Answer:

The discrete time series was constructed using amplitudes of [1, 1.5, 2, 2.5, 3] and
frequencies of [0.5, 1, 1.5, 2, 2.5], this produced a noiseless sinusoidal plot shown below in
figure 1. This plot shows the superimposition of four sinusoids created using equation 1 with
the four different frequencies and amplitudes stated previously. These were produced using
two for loops to input the frequencies and amplitudes into equation 1. Varying amplitudes
and frequencies were tested to see what effect these had on the plot, this produced the
expected result of lower frequencies producing shorter periods and large amplitudes
producing just that, larger amplitudes. As this is a superimposition of the four waves the
difference in amplitude and frequency affected the final outcome in many different ways, due
to constructive and destructive interference.


Figure 1: Noiseless raw signal produced using equation 1 and the frequencies and amplitudes
shown above. This is a superimposition of four sinusoids each with the varying amplitudes
and frequencies as stated above.

Noise was then added to the clean raw signal shown in figure 1, varying amplitudes of noise
were tested and these did as was expected, larger amplitudes of noise created larger noise.
This noisy raw signal (referred to as the raw signal) is plotted in figure 2 below where the
noise, which is Gaussian noise with a variance of 1, is multiplied by 0.5.



Figure 2: Noisy raw signal created from addition of randomly generated Gaussian noise to the
clean noiseless raw signal.

This raw signal (noisy) is then smoothed using both a simple centred moving average
smoother and a binomial filter. Initially a 5 point centred moving average filter was used and
produced the following. The centred 5 point moving average smoother was constructed using
the following formula:

( )
5
2 1 1 2 + +
+ + + +
=
k k k k k
k
y y y y y
y . (4)

This filter does remove the majority of the noise returning it to a relatively clean data set that
retains the shape of the original clean signal, just the smoothed plot is shown in figure 3 and
the smooth plot against the original data is plotted in figure 4. This plot of the original data
against the filtered noisy data shows the difference between the two.

Figure 3: Filtered noisy signal. Filtered using a 5 point centred moving average smoother.




Figure 4: Filtered noisy signal plotted against the original clean signal, this shows that while
the signal is almost bought back to the original it does lose some resolution in the peaks due
to attenuation of the data and there is still some noise present.

A fourth order binomial smoother was then trialled to see how this type of smoother filters
noisy data, a binomial smother of order 5 that was used is shown below (equation 5):


16 16
4
16
6
16
4
16
2 1 1 2 + +
+ + + + =
k k k k k
k
y y y y y
y . (5)

This type of smoother preferentially weights the smoothing so that the point being replaced,
y
k
, is given more importance than the outside terms. Below in figure 5 is a plot of the fourth
order binomial smoothed noisy data is shown below followed by a plot of this smoothed data
against the clean signal.


Figure 5: Filtered noisy signal, filtered using a fourth order binomial smoother

Figure 6: Filtered (using a binomial smoother, 4
th
order) noisy signal plotted against the
original clean signal, this shows that the signal is quite similar to the original there is some
attenuation of the data.

To properly compare the data presented by these two filters one of the best methods would be
to compare the variance of the clean signal to the variance of the noisy data. The smoother
with a variance that is closest to that of the clean signal would have smoothed the data best
and attenuated the data the least, also the mean of the smoothed data should still be close to
that of the clean signal. Table 1 shows the mean and variance of the clean signal and both
smoothers.

Table 1: The mean and variance of the clean signal, the noisy signal after it has been filtered
by the centred 5 point moving average smoother and the noisy signal after it has been filtered
by the 4
th
order binomial smoother.
Mean
Mean
(from clean)
Variance
Variance
(from clean)
Clean
Signal
0.012425 11.8078
Centred 5
Point
0.013343 0.0009183 11.271 0.5368
4th Order
Binomial
0.012716 0.0002907 11.3401 0.4677


This table shows that the binomial smoother for this data set is marginally better than that of
the centred 5 point filter as the mean is closer to the expected mean and the variance is closer
to the expected variance.