time series analysis matlab tutorial - university of...
TRANSCRIPT
![Page 1: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/1.jpg)
Time series analysis Matlab tutorial
Joachim Gross
![Page 2: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/2.jpg)
Outline• Terminology• Sampling theorem• Plotting• Baseline correction• Detrending• Smoothing• Filtering• Decimation
![Page 3: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/3.jpg)
Remarks• Focus on practical aspects, exercises, getting
experience (not on equations, theory)• Focus on “How to do …”• Learn some basic skills for TS analysis
• Note: Usually there is not a single perfectly correct way of doing a TS operation! => learn the limitations!
![Page 4: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/4.jpg)
What is a time series?A sequence of measurements over time
0 2 4 6 8 10-1
-0.5
0
0.5
1
1.5
![Page 5: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/5.jpg)
Terminology– Continuous TS: continuous observations– Discrete TS: observations at specific times usually equally spaced
– Deterministic TS: future values can be exactly predicted from past values
– Stochastic TS: exact prediction not possible
![Page 6: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/6.jpg)
Objectives of TS analysis• Description• Explanation• Prediction• Control
![Page 7: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/7.jpg)
Simple descriptive analysisSummary statistics (mean, std) is not always meaningful for
TS
0 50 100 150 200 250 300 350 400 450 5000
2
4
6
8
10
12
![Page 8: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/8.jpg)
Sampling• Converting a continuous signal into a discrete time series• Reconstruction is possible if sampling frequency is greater than twice
the signal bandwidth
0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
Time (s)0 0.2 0.4 0.6 0.8 1
-1
-0.5
0
0.5
1
75 Hz sampling
![Page 9: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/9.jpg)
Sampling• Nyquist frequency: half of sampling frequency
10 Hz sampling 10 Hz reconstruction
0 0.2 0.4 0.6 0.8 1-3
-2
-1
0
1
2
3
0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
![Page 10: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/10.jpg)
Sampling
8 Hz sampling
0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
• Aliasing: Frequencies above Nyquist frequency are reconstructed below Nyquist frequency
![Page 11: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/11.jpg)
Sampling
8 Hz sampling
• Aliasing: Frequencies above Nyquist frequency are reconstructed below Nyquist frequency
0 1 2 3 4-140
-120
-100
-80
-60
-40
-20
0
20
Frequency
Pow
er S
pect
rum
Mag
nitu
de (d
B)
0 5 10 15 20-140
-120
-100
-80
-60
-40
-20
0
20
Frequency
Pow
er S
pect
rum
Mag
nitu
de (d
B)
40 Hz sampling
![Page 12: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/12.jpg)
Simple operations on TS• Plotting• Removing a baseline• Removing a trend• Smoothing• Filtering• Decimation
![Page 13: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/13.jpg)
Plotting in Matlab• For visual inspection of TS• For publications/talks
• plot• sptool
![Page 14: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/14.jpg)
Data preprocessing I• Removing offset• ts=ts-mean(ts);
0 0.2 0.4 0.6 0.8 10
2
4
6
8
10
0 0.2 0.4 0.6 0.8 1-1
-0.5
0
0.5
1
![Page 15: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/15.jpg)
Data preprocessing I• Removing a baseline• basel=find(t<=0);• ts=ts-mean(ts(basel));
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6-2
-1
0
1
2
3
4
5
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6-3
-2
-1
0
1
2
3
4
![Page 16: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/16.jpg)
Data preprocessing II• Removing a trend• ts=detrend(ts);• subtracts best fitting line• detrend can be used to subtract mean: detrend(ts,’constant’)
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6-2
-1
0
1
2
3
4
5
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6-3
-2
-1
0
1
2
3
4
5
data 1 lineardata 2
![Page 17: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/17.jpg)
Data preprocessing III• Smoothing• ts=filter(ones(1,30)/30,1,ts); %mean filter, moving average• uses zeros at beginning! • => baseline correction or do not use first 30 samples
-0.2 0 0.2 0.4 0.60
1
2
3
4
5
6
7
![Page 18: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/18.jpg)
Data preprocessing III• introduces a shift! => either correct for it or • ts=filtfilt(ones(1,15)/15,1,ts); %mean filter, forward and reverse• no shift!• filter can take any smoothing kernel (gaussian, etc)
-0.2 0 0.2 0.4 0.6-3
-2
-1
0
1
2
3
4
-0.2 0 0.2 0.4 0.60
1
2
3
4
5
6
7shifted by 15 samples filtfilt
![Page 19: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/19.jpg)
Data preprocessing III• Smoothing• ts=medfilt1(ts,30); %median filter, takes into account the shift• uses 0 at beginning and end !
-0.2 -0.18 -0.16 -0.14 -0.12 -0.11
1.5
2
2.5
3
3.5
4
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.60
1
2
3
4
5
6
7
![Page 20: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/20.jpg)
Data preprocessing III• Smoothing• ts=sgolayfilt(ts,3,41); %Savitzky-Golay filter• fits 3rd order polynomial to frames of size 41• good at preserving high frequencies in the data
-0.2 0 0.2 0.4 0.60
1
2
3
4
5
6
7
![Page 21: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/21.jpg)
Data preprocessing III• Smoothing• compare unsmoothed and smoothed data• check for shift• check beginning (and end) of the smoothed time series
![Page 22: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/22.jpg)
Exercise 1
![Page 23: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/23.jpg)
Data preprocessing IV• Filtering• FIR-Filter (finite impulse
response)• stable• high filter order• usually have linear phase(phase change is proportional to
frequency)
• IIR-Filter (infinite impulse response)
• potentially unstable• low filter order• non-linear phase distortion• computationally efficient
![Page 24: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/24.jpg)
Data preprocessing IV• IIR-Filter:
– Butterworth– Elliptic– Chebychev Typ 1– Chebychev Typ 2– Bessel
• FIR-Filter:– fir1
![Page 25: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/25.jpg)
Data preprocessing IV• lowpass• highpass• bandpass• bandstop
0 5 10 15 20 25 30 35 40 45
-400
-350
-300
-250
-200
-150
-100
-50
0
Frequency (Hz)
Mag
nitu
de (d
B)
Magnitude Response (dB)
dB is logarithmic unit0dB = factor of 13dB = factor of 210dB= factor of 10
5 Hz lowpass
![Page 26: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/26.jpg)
Data preprocessing IV• lowpass• highpass• bandpass• bandstop
dB is logarithmic unit0dB = factor of 13dB = factor of 210dB= factor of 10
0 5 10 15 20 25 30 35 40 45
-100
-90
-80
-70
-60
-50
-40
-30
-20
-10
0
Frequency (Hz)
Mag
nitu
de (d
B)
Magnitude Response (dB)
30 Hz highpass
![Page 27: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/27.jpg)
Data preprocessing IV• lowpass• highpass• bandpass• bandstop
dB is logarithmic unit0dB = factor of 13dB = factor of 210dB= factor of 10
2-30 Hz bandpass
0 5 10 15 20 25 30 35 40 45
-40
-35
-30
-25
-20
-15
-10
-5
0
Frequency (Hz)
Mag
nitu
de (d
B)
Magnitude Response (dB)
![Page 28: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/28.jpg)
Data preprocessing IV• lowpass• highpass• bandpass• bandstop
dB is logarithmic unit0dB = factor of 13dB = factor of 210dB= factor of 10
30-40 Hz bandstop
0 5 10 15 20 25 30 35 40 45
-25
-20
-15
-10
-5
0
Frequency (Hz)
Mag
nitu
de (d
B)
Magnitude Response (dB)
![Page 29: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/29.jpg)
Simple design: FIR• [b]=fir1(4,2*4/sf); %4 Hz lowpass• [b]=fir1(4,2*4/sf,’high’); %4 Hz highpass• [b]=fir1(4,2*[4 10]/sf); %4-10 Hz bandpass• [b]=fir1(4,2*[4 10]/sf,’stop’); %4-10 Hz bandstop
• tsf=filter(b,1,ts);• tsf=filtfilt(b,1,ts); %forward and reverse
![Page 30: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/30.jpg)
Simple design: IIR• [b,a]=butter(4,2*4/sf); %4 Hz lowpass• [b,a]=butter(4,2*4/sf,’high’); %4 Hz highpass• [b,a]=butter(4,2*[4 10]/sf); %4-10 Hz bandpass• [b,a]=butter(4,2*[4 10]/sf,’stop’); %4-10 Hz bandstop
• tsf=filter(b,a,ts);• tsf=filtfilt(b,a,ts); %forward and reverse
![Page 31: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/31.jpg)
Simple Inspection
0 5 10 15 20 25 30 35 40 45 50-400
-300
-200
-100
0
Frequency (Hz)
Pha
se (d
egre
es)
0 5 10 15 20 25 30 35 40 45 50-300
-200
-100
0
Frequency (Hz)
Mag
nitu
de (d
B)
freqz(b,a,100,100);sf
number of frequencies
![Page 32: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/32.jpg)
Complex design• fdatool
– magnitude response– phase response– impulse response– compare filters– effect of changing filter order
![Page 33: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/33.jpg)
Filter artifacts• onset transients
0 2 4 6 8-6
-5
-4
-3
-2
-1
0
1
2x 10-3 1 Hz lowpass (4th order Butterworth)
0 2 4 6 8-6
-5
-4
-3
-2
-1
0
1
2x 10-3
filtfiltfilter
![Page 34: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/34.jpg)
Filter artifacts• ringing
with artifact
3.5 4 4.5-5
0
5
10
15
20x 10-3
3.5 4 4.5-5
0
5
10
15
20x 10-3
without artifact
![Page 35: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/35.jpg)
Filter artifacts• ringing
filter, 20 Hz lowpass (12th order Butterworth)
3.5 4 4.5-5
0
5
10
15
20x 10-3
3.5 4 4.5
-2
-1
0
1
2
3
4x 10-3
filtfilt
![Page 36: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/36.jpg)
Filter artifacts• beginning and end of filtered ts is distorted• filtering artifacts is dangerous• filtering may change the latency of effects!• filtering may change the phase
![Page 37: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/37.jpg)
Suggestions• be careful with low frequencies• use low order butterworth forward and reverse (to avoid
phase distortions)• carefully check beginning and end of filtered ts• make sure you don’t have artifacts in the data• use surrogate data (filtered noise)
![Page 38: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/38.jpg)
Data preprocessing V• Decimation• ts=decimate(ts,4);• decimate uses a lowpass filter to avoid aliasing artifacts
![Page 39: Time series analysis Matlab tutorial - University of Glasgojoachim/TSA/Time_series_analysis_tutorial1.pdf · Time series analysis Matlab tutorial Joachim Gross. Outline • Terminology](https://reader034.vdocuments.net/reader034/viewer/2022051508/5aa4aa897f8b9ac8748c39af/html5/thumbnails/39.jpg)
Exercises 2-4