information theory + review & applications · information theory + review & applications...

1

Information Theory + Review & Applications

© 2014 School of Information Technology and Electrical Engineering at The University of Queensland

TexPoint fonts used in EMF.

Read the TexPoint manual before you delete this box.: AAAAA

http://elec3004.com

• Week 13 is an “Open Tutorial / Lab” – You are welcome to work on *.*

ELEC 3004: Systems 1 June 2015 2

Lab 4++

http://itee.uq.edu.au/~metr4202/

http://itee.uq.edu.au/~metr4202/

http://creativecommons.org/licenses/by-nc-sa/3.0/au/deed.en_US

http://elec3004.com/

http://elec3004.com/

2

PS 4 & PS 5 Due Soon

• PS 4: June 2 (Tomorrow!)

• Good Progress!

• PS 5: Saturday, June 6

• Good Progress?


ELEC 3004:

A Review

1 June 2015 ELEC 3004: Systems 4

3

AKA ELEC 3004:

What do I need to

know about *.* ???


To Review: Back to the Beginning…Lecture 1 Slide 9

• Systems

• Signal Abstractions

• Signals as Vectors / Systems as Maps

• Linear Systems and Their Properties

• LTI Systems

• Autonomous Linear Dynamical Systems

• Convolution

• FIR & IIR Systems

• Frequency domain

• Fourier Transform (CT)

• Fourier Transform (DT)

• Even and Odd Signals

• Likelihood

• Causality

• Impulse Response

• Root Locus

• Bode Functions

• Left-hand Plane

• Frequency Response

• Discrete Time

• Continuous Time

• Laplace Transformation

• Feedback and Control

• Additional Applications

• Linear Functions

• Linear Algebra Review

• Least Squares

• Least Squares Problems

• Least Squares Applications

• Matrix Decomposition and Linear

Algebra

• Regularized Least Squares

• Least-squares

• Least-squares applications

• Orthonormal sets of vectors

• Eigenvectors and diagonalization

• Linear dynamical systems with inputs

and outputs

• Symmetric matrices, quadratic forms,

matrix norm, and SVD

• Controllability and state transfer

• Observability and state estimation

• And that, of course,

Linear Systems are Cool!


4

Lots of Stuff To Cover… • Systems

• Signal Abstractions

• Signals as Vectors / Systems as Maps

• Linear Systems and Their Properties

• LTI Systems

• Autonomous Linear Dynamical Systems

• Convolution

• FIR & IIR Systems

• Frequency domain

• Fourier Transform (CT)

• Fourier Transform (DT)

• Even and Odd Signals

• Likelihood

• Causality

• Impulse Response

• Root Locus

• Bode Functions

• Left-hand Plane

• Frequency Response

• Discrete Time

• Continuous Time

• Laplace Transformation

• Feedback and Control

• Additional Applications

• Linear Functions

• Linear Algebra Review

• Least Squares

• Least Squares Problems

• Least Squares Applications

• Matrix Decomposition and Linear

Algebra

• Regularized Least Squares

• Least-squares

• Least-squares applications

• Orthonormal sets of vectors

• Eigenvectors and diagonalization

• Linear dynamical systems with inputs

and outputs

• Symmetric matrices, quadratic forms,

matrix norm, and SVD

• Controllability and state transfer

• Observability and state estimation

• And that, of course,

Linear Systems are Cool!

✔ ✔ ✔

✔ ✔ ✔

✔ ✔ ✔

✔ ✔ ✔ ✔ ✗

✔

✔ ✔ ✔

✔

✔

✔

✔

✔

✔

✔

✔

✔

✔ ✗

✔

✗

✗

✔

✔ ✗

✔

✔

✔

✔ ✔


• System?

• ODE?

• Linear Algebra?

• Joy?

• Excitement?

• Shock and Awe??

Review • What do you think when you see?

Linear algebra provides the tools/foundation for

working with (linear) differential equations.


5

• Signals are vectors. Systems are matrices.

Signals & Systems

Linear algebra provides the tools/foundation for

working with (linear) differential equations.


Linear Systems


6

• Signals Are Vectors

• Systems Are Matricies

Linear Systems in 1-Slide

F(x) signal

(input)

F(…)=system

signal

(output)


From Last Week:

• LDS:

To Review:

• Continuous-time linear dynamical system (CT LDS):

• t ∈ ℝ denotes time

• x(t) ∈ ℝn is the state (vector)

• u(t) ∈ ℝm is the input or control

• y(t) ∈ ℝp is the output

Types of Linear Systems


7

• LDS:

• A(t) ∈ ℝn×n is the dynamics matrix

• B(t) ∈ ℝn×m is the input matrix

• C(t) ∈ ℝp×n is the output or sensor matrix

• D(t) ∈ ℝp×m is the feedthrough matrix

state equations, or “m-input, n-state, p-output’ LDS

Types of Linear Systems


• Causal = The output before some time t does not depend on

the input after time t.

Given:

For:

Then for a T>0:

Causality: Looking at this from the output’s perspective…

if:

then:

Causal Noncausal

else:


8

Is it Useful?

Yes. (For example … Next Year – ELEC/METR 3800)


It Can Rock Your Boat Gently Down The Stream: IMU Deaduced Reckoning (Navigation)

Idea: Integrate your motion (twice for

𝑥 → 𝑥 and once for 𝜃 → 𝜃)

Problem:

• (DC) bias in accelerometer drift

Solution:

• IIR Bandpass filter (0.1-10 Hz)

A = [1.0000 -5.8235 14.1332 -18.2966 13.3259 -5.1772 0.8382];

B = [0.9155 -5.4933 13.7331 -18.3109 13.7331 -5.4933 0.9155];

0 5 10 15 20 25 30 35 40 45

-70

-60

-50

-40

-30

-20

-10

0

Frequency (Hz)

Mag

nitu

de (d

B)

Magnitude Response (dB)


9

It Can Rock Your Boat Gently Down The Stream: IMU Deaduced Reckoning (Navigation) [2]

Solution:


Signal Processing


10

• Aliasing - through sampling, two entirely different analogue

sinusoids take on the same “discrete time” identity

For f[k]=cosΩk, Ω=ωT:

The period has to be less than Fh (highest frequency):

Thus:

ωf: aliased frequency:

Aliasing


• Zero-Order Hold [ZOH]

Reconstruction


11

Properties:

• Commutative:

• Distributive:

• Associative:

• Shift:

if f1(t)*f2(t)=c(t), then f1(t-T)*f2(t)= f1(t)*f2(t-T)=c(t-T)

• Identity (Convolution with an Impulse):

• Total Width:

Convolution & Properties

Based on Lathi, SPLS, Sec 2.4-1


• Convolution systems are linear:

• Convolution systems are causal: the output y(t) at time t

depends only on past inputs

• Convolution systems are time-invariant

(if we shift the signal, the output similarly shifts)

Convolution & Properties [II]


12

• Composition of convolution systems corresponds to: – multiplication of transfer functions

– convolution of impulse responses

• Thus: – We can manipulate block diagrams with transfer functions as if

they were simple gains

– convolution systems commute with each other

Convolution & Properties [III]


For c(τ)= :

1. Keep the function f (τ) fixed

2. Flip (invert) the function g(τ) about the vertical axis (τ=0)

= this is g(-τ)

3. Shift this frame (g(-τ)) along τ (horizontal axis) by t0.

= this is g(t0 -τ)

For c(t0):

4. c(t0) = the area under the product of f (τ) and g(t0 -τ)

5. Repeat this procedure, shifting the frame by different values

(positive and negative) to obtain c(t) for all values of t.

Graphical Understanding of Convolution


13

Matrix Formulation of Convolution

3

8

14

20

26

14

5

1 2 3 0 0 0 0

0 1 2 3 0 0 0

0 0 1 2 3 0 0

0 0 0 1 2 3 0

0 0 0 0 1 2 3

0 0 0 0 0 1 2

0 0 0 0 0 0 1

0 0

0 0

0 0

0 0

0 0

3 0

2 3

0

0

1

2

3

4

5

0

0

.

y Hx

Toeplitz Matrix


The z-transform

• The discrete equivalent is the z-Transform†:

𝒵 𝑓 𝑘 = 𝑓(𝑘)𝑧−𝑘∞

𝑘=0

= 𝐹 𝑧

and

𝒵 𝑓 𝑘 − 1 = 𝑧−1𝐹 𝑧

Convenient!

†This is not an approximation, but approximations are easier to derive

F(z) y(k) x(k)


14

The z-Transform

• It is defined by:

Or in the Laplace domain:

𝑧 = 𝑒𝑠𝑇

• Thus: or

• I.E., It’s a discrete version of the Laplace:

𝑓 𝑘𝑇 = 𝑒−𝑎𝑘𝑇 ⇒ 𝒵 𝑓 𝑘 =𝑧

𝑧 − 𝑒−𝑎𝑇


Digital Controls


15

Implementation of Digital PID Controllers

Source: Dorf & Bishop, Modern Control Systems, §13.9, pp. 1030-1


Implementation of Digital PID Controllers (2)

Source: Dorf & Bishop, Modern Control Systems, §13.9, pp. 1030-1


16

Final Exam Notes


• The final exam is worth 40% total

• Friday, June 26 from 2:30pm to 5:30pm (in various rooms, see online)

– It is managed by the UQ Central Examinations Office. • All logistics questions to them (Please!)

• Closed Book

• Students are additionally permitted a doubled-sided A4

equation sheet of their own design

• In addition it comes with an standard equation sheet (see online)

• Past/Practice exams online: 2014 final exam, 2013 final

exam, and a general practice exam.


Final Exam Notes

https://my.uq.edu.au/my_login_screen.html?id=396

https://my.uq.edu.au/my_login_screen.html?id=396

http://robotics.itee.uq.edu.au/~elec3004/2013/tutes/ELEC3004_Final_FormulaTable.pdf

http://robotics.itee.uq.edu.au/~elec3004/2014/assignments/ELEC3004_EXAM_2014.pdf

http://robotics.itee.uq.edu.au/~elec3004/2014/assignments/ELEC3004_EXAM_2014.pdf

http://robotics.itee.uq.edu.au/~elec3004/2013/assignments/ELEC3004_final_exam.v2013.pdf





17

• Three Sections | 180 Points: – Digital Linear Dynamical Systems -- 50 Points (28 %)

– Digital Processing/Filtering of Signals -- 60 Points (33 %)

– Digital & State-Space Control -- 70 Points (39 %)

Also

⇨ Please answer ALL questions ⇦

⇨ ALL Answers MUST Be Justified ⇦

& Remember:

PLEASE RECORD ALL ANSWERS IN THE ANSWER BOOKLET

( Any material not in Answer Booklet(s) will not be seen! )


Final Exam Structure

• http://robotics.itee.uq.edu.au/~elec3004/2014/lectures/SomeRe

viewNotes.ELEC3004.pdf

• Select pages (55) from the textbooks used in the course


Final Exam “Review” Reader

http://robotics.itee.uq.edu.au/~elec3004/2014/lectures/SomeReviewNotes.ELEC3004.pdf













information theory + review & applications · information theory + review & applications...

Documents