cs 357: numerical methods lecture 12: qr decomposition ... · lecture 12: qr decomposition least...
TRANSCRIPT
![Page 1: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/1.jpg)
CS 357: Numerical Methods
Lecture 12:
QR Decomposition
Least Squares
Eric Shaffer
![Page 2: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/2.jpg)
QR review
Reduced QR
Let A by m by n and A=QR
What is the shape of Q?
What is the shape of R?
What are they for Full QR?
![Page 3: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/3.jpg)
QR review
What is special about Q?
What space do the q1,…,qn columns span?
![Page 4: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/4.jpg)
Solving Linear Systems with QR
Ax=b for A and n by n matrix
If A=QR, how can we solve for x?
What is the computational cost?
![Page 5: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/5.jpg)
Least Squares with QR
Tall and skinny systems
No exact solution….
![Page 6: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/6.jpg)
“Best” solution
Find some x so that Ax is as close to b as we can get
Take the difference…and measure the magnitude using a norm
We want to minimize 𝐴𝑥 − 𝑏 22
The residual is 𝑟 = 𝐴𝑥 − 𝑏
We are minimizing the sum of squares of the residual
This process is called a least squares problem
![Page 7: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/7.jpg)
Notation
Lots of different notation that means the same thing
Find x so that 𝐴𝑥 − 𝑏 22 is minimized
𝑥 = 𝑎𝑟𝑔𝑚𝑖𝑛𝑥 𝐴𝑥 − 𝑏 22
𝐴𝑥 ≅ 𝑏
![Page 8: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/8.jpg)
QR and Least Squares
![Page 9: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/9.jpg)
QR and Least Squares
![Page 10: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/10.jpg)
The Normal Equations
Another way of solving Least Squares
![Page 11: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/11.jpg)
Question….
Why is it called “linear least squares”
![Page 12: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/12.jpg)
Data Fitting
Lots of interesting problems lack an exact solution….
Fit a function to a set of points….
![Page 13: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/13.jpg)
Data Fitting
Lots of interesting problems lack an exact solution….
Fit a function to a set of points….
![Page 14: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/14.jpg)
Fitting a line
![Page 15: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/15.jpg)
Fitting a curve
![Page 16: CS 357: Numerical Methods Lecture 12: QR Decomposition ... · Lecture 12: QR Decomposition Least Squares Eric Shaffer. QR review ... Data Fitting Lots of interesting problems lack](https://reader036.vdocuments.net/reader036/viewer/2022070801/5f028be47e708231d404cd2c/html5/thumbnails/16.jpg)
Fitting a curve