andrew ng linear regression with one variable model representation machine learning
TRANSCRIPT
![Page 1: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/1.jpg)
Andrew Ng
Linear regression with one variable
Model representation
Machine Learning
![Page 2: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/2.jpg)
Andrew Ng
500 1000 1500 2000 2500 30000
100000
200000
300000
400000
500000
500 1000 1500 2000 2500 30000
100000
200000
300000
400000
500000Housing Prices(Portland, OR)
Price(in 1000s of dollars)
Size (feet2)
Supervised Learning
Given the “right answer” for each example in the data.
Regression Problem
Predict real-valued outputClassification : Discrete-valued output
220
1250
![Page 3: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/3.jpg)
Andrew Ng
Notation:
m = Number of training examples x’s = “input” variable / features y’s = “output” variable / “target” variable
Size in feet2 (x) Price ($) in 1000's (y)2104 4601416 2321534 315852 178… …
Training set ofhousing prices(Portland, OR)
(x, y) – one training example(x(i), y(i)) – ith trainingg example
x(1) = 2104x(2) = 1416y(1) = 460
m
![Page 4: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/4.jpg)
Andrew Ng
Training Set
Learning Algorithm
How do we represent h ?
h maps from x’s to y’s
Size of house
x
h
hypothesis
Estimated price
Estimated value Linear regression with one variable.
Univariate linear regression.
One variable
![Page 5: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/5.jpg)
Andrew Ng
Cost function
Machine Learning
Linear regression with one variable
![Page 6: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/6.jpg)
Andrew Ng
How to choose ‘s ?
Training Set
Hypothesis:
‘s: Parameters
Size in feet2 (x) Price ($) in 1000's (y)2104 4601416 2321534 315852 178… …
![Page 7: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/7.jpg)
Andrew Ng
0 1 2 30
1
2
3
0 1 2 30
1
2
3
0 1 2 30
1
2
3
h(x) = 1.5 + 0·x h(x) = 0.5·x
h(x) = 1 + 0.5·x
![Page 8: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/8.jpg)
Andrew Ng
y
x
Idea: Choose so that is close to for our training examples
h(x) = +
(x(i), y(i))
minimize Θ0Θ1Θ0, Θ1
J() =
Minimize J() : Cost Function Θ0Θ1
Squared error function
![Page 9: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/9.jpg)
Andrew Ng
Cost functionintuition I
Machine Learning
Linear regression with one variable
![Page 10: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/10.jpg)
Andrew Ng
Hypothesis:
Parameters:
Cost Function:
Goal:
Simplified
h(x) h(x) = 0
![Page 11: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/11.jpg)
Andrew Ng
0 1 2 30
1
2
3
y
x
(for fixed , this is a function of x) (function of the parameter )
-0.5 0 0.5 1 1.5 2 2.50
1
2
3
J() =
=
=
𝐽 (1 )=0
![Page 12: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/12.jpg)
Andrew Ng
0 1 2 30
1
2
3
y
x
(for fixed , this is a function of x) (function of the parameter )
-0.5 0 0.5 1 1.5 2 2.50
1
2
3
J() =
= (3.5) = 0.58
y ( i )
hΘ ( x ( i ))
![Page 13: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/13.jpg)
Andrew Ng
-0.5 0 0.5 1 1.5 2 2.50
1
2
3
y
x
(for fixed , this is a function of x) (function of the parameter )
0 1 2 30
1
2
3
J() =
= = 2.3
![Page 14: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/14.jpg)
Andrew Ng
Cost functionintuition II
Machine Learning
Linear regression with one variable
![Page 15: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/15.jpg)
Andrew Ng
Hypothesis:
Parameters:
Cost Function:
Goal:
![Page 16: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/16.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
500 1000 1500 2000 2500 30000
100000
200000
300000
400000
500000
Price ($) in 1000’s
Size in feet2 (x)
= 50
= 0.06
![Page 17: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/17.jpg)
Andrew Ng
Contour plots
![Page 18: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/18.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 19: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/19.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
h(x) = 360 + 0·x
= 360
= 0
![Page 20: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/20.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 21: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/21.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 22: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/22.jpg)
Andrew Ng
Gradient descent
Machine Learning
Linear regression with one variable
![Page 23: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/23.jpg)
Andrew Ng
Have some function
Want
Outline:
• Start with some
• Keep changing to reduce
until we hopefully end up at a minimum
![Page 24: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/24.jpg)
Andrew Ng
1
0
J(0,1)
![Page 25: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/25.jpg)
Andrew Ng
0
1
J(0,1)
![Page 26: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/26.jpg)
Andrew Ng
Gradient descent algorithm
Correct: Simultaneous update Incorrect:
Simultaneously update & Learning rate
assignmenta:=b
![Page 27: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/27.jpg)
Andrew Ng
Gradient descentintuition
Machine Learning
Linear regression with one variable
![Page 28: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/28.jpg)
Andrew Ng
Gradient descent algorithm
Learning rate derivative
![Page 29: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/29.jpg)
Andrew Ng
:= -
:= - ≥0
:= -
:= - ≤0
![Page 30: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/30.jpg)
Andrew Ng
If α is too small, gradient descent can be slow.
If α is too large, gradient descent can overshoot the minimum. It may fail to converge, or even diverge.
![Page 31: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/31.jpg)
Andrew Ng
at local optima
Current value of
![Page 32: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/32.jpg)
Andrew Ng
Gradient descent can converge to a local minimum, even with the learning rate α fixed.
As we approach a local minimum, gradient descent will automatically take smaller steps. So, no need to decrease α over time.
![Page 33: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/33.jpg)
Andrew Ng
Gradient descent for linear regression
Machine Learning
Linear regression with one variable
![Page 34: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/34.jpg)
Andrew Ng
Gradient descent algorithm Linear Regression Model
![Page 35: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/35.jpg)
Andrew Ng
=
![Page 36: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/36.jpg)
Andrew Ng
Gradient descent algorithm
update and
simultaneously
![Page 37: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/37.jpg)
Andrew Ng
1
0
J(0,1)
![Page 38: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/38.jpg)
Andrew Ng
Convex function
Bowl-shaped
![Page 39: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/39.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 40: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/40.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 41: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/41.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 42: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/42.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 43: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/43.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 44: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/44.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 45: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/45.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 46: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/46.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 47: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/47.jpg)
Andrew Ng
(for fixed , this is a function of x) (function of the parameters )
![Page 48: Andrew Ng Linear regression with one variable Model representation Machine Learning](https://reader033.vdocuments.net/reader033/viewer/2022061520/56649ebf5503460f94bca538/html5/thumbnails/48.jpg)
Andrew Ng
“Batch” Gradient Descent
“Batch”: Each step of gradient descent uses all the training examples.