basics of neural network programming - stanford universitycs230.stanford.edu/files/c1m2.pdf ·...
TRANSCRIPT
![Page 1: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/1.jpg)
Basics of Neural Network Programming
Binary Classificationdeeplearning.ai
![Page 2: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/2.jpg)
Andrew Ng
1 (cat) vs 0 (non cat)
255 134 93 22123 94 83 234 44 187 3034 76 232 12467 83 194 142
255 134 202 22123 94 83 434 44 187 19234 76 232 3467 83 194 94
255 231 42 22123 94 83 234 44 187 9234 76 232 12467 83 194 202
RedGreen
Blue
Binary Classification
![Page 3: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/3.jpg)
Andrew Ng
Notation
![Page 4: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/4.jpg)
Basics of Neural Network Programming
Logistic Regressiondeeplearning.ai
![Page 5: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/5.jpg)
Andrew Ng
Logistic Regression
![Page 6: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/6.jpg)
Basics of Neural Network Programming
Logistic Regressioncost functiondeeplearning.ai
![Page 7: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/7.jpg)
Andrew Ng
Logistic Regression cost function!" = % &!' + ) , where % * = "
"#$+,
Given ('(.), !(.)),…,('(1), !(1)) ,want !"(2) ≈ ! 2 .
Loss (error) function:
![Page 8: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/8.jpg)
Basics of Neural Network Programming
Gradient Descentdeeplearning.ai
![Page 9: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/9.jpg)
Andrew Ng
Gradient DescentRecap: !" = % &'( + * , % + = ,
,-./0
1 &, * =1456
78,ℒ(!" 7 , !(7)) =− 1
456
78,!(7) log !" 7 + (1 − !(7)) log(1 − !" 7 )
Want to find &, * that minimize 1 &, *
*
1 &, *
&
![Page 10: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/10.jpg)
Andrew Ng
Gradient Descent
!
![Page 11: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/11.jpg)
Basics of Neural Network Programming
Derivativesdeeplearning.ai
![Page 12: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/12.jpg)
Andrew Ng
Intuition about derivatives! " = 3"
"
![Page 13: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/13.jpg)
Basics of Neural Network Programming
More derivatives examplesdeeplearning.ai
![Page 14: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/14.jpg)
Andrew Ng
Intuition about derivatives
! " = "$
"
![Page 15: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/15.jpg)
Andrew Ng
More derivative examples
![Page 16: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/16.jpg)
Basics of Neural Network Programming
Computation Graphdeeplearning.ai
![Page 17: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/17.jpg)
Andrew Ng
Computation Graph
![Page 18: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/18.jpg)
Basics of Neural Network Programming
Derivatives with a Computation Graphdeeplearning.ai
![Page 19: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/19.jpg)
Andrew Ng
Computing derivatives
!="#$ = & + ! ) = 3$6
11 33& = 5
# = 2" = 3
![Page 20: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/20.jpg)
Andrew Ng
Computing derivatives
!="#$ = & + ! ) = 3$6
11 33& = 5
# = 2" = 3
![Page 21: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/21.jpg)
Basics of Neural Network Programming
Logistic Regression Gradient descentdeeplearning.ai
![Page 22: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/22.jpg)
Andrew Ng
! = $%& + ()* = + = ,(!)ℒ +, ) = −() log(+) + (1 − )) log(1 − +))
Logistic regression recap
![Page 23: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/23.jpg)
Andrew Ng
Logistic regression derivatives
! = $%&% + $(&( + )
&%$%&($(b
* = +(!) ℒ(a, 1)
![Page 24: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/24.jpg)
Basics of Neural Network Programming
Gradient descenton m examplesdeeplearning.ai
![Page 25: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/25.jpg)
Andrew Ng
Logistic regression on m examples
![Page 26: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/26.jpg)
Andrew Ng
Logistic regression on m examples
![Page 27: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/27.jpg)
Basics of Neural Network Programming
Vectorizationdeeplearning.ai
![Page 28: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/28.jpg)
Andrew Ng
What is vectorization?
![Page 29: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/29.jpg)
Basics of Neural Network Programming
More vectorizationexamplesdeeplearning.ai
![Page 30: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/30.jpg)
Andrew Ng
Neural network programming guideline
Whenever possible, avoid explicit for-loops.
![Page 31: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/31.jpg)
Andrew Ng
Vectors and matrix valued functionsSay you need to apply the exponential operation on every element of a matrix/vector.
! = !$⋮!&
u[i]=math.exp(v[i])
u = np.zeros((n,1))for i in range(n):
![Page 32: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/32.jpg)
Andrew Ng
Logistic regression derivativesJ = 0, dw1 = 0, dw2 = 0, db = 0for i = 1 to n:
!(") = "$#(") + %&(") = '(!("))*+= − -(") log -1 " + (1 − - " ) log(1 − -1 " )d!(") = &(")(1 − &("))d"%+= #%
(")d!(")d"'+= #'(")d!(")db += d!(")
J = J/m, d"% = d"%/m, d"' = d"'/m, db = db/m
![Page 33: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/33.jpg)
Basics of Neural Network Programming
Vectorizing Logistic Regressiondeeplearning.ai
![Page 34: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/34.jpg)
Andrew Ng
Vectorizing Logistic Regression!(#) = &'((#) + *+(#) = ,(!(#))
!(-) = &'((-) + *+(-) = ,(!(-))
!(.) = &'((.) + *+(.) = ,(!(.))
![Page 35: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/35.jpg)
Basics of Neural Network Programming
Vectorizing Logistic Regression’s Gradient
Computationdeeplearning.ai
![Page 36: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/36.jpg)
Andrew Ng
Vectorizing Logistic Regression
![Page 37: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/37.jpg)
Andrew Ng
Implementing Logistic Regression
J = 0, d!! = 0, d!" = 0, db = 0for i = 1 to m:
"($) = !&#($) + %&($) = '("($))*+= − -($) log & $ + (1 − - $ ) log(1 − & $ )d"($) = &($) −-($)d!!+= #!($)d"($)d!"+= #"($)d"($)db += d"($)
J = J/m, d!! = d!!/m, d!" = d!"/mdb = db/m
![Page 38: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/38.jpg)
Basics of Neural Network Programming
Broadcasting inPythondeeplearning.ai
![Page 39: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/39.jpg)
Broadcasting example
cal = A.sum(axis = 0)percentage = 100*A/(cal.reshape(1,4))
Apples Beef Eggs PotatoesCarb
Fat
56.0 0.0 4.4 68.01.2 104.0 52.0 8.01.8 135.0 99.0 0.9
Protein
Calories from Carbs, Proteins, Fats in 100g of different foods:
![Page 40: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/40.jpg)
101 102 103204 205 206
1 2 34 5 6
100200+ =
101 202 303104 205 306100 200 3001 2 3
4 5 6 + =
1234
100101102103104
+ =
Broadcasting example
![Page 41: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/41.jpg)
General Principle
![Page 42: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/42.jpg)
BasicsofNeuralNetworkProgramming
Explanationoflogisticregressioncostfunction
(Optional)deeplearning.ai
![Page 43: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/43.jpg)
AndrewNg
Logistic regression cost function
![Page 44: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/44.jpg)
AndrewNg
If$ = 1: ( $ ) = $*If$ = 0: ( $ ) = 1 − $*
Logistic regression cost function
![Page 45: Basics of Neural Network Programming - Stanford Universitycs230.stanford.edu/files/C1M2.pdf · Basics of Neural Network Programming Explanation of logistic regression cost function](https://reader035.vdocuments.net/reader035/viewer/2022081620/6116e5c0950b8f17a4652eb5/html5/thumbnails/45.jpg)
AndrewNg
Cost on m examples