assignment 6 - university of british columbiaschmidtm/courses/340-f15/t6.pdf · assignment 6....
TRANSCRIPT
Assignment 6
Question 1.1Odds ratio
Linear model
Objective function
Question 1.1Odds ratio
Linear model
Objective function
First step
replace withusing the fact that,
Second step
Apply “exp” on both sides to get rid of the log
Third stepSolve for and plug it into the objective function
Objective function
+ = 1
Starting from equation 1
Question 1.2 One-vs-all Logistic Regression
Question 1.2 One-vs-all Logistic Regression
use findMin with LogisticLoss instead(see assignment 4 for the LogisticLoss function)
Matrix WMatrix X Matrix y
=*
n samples, k classes, p features
dimensions = ? dimensions = ? dimensions = ?
Question 1.2 One-vs-all Logistic Regression
use findMin with LogisticLoss instead(see assignment 4 for the LogisticLoss function)
Matrix WMatrix X Matrix y
=*
n samples, k classes, p features
dimensions = n x p dimensions = p x k dimensions = n x k
Question 1.3 Softmax Loss and derivative● The softmax probability function is given as,
● Assume
● Convert y to binary form; i.e.
Question 1.3 Softmax Loss and derivative● The softmax probability function is given as,
● Assume
● Convert y to binary form; i.e.
● Therefore,
● C is the number of classes ● W j refers to column j of W● is the binary predicted target value for class ‘c’ of
sample ‘i’
Question 1.3 Softmax Loss and derivative● The softmax probability function is now formulated as,
Question 1.3 Softmax Loss and derivative● The softmax probability function is now formulated as,
Only one of these terms is non-zero for any training example ‘i’
Question 1.3 Softmax Loss and derivative● The softmax probability function is now formulated as,
(apply log)
(multiply by -1)
● The negative logarithm of the probability:
= ?
= ?
● The derivative of the negative log probability with respect to can be broken into two cases :
Question 1.3 Softmax Loss and derivative● The softmax probability function is now formulated as,
(apply log)
(multiply by -1)
● The negative logarithm of the probability:
= ?
= ?
● The derivative of the negative log probability with respect to can be broken into two cases :
is column c of row j of Wwhich corresponds to the coefficient offeature j of class c
Question 1.3 Softmax Loss and derivative● The softmax probability function is now formulated as,
(apply log)
(multiply by -1)
● The negative logarithm of the probability:
= ?
= ?
● The derivative of the negative log probability with respect to :
Hint: Use the indicator function to distinguishbetween the two cases
Question 1.4 - Softmax Classifier
Question 1.4 - Softmax Classifier
Use findMin instead with the softmax loss grad function
Question 1.4 - Softmax Classifier
Change the contents of the green box on the left using that of the green boxes on the right
Question 1.4 - Softmax Classifier
Change the contents of the green box on the left using that of the green boxes on the right
Question 1.5 - Cost of Multinomial Logistic Regression
Time complexity for processing one example = ?
Time complexity for predicting one example = ?
Question 1.5 - Cost of Multinomial Logistic Regression
Question 2 random walk
un
un
-1
un
1
1
1
2 3
4
56
1
2
3
4
5
6
1 2 3 4 5 6
0 1 0 0 0 1
1 0 1 0 0 0
0 1 0 0 0 0
0 0 0 0 1 0
0 0 0 1 0 1
1 0 0 0 1 0
matrix AAdjacency matrix
3 1
4 1
5 -1
matrix labelList
Question 2 random walk
un
un
-1
un
1
1
1
2 3
4
56
1
2
3
4
5
6
1 2 3 4 5 6
0 1 0 0 0 1
1 0 1 0 0 0
0 1 0 0 0 0
0 0 0 0 1 0
0 0 0 1 0 1
1 0 0 0 1 0
matrix AAdjacency matrix
v=
3 1
4 1
5 -1
matrix labelList
Question 2 random walk
un
un
-1
un
1
1
1
2 3
4
56
1
2
3
4
5
6
1 2 3 4 5 6
0 1 0 0 0 1
1 0 1 0 0 0
0 1 0 0 0 0
0 0 0 0 1 0
0 0 0 1 0 1
1 0 0 0 1 0
matrix AAdjacency matrix
prob = 1/2
prob = 1/2v=
3 1
4 1
5 -1
matrix labelList
Question 2 random walk
un
un
-1
un
1
1
1
2 3
4
56
1
2
3
4
5
6
1 2 3 4 5 6
0 1 0 0 0 1
1 0 1 0 0 0
0 1 0 0 0 0
0 0 0 0 1 0
0 0 0 1 0 1
1 0 0 0 1 0
matrix AAdjacency matrix
v=
3 1
4 1
5 -1
matrix labelList
Question 2 random walk
un
un -1
un
1
1
1
2 3
4
56
1
2
3
4
5
6
1 2 3 4 5 6
0 1 0 0 0 1
1 0 1 0 0 0
0 1 0 0 0 0
0 0 0 0 1 0
0 0 0 1 0 1
1 0 0 0 1 0
matrix AAdjacency matrix
v=prob = 1/2
prob = 1/2
3 1
4 1
5 -1
matrix labelList
Question 2 random walk
un
un
-1
un
1
1
1
2 3
456
1
2
3
4
5
6
1 2 3 4 5 6
0 1 0 0 0 1
1 0 1 0 0 0
0 1 0 0 0 0
0 0 0 0 1 0
0 0 0 1 0 1
1 0 0 0 1 0
matrix AAdjacency matrix
v=
yhat = -1 withprob = 1/(1+2) = 1/3
prob = 1/3
prob = 1/3
3 1
4 1
5 -1
matrix labelList
Question 2 random walk
un
un
-1
-1
1
1
i=1
2 3
456
1
2
3
4
5
6
1 2 3 4 5 6
0 1 0 0 0 1
1 0 1 0 0 0
0 1 0 0 0 0
0 0 0 0 1 0
0 0 0 1 0 1
1 0 0 0 1 0
matrix AAdjacency matrix
v=
yhat = -1 withprob = 1/(1+2) = 1/3
prob = 1/3
prob = 1/3
3 1
4 1
5 -1
matrix labelList