machine learning sessions parisa...
TRANSCRIPT
![Page 1: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/1.jpg)
Linear ClassificationMachine Learning Sessions
Parisa Abedi
![Page 2: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/2.jpg)
Definition
Classification:
Use an object characteristics to identify which class/category it belongs to
Example:
a new email is ‘spam’ or ‘non-spam’
A patient diagnosed with a disease or not
Classification is an example of pattern recognition
![Page 3: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/3.jpg)
Definition
![Page 4: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/4.jpg)
Definition
Linear classification
A classification algorithm (Classifier) that makes its classification based on a linear
predictor function combining a set of weights with the feature vector
Decision boundaries is flat
Line, plane, ….
May involve non-linear operations
![Page 5: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/5.jpg)
Different approaches
Explicitly creating the discriminant function (Discriminant function)
Perceptron
Support vector machine
Probabilistic approach
Model the posterior distribution
Algorithms
Logistic regression
![Page 6: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/6.jpg)
Discriminant functions
Two classes: y 𝑋 = 𝑤𝑇𝑋 + 𝑤0
More than two classes (K
classes):
There are K indicators
𝑌𝑘 = 1 if G = K, else 0
𝑌 = 𝑌1, … , 𝑌𝐾
Example: (0,0,1,0) for a feature in
class 3 when there are 4 classes!
![Page 7: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/7.jpg)
Discriminant functions
Least squares
Simultaneously fit a linear regression model to each of the columns of Y
Weights will have a close form 𝑊 = (𝑋𝑇𝑋)−1𝑋𝑇𝑌
Classify a new observation x:
For each class calculate the f(x) = W.X
Select the class with higher value for f(x)
![Page 8: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/8.jpg)
Discriminant functions
Least squares
Works well
Linearly separable
Few outliers
K = 2
![Page 9: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/9.jpg)
Discriminant functions
Least squares
Not good for
K ≥ 3
![Page 10: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/10.jpg)
Discriminant functions
Fischer’s method
Dimensionality reduction
Still need a decision rule
![Page 11: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/11.jpg)
Discriminant functions
Perceptron
How to drive w?
Choose some initial weights
For each example of 𝑥𝑗in training set calculate the y
Update the weights
![Page 12: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/12.jpg)
Discriminant functions
Perceptron
![Page 13: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/13.jpg)
Discriminant functions
Perceptron
Works well for linearly
separable
It’s fast
Can work online
Not able to choose the
best boundaries
![Page 14: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/14.jpg)
Probabilistic approaches
Determine the class-conditional densities for each class
Bayes’ theorem
![Page 15: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/15.jpg)
Probabilistic approaches
Use logistic sigmoid function
![Page 16: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/16.jpg)
Probabilistic approaches
Gaussian input with common covariance matrix
In the case of two classes
![Page 17: Machine Learning Sessions Parisa Abedicompneurosci.com/wiki/images/c/c0/Linear_Classification.pdf · Linear Classification Machine Learning Sessions Parisa Abedi. Definition Classification:](https://reader030.vdocuments.net/reader030/viewer/2022041107/5f0a97617e708231d42c62ac/html5/thumbnails/17.jpg)
Probabilistic approaches
Estimate the parameters
Maximum likelihood estimation
Data set
𝑡𝑛 = 1 denotes class 𝐶1