introduction to machine learning
DESCRIPTION
Introduction to machine learning. The audience is expected to have no prior knowledge about machine learningTRANSCRIPT
Shao-Chuan Wang, Academia Sinica 1
Introduction to Machine Learning
Shao-Chuan WangResearch Center for IT Innovation
Multimedia and Machine Learning LabAcademia Sinica
中央研究院資訊科技創新研究中心多媒體與機器學習實驗室
NTNU
Shao-Chuan Wang, Academia Sinica 2
Outline
• What is involved in intelligence?• Why is machine learning important?• What can machine learning do?• Overview of machine learning applications• Challenges of machine learning• Future of machine learning
Shao-Chuan Wang, Academia Sinica 3
What Is Intelligence?
Shao-Chuan Wang, Academia Sinica 4
What Is Involved in Intelligence?
• From Merriam-Webster: – “intelligence”: (1) the ability to learn or understand or
to deal with new or trying situations. (2) the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria
• Abstraction (finding the common patterns)
– V.S.• Adaptation– Learning is dynamic; e.g. a computer chess.
Shao-Chuan Wang, Academia Sinica 5
Why Is Machine Learning Important?(1/4)
The explosion of data
Shao-Chuan Wang, Academia Sinica 6
Why Is Machine Learning Important?(2/4)
Some places areNOT for humans
Shao-Chuan Wang, Academia Sinica 7
Why Is Machine Learning Important?(3/4)
Machine learning can help us understand human learning
Shao-Chuan Wang, Academia Sinica 8
Why Is Machine Learning Important?(4/4)
Intelligent machines can help!
Shao-Chuan Wang, Academia Sinica 9
What Can Machine Learning Do?
Shao-Chuan Wang, Academia Sinica 10
Application One:Handwriting Recognition
Video
Shao-Chuan Wang, Academia Sinica 11
Application Two: Face Detection and Tracking
Video
Shao-Chuan Wang, Academia Sinica 12
Application Three: Autonomous Driving
Video
Shao-Chuan Wang, Academia Sinica 13
Overview of Machine Learning Applications
• Speech recognition
• Computer vision
• Bio-surveillance
• Robotics
• Data mining
Shao-Chuan Wang, Academia Sinica 14
What Is Learning?
15Shao-Chuan Wang, Academia Sinica
A Tree Recognition Example (1/2)
• Suppose that you have never seen trees before, and I give you some “EXAMPLES”
Trees examples ‘Not’ Trees examples
Shao-Chuan Wang, Academia Sinica 16
A Tree Recognition Example (2/2)
• I will ask you if these unseen photos are trees or not.
Is it a tree?YES
NOor
Query Images
(AND) How much confidence?
Shao-Chuan Wang, Academia Sinica 17
What Is Learning?
• (Mitchell 2002) Learning is to improve the performance measure P of the task T based on the past experience E.– T: To recognize a tree– P: Recognition accuracy– E: The examples that I gave to you
• Two key elements of learning:– Memorization of past experiences.– “Generalization” ability ( 舉一反三 ).
18Shao-Chuan Wang, Academia Sinica
A Simple Algorithm: Nearest Neighbor
• For a given query image– Find the nearest image to the query image in the
database– Assign the label of the nearest one to the query
image. Query
Difference = 1.5
Difference = 5.5 Difference = 10
Difference = 13
Difference = 11
Tree!
19
What Were We Modeling?
Human Concept(exist but unknown)
YESYESYESNONONO…
A Machine(A learning algorithm)
Infer
Query
Prediction: NO
TREE
Shao-Chuan Wang, Academia Sinica
Training…
Shao-Chuan Wang, Academia Sinica 20
What if we do not have label ground truth?? (or labels are very expensive)
Shao-Chuan Wang, Academia Sinica 21
Unsupervised Learning
• Clustering。 Each segment forms a “Cluster”.。 Pattern discover
Shao-Chuan Wang, Academia Sinica 22
Examples: Amazon.com
• Marketing– Recommendation on the similar goods.
Shao-Chuan Wang, Academia Sinica 23
Challenges of Machine Learning
• How do we model the “difference” between two images?– Data Representation– Difference Metric• What is the “score” or “difference” function?
– How did we calculate the distance value in the tree example?
• Learning– Does it model well? (can it accurately predict the
seen data?)– Does it generalize well? (can it be proved?)
OR ?
Shao-Chuan Wang, Academia Sinica 24
Example: sea bass or salmon?
• Suppose that we have only two kinds of fish, and we want a computer system that aids our distinction between sea bass and salmon.
• Process:
Take A
PictureComputer Decision
Shao-Chuan Wang, Academia Sinica 25
Example: sea bass or salmon?
• How do we describe a fish? (Data representation)– What kinds of information can help us distinguish one
from the other?• Length, width, size of fins, tail shape, color, etc?
• How do we measure its distinctness under the chosen data representation? (Difference metric)– E.g. if we choose length, than their “distinctness” can
be measured using its absolute relative values.
|| basssalmon LLD
Shao-Chuan Wang, Academia Sinica 26
Example: sea bass or salmon?
• Assume that a fisherman (prior domain knowledge) told us that salmon is generally longer than a sea bass.
• We may use length as a feature to discriminate between them.
• But how?
|| basssalmon LLD
Shao-Chuan Wang, Academia Sinica 27
Example: sea bass or salmon?
• We use “past experiences” and we calculate a histogram of lengths for two types of fishes.
• Apply Nearest Neighbor to their average length.
Shao-Chuan Wang, Academia Sinica 28
Example: sea bass or salmon?
• The difficulty comes from the ambiguity around the threshold value.
• Length itself is insufficient to “describe” the fishes.– Use more features like width and color, etc.– Other manipulation. E.g. use nearest neighbor to
“median” of the length; will it be better?• Let’s try one more feature: width
Shao-Chuan Wang, Academia Sinica 29
Example: sea bass or salmon?
• We can use two features and wrote them down as a vector:
• Each fish image is represented as a 2-D feature vector:
2
1
x
xx
Length : x1
Width : x2
Shao-Chuan Wang, Academia Sinica 30
Example: sea bass or salmon?
There are still misclassified training examples
Shao-Chuan Wang, Academia Sinica 31
Example: sea bass or salmon?
• Why use Line?
• We can use complex boundary, but we radically change the boundary just because of some heretics. => may not generalize well.
Shao-Chuan Wang, Academia Sinica 32
Challenges of Machine Learning
• Conclusion on this example:– We have to incorporate prior knowledge to decide
which features we are going to use. At present, there is no universal learning machines.
– We want a feature that is invariant within certain specie but distinct between different species.
– There is a trade-off between complexity of decision model s and their “training errors”.
Shao-Chuan Wang, Academia Sinica 33
The Future of Machine Learning
• Theoretic foundations of learning• Scalability (Parallel)• Robustness to dynamic environment
Shao-Chuan Wang, Academia Sinica 34
Questions?
Shao-Chuan Wang, Academia Sinica 35
Thank you for your attention!
Shao-Chuan Wang, Academia Sinica 36
Learning schemes
• Supervised learning:– The tree example is a supervised learning
problem.– Supervised learning provides label ground truth.
• Unsupervised learning:– Unsupervised learning DOES NOT provide label
ground truth.• Reinforcement learning:– The way you train your pets.