roles of local and global information in image classification devi parikh
TRANSCRIPT
Coast Highway Mountain Street
Forest Inside city Open country Tall buildings
Bathroom Bedroom Dining room Gym
Kitchen Living room Movie theater Stair case
Aeroplane Car-rear Face
Ketch Motorbike Watch
OSR
ISR
CAL
2
Human Recognition of Jumbled Images
OSR ISR CAL0
10
20
30
40
50
60
70
80
90
100
Intact
JumbledAccu
racy
6
How do we recognize them?
Majority Vote of Local Independent
Decisions?
MRF Inference?
…
Are existing machine implementations of this model good enough?
7
Machine Experiments
• Assign blocks to codewords
• Accumulate p(scene|codewords)
• Pick class with highest votes
Majority Vote ResultsPr
opor
tion
of m
atch
ed re
spon
ses
OSR ISR CAL0
10
20
30
40
50
60
70
80
90
100
Random
Human Most Confident Block
Human Majority-Vote
Machine Majority-Vote
Inter-Human Agreement
11