Odd Leaf OutCombining Human and Computer Vision

Arijit Biswas, Computer Science and Darcy Lewis, iSchool

Derek Hansen, Jenny Preece, Dana Rotman-University of Maryland’s iSchoolDavid Jacobs, Eric Stevens-University of Maryland Computer Science

Jen Hammock, Cynthia Parr-The Smithsonian Institution

Refining Metadata Associated with Images

Existing Image Crowdsourcing Games

How our game is different

• Anyone can play and can provide us with useful information.

• No expertise necessary

• Capitalizes on strengths of humans and algorithms– Humans are better than algorithms at identifying

similarity of images

Game Mechanics

Game Mechanics

How Leaf Sets Are Constructed

• Designed to bring in useful data

• Not too easy or too hard

• Curvature based histograms used to get features from leaf shapes.– These features are used to find distance between

all possible pairs of leaves.

What’s in it for us if people play this game?

• Identify errors in the dataset

• Discover if color helps humans identify leaves

• Feedback on how enjoyable or difficult the game is

Game Variations

Before Leaf is Chosen

Multiple Guesses Skip

After Leaf is Chosen

Contest after Game is Finished Contest Previous Round

Feedback Mechanism

When Feedback Occurs

Mechanical Turk Trial

1 2 3 4 50

5

10

15

20

25

30

Enjoyment

Num

ber C

orre

ct

Mechanical Turk Trial

1 2 3 4 51

2

3

4

5

Difficulty

Enjo

ymen

t

Summary

• Anyone can help in Computer Vision research work.

• Games can be fun for players and useful for researchers.

• Humans are better than machines in judging the similarity of two images.

Funding

This work is made possible by National Science Foundation grant number 0968546