computer vision technights, 4 april 2016 manages to find our … vision.pdf · 2016-04-08 ·...

Computer Vision – TechNights, 4 April 2016

In this session we will learn about Computer Vision, a branch of Artificial Intelligence

that teaches computers to see the world. We look at how images are stored and processed

on computers, and how a computer sees the world. We will think about how our brain

manages to find our cat effortlessly, and we will look at how we can train a computer to

recognize cat images!

A robot was asked to make a picture of a dog look more dog-like. This was the result.

Computer VisionTech Nights April 4th

Speaker: Gunnar Sigurdsson

What is Computer Vision?

How can we use computer vision in real life?

Why Images?

● 8 years of video is uploaded to YouTube every day!● 250 million photos are uploaded to Facebook every day!

Why is this hard to do?

● But this is easy! We can see all these things!

● Our brain is very smart!

● Our brain does a lot of tricks we do not realize!

● We will look at why this is hard, and how our brain does it!

30% of our brain is used to see!

What color is the dress?

???

Do you believe me if I say A and B are the same?

Do you believe me if I say A and B are the same?

Can anyone guess what this is?

What about now?

We sometimes need to look at the whole image!

What about these things?

What about these things?

They are actually the same image, just rotated!

Our eyes are like cameras

Can you guess what is in this image?

What a computer sees!

A table of numbers!

How does a computer see images?

What we see. What a computer sees!

A table of numbers!

What about color? How do we store color?

● Our eyes are sensitive to three colors! Red, green, and blue.● We use those to see the world!

What about color? How do we store color?

● Our eyes are sensitive to three colors! Red, green, and blue.● We use those to see the world!

We can combine R, G, and B to make all the colors

How does a computer see color images?

What we see.

How does a computer see color images?

What we see. What a computer sees!

Three tables of numbers! R, G, B

What happens if we set R to zero?

What a computer sees!

What happens if we set R to zero?

What we see. No red! (only G,B) What a computer sees!

Pixels! (Picture element)

● One point of the image, a pixel, has three numbers R, G, B

Where do we go from there?

Color filtering images

● Let’s first talk about changing each pixel (picture element) in some way

● A pixel is three numbers R, G, B

● We can make a new image with r, g, b

● For example, r = 1 × R g = 2 × G b = 1 × B so each pixel becomes more green!

● This how many “image filters” on Snapchat/Instagram work!

Functions and Variables in programming

● x = 5 This is a variable

● y = multiply_this_by_two(x)

○ the function multiply_this_by_two

○ takes the value of the variable x

○ and gives a new value the the variable y

○ What do you think y is?

● x could also be an image!

○ y = make_this_image_red(x)

○ What is y now?

x

y

5

multiply_this_by_two Made by me!

x

Function to filter the image!

change_rgb(image, r='1*R' , g='2*G' , b='1*B' )

change_rgb(image, r= '1*R-.5' , g='1*G-.5' , b='1*B-.5')

Make the image darker

change_rgb(image, r='1*R-.5' , g='1*G-.5' , b='1*B-.5')

change_rgb(image, r='1*R' , g='1*R' , b='1*R')

Make the image gray

change_rgb(image, r='1*R ' , g='1*R' , b='1*R')

Make the image gray

change_rgb(image, r='.44*R ' , g='.26*R' , b='.08*R')

change_rgb(image, r='-1*R+1' , g='-1*G+1' , b='-1*B+1')

Invert the image!

change_rgb(image, r='-1*R+1' , g='-1*G+1' , b='-1*B+1')

Now you try!

● Double-click on color.bat!

● Edit as you like, and then press F5 to run the program!

● Can you make the image warmer? (blue is a cold color)

● What about colder?

● Can you make the image brighter?

● Can you make it purple?

● How crazy can you make the image?

Now you try!

● Can you make the image warmer? (blue is a cold color)○ change_rgmb(image, r='2*R' , g='1*G' , b='0.5*B')

● What about colder?

○ change_rgb(image, r='0.5*R' , g='1*G' , b='2*B')

● Can you make the image brighter?

○ change_rgb(image, r='1*R+1' , g='1*G+1' , b='1*B+1')

● Can you make it purple?

○ change_rgb(image, r='1*R+1' , g='1*G' , b='1*B+1')

● How crazy can you make the image?

1 1 1

1

111

1 1

Filtering images

● Let’s modify a pixel using the neighboring pixels

● We use a filter

○ Says how much each neighbor changes the pixel 1 1 11 1 11 1 1

This kind of filter blurs the image

1 1 11 1 11 1 1

This kind of filter finds the edges!

0 1 01 -4 10 1 0

Now what?

Image Classification

● Teach a robot the difference between Cats and Not cats!

???

Why is this challenging?

● Can we just look at the color?

Why is this hard for the robot?

● There are many kinds of cats!

Why is this hard for the robot?

● And things that look like cats...

What does the robot see?

● Can you help the robot?

● Last week we sorted characters into Hogwarts houses! (With SVM!)

● This is the same idea!

Image Classification

Classifying cat, or not cat.

● Last week we used Intelligence and Bravery to sort Hogwarts students!

● We can do the same with images!

○ First the computer does some filtering

○ Then the computer counts how many pixels have some properties!

Classifying cat, or not cat.

● Last week we used Intelligence and Bravery to sort Hogwarts students!

● We can do the same with images!

○ First the computer does some filtering

○ Then the computer counts how many pixels have some properties!

■ Like white fluffy fur

■ Or pink nose

■ Or green grass

● Maybe then not a cat!

White fluffy fur: 0.9Pink nose: 0.3Green grass: 1.1

Training a classifier

Images of cats Images of not cats

Training a classifier

Images of cats Images of not cats

Cats

Not cats

Testing a classifier

“It’s a cat!”

Cats

Not cats

Try training your own!

● Double-click on classify.bat, then edit, and press F5 to run!

● Select the type of data (cats, dogs, flowers, or pandas!)

● What should they computer look for? Only color? or any texture?

● Select the number of dimensions!○ You can select 2, 10, or 100! (or any number you want)

● How strict should the boundary be? 100%? 50%?

images = get_images( data=cat )

features = features( images , type_=color , dim=200 )

classifier = train( features , strict=50% _)

test( classifier )

strict

Not strict

Computer thinks these are cats! ..and thinks these are not!

Oops!

Where are we now?

The best classifiers today

Convolutional Neural Networks (Deep learning)

Filtering Filtering Filtering Filtering Filtering

“It’s a Robot!”

What is Computer Vision?

Sittable! Cat is walking this way!

3D Layout

Couch

Cat

TV

● “Fixing” images with computers.

● Remove fog, fix out-of-focus, change pictures to make them pretty

Computational Photography

3D from images

● Take lots of pictures, and make a 3D model

Predicting motion

● Have a robot predict what a person will do next!

What else might computer vision do?

Color Filters!   

1. Can you make the image warmer? (blue is a cold color) 

change_rgb(image, r='               '  , g='               '  , b='               ') 

2. What about colder? 

change_rgb(image, r='               '  , g='               '  , b='               ') 

3. Can you make the image brighter? 

change_rgb(image, r='               '  , g='               '  , b='               ') 

4. Can you make it purple? 

change_rgb(image, r='               '  , g='               '  , b='               ') 

5. How crazy can you make the image? 

change_rgb(image, r='               '  , g='               '  , b='               ') 

Help the robots identify the objects!