#360iDev@CarlBrwn
Apps That Can SeeCarl Brown
Missing Last Vowel -I blame Flickr/Tumblr
1Monday, September 9, 13
@CarlBrwn #360iDev
Computer Vision?Turning Photos Into Useable Data
Image Analysis
Colors
Shapes
Numbers & Letters
2Monday, September 9, 13
@CarlBrwn #360iDev
Word Lens
http://www.youtube.com/watch?v=h2OfQdYrHRs
3Monday, September 9, 13
@CarlBrwn #360iDev
Word Lens
http://www.youtube.com/watch?v=h2OfQdYrHRs
3Monday, September 9, 13
@CarlBrwn #360iDev
Today's TalkHow quick and easy it is to get actionable info from photos
4Monday, September 9, 13
@CarlBrwn #360iDev
Today's TalkHow quick and easy it is to get actionable info from photos
Demo Code with Source
4Monday, September 9, 13
@CarlBrwn #360iDev
Today's TalkHow quick and easy it is to get actionable info from photos
Demo Code with Source
Not about HOW OpenCV works
4Monday, September 9, 13
@CarlBrwn #360iDev
Today's TalkHow quick and easy it is to get actionable info from photos
Demo Code with Source
Not about HOW OpenCV works
"OpenCV by Example"
4Monday, September 9, 13
@CarlBrwn #360iDev
CubeCheater &Sudoku Grab
http://www.youtube.com/watch?v=HNwx0nbgm7M http://www.youtube.com/watch?v=wt96OomJY9A
5Monday, September 9, 13
@CarlBrwn #360iDev
Both have had Legal Problems
http://cubecheater.efaller.com http://sudokugrab.blogspot.com/2010/12/sudoku-grab-removed-from-sale-in-japan.html
6Monday, September 9, 13
#360iDev@CarlBrwn
Based onCode From:github.com/carlbrown/OpenCVDemo360iDev2013
7Monday, September 9, 13
@CarlBrwn #360iDev
About the CodeExplicit and Redundant to make it easier to follow/explain
8Monday, September 9, 13
@CarlBrwn #360iDev
About the CodeExplicit and Redundant to make it easier to follow/explain
Not production ready - for teaching purposes
8Monday, September 9, 13
@CarlBrwn #360iDev
About the CodeExplicit and Redundant to make it easier to follow/explain
Not production ready - for teaching purposes
Minimum Viable Interface
8Monday, September 9, 13
@CarlBrwn #360iDev
About the CodeExplicit and Redundant to make it easier to follow/explain
Not production ready - for teaching purposes
Minimum Viable Interface
Don't hate, okay?
8Monday, September 9, 13
#360iDev@CarlBrwn
Face CounterDemo 1/3
9Monday, September 9, 13
#360iDev@CarlBrwn
Core ImageOption 1
10Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
[CIImage imageWithCGImage:]Detect
11Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
[CIImage imageWithCGImage:][CIDetector detectorOfType:! ! ! ! CIDetectorTypeFace]
Detect
11Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
[CIImage imageWithCGImage:][CIDetector detectorOfType:! ! ! ! CIDetectorTypeFace]NSArray *features= [detector ! ! ! ! ! ! featuresInImage:]
Detect
11Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
[CIImage imageWithCGImage:][CIDetector detectorOfType:! ! ! ! CIDetectorTypeFace]NSArray *features= [detector ! ! ! ! ! ! featuresInImage:]for(CIFaceFeature* faceFeature !! ! ! ! ! ! ! in features)
Detect
11Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
for(CIFaceFeature* faceFeature !! ! ! ! ! ! ! in features)
Debugging
12Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
for(CIFaceFeature* faceFeature !! ! ! ! ! ! ! in features)frame = faceFeature.bounds
Debugging
12Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
for(CIFaceFeature* faceFeature !! ! ! ! ! ! ! in features)frame = faceFeature.boundsframe *= contentScaleFactor
Debugging
12Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
for(CIFaceFeature* faceFeature !! ! ! ! ! ! ! in features)frame = faceFeature.boundsframe *= contentScaleFactor//flip to CIImage Coords
Debugging
12Monday, September 9, 13
@CarlBrwn #360iDev
Coordinate Systems
UIView has (0,0) at Upper Left
Core Image has (0,0) Lower Left
(Like OS X)
13Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
for(CIFaceFeature* faceFeature !! ! ! ! ! ! ! in features)frame = faceFeature.boundsframe *= contentScaleFactor//flip to CIImage Coords
Debugging
14Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
for(CIFaceFeature* faceFeature !! ! ! ! ! ! ! in features)frame = faceFeature.boundsframe *= contentScaleFactor//flip to CIImage CoordsfaceView.layer.borderWidth = 4;
Debugging
14Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- coreImageButtonPressed:
for(CIFaceFeature* faceFeature !! ! ! ! ! ! ! in features)frame = faceFeature.boundsframe *= contentScaleFactor//flip to CIImage CoordsfaceView.layer.borderWidth = 4;addSubview:faceView
Debugging
14Monday, September 9, 13
#360iDev@CarlBrwn
Core ImageOption 1
15Monday, September 9, 13
#360iDev@CarlBrwn
OpenCVOption 2
16Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
convert to cv::MatDetect
17Monday, September 9, 13
@CarlBrwn #360iDev
Convert to cv::MatCGImageGetColorSpace(image.CGImage)
cv::Mat cvMat()
CGBitmapContextCreate(cvMat.data,...)
CGContextDrawImage(ctx,r,image.CGImage)
CGContextRelease()
18Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
convert to cv::MatDetect
19Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
convert to cv::Matcv::cvtColor( CV_BGR2GRAY )
Detect
19Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
convert to cv::Matcv::cvtColor( CV_BGR2GRAY )cv::equalizeHist()
Detect
19Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
convert to cv::Matcv::cvtColor( CV_BGR2GRAY )cv::equalizeHist()cv::face_cascade.\! ! ! ! ! ! detectMultiScale()
Detect
19Monday, September 9, 13
@CarlBrwn #360iDev
Cascade Classifierhttp://docs.opencv.org/doc/tutorials/objdetect/cascade_classifier/cascade_classifier.html
20Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
convert to cv::Matcv::cvtColor( CV_BGR2GRAY )cv::equalizeHist()face_cascade.detectMultiScale()
Detect
21Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
convert to cv::Matcv::cvtColor( CV_BGR2GRAY )cv::equalizeHist()face_cascade.detectMultiScale()for( int i = 0; ! ! ! ! i < faces.size(); i++ )
Detect
21Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
for( int i = 0; ! ! ! ! i < faces.size(); i++ )
Debugging
22Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
for( int i = 0; ! ! ! ! i < faces.size(); i++ )CGRectMake(faces[i].frame)
Debugging
22Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
for( int i = 0; ! ! ! ! i < faces.size(); i++ )CGRectMake(faces[i].frame)frame *= contentScaleFactor
Debugging
22Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
for( int i = 0; ! ! ! ! i < faces.size(); i++ )CGRectMake(faces[i].frame)frame *= contentScaleFactorfaceView.layer.borderWidth = 4;
Debugging
22Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- openCVButtonPressed:
for( int i = 0; ! ! ! ! i < faces.size(); i++ )CGRectMake(faces[i].frame)frame *= contentScaleFactorfaceView.layer.borderWidth = 4;addSubview:faceView
Debugging
22Monday, September 9, 13
#360iDev@CarlBrwn
OpenCVOption 2
23Monday, September 9, 13
#360iDev@CarlBrwn
CubeFaceGrabberDemo 2/3
24Monday, September 9, 13
#360iDev@CarlBrwn
DetectEdgesdetectFacesPressed:
25Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
convert to cv::MatDetect
26Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
convert to cv::Mat
cv::Canny()
Detect
26Monday, September 9, 13
@CarlBrwn #360iDev
Canny27Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
convert to cv::Mat
cv::Canny()
Detect
28Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
convert to cv::Mat
cv::Canny()cv::HoughLinesP()
Detect
28Monday, September 9, 13
@CarlBrwn #360iDev
HoughLinesP29Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
convert to cv::Mat
cv::Canny()cv::HoughLinesP()
Detect
30Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
convert to cv::Mat
cv::Canny()cv::HoughLinesP()for( size_t i = 0; ! ! ! i < lines.size(); i++ )
Detect
30Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
convert to cv::Mat
cv::Canny()cv::HoughLinesP()for( size_t i = 0; ! ! ! i < lines.size(); i++ )
if (LineIntersect(l, l2))
Detect
30Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
convert to cv::Mat
cv::Canny()cv::HoughLinesP()for( size_t i = 0; ! ! ! i < lines.size(); i++ )
if (LineIntersect(l, l2))//Measure Distance to Image corners
Detect
30Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
for( size_t i = 0; ! ! ! i < lines.size(); i++ )
Debugging
31Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
for( size_t i = 0; ! ! ! i < lines.size(); i++ )cv::line() //Draw
Debugging
31Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
for( size_t i = 0; ! ! ! i < lines.size(); i++ )cv::line() //Draw//Check for bounding corners
Debugging
31Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
for( size_t i = 0; ! ! ! i < lines.size(); i++ )cv::line() //Draw//Check for bounding cornerscv::circle() //Draw
Debugging
31Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
for( size_t i = 0; ! ! ! i < lines.size(); i++ )cv::line() //Draw//Check for bounding cornerscv::circle() //Drawconvert to UIImage //Display
Debugging
31Monday, September 9, 13
@CarlBrwn #360iDev
Convert to UIImage
[NSData dataWithBytes:cvMat.data length:]
CGDataProviderCreateWithCFData()
CGImageCreate()
[[UIImage alloc] initWithCGImage:]
CGXXXRelease()
32Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- detectFacesPressed:
for( size_t i = 0; ! ! ! i < lines.size(); i++ )cv::line() //Draw//Check for bounding cornerscv::circle() //Drawconvert to UIImage //Display
Debugging
33Monday, September 9, 13
#360iDev@CarlBrwn
DetectEdgesdetectFacesPressed:
34Monday, September 9, 13
#360iDev@CarlBrwn
UnskewgetAndApplyTransformPressed:
35Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- getAndApplyTransformPressed:
convert to cv::Mat
36Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- getAndApplyTransformPressed:
convert to cv::MatgetPerspectiveTransform()
36Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- getAndApplyTransformPressed:
convert to cv::MatgetPerspectiveTransform()warpPerspective(transform)
36Monday, September 9, 13
@CarlBrwn #360iDev
Map the Corners circled on the Left
To the Corners of the Images on the Right
37Monday, September 9, 13
@CarlBrwn #360iDev
Map the Corners circled on the Left
To the Corners of the Images on the Right
37Monday, September 9, 13
@CarlBrwn #360iDev
Map the Corners circled on the Left
To the Corners of the Images on the Right
37Monday, September 9, 13
@CarlBrwn #360iDev
Map the Corners circled on the Left
To the Corners of the Images on the Right
37Monday, September 9, 13
@CarlBrwn #360iDev
Map the Corners circled on the Left
To the Corners of the Images on the Right
37Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- getAndApplyTransformPressed:
convert to cv::MatgetPerspectiveTransform()warpPerspective(transform)
38Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- getAndApplyTransformPressed:
convert to cv::MatgetPerspectiveTransform()warpPerspective(transform)convert to UIImage //Display
38Monday, September 9, 13
#360iDev@CarlBrwn
UnskewgetAndApplyTransformPressed:
39Monday, September 9, 13
#360iDev@CarlBrwn
ExtractColorsextractColorsPressed:
40Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
convert to cv::Matcv::cvtColor(CV_RGB2HSV)
Detect
41Monday, September 9, 13
@CarlBrwn #360iDev
HSV Space
Hue/Saturation/Value
Easier to get Color out
http://en.wikipedia.org/wiki/HSV_color_space42Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
convert to cv::Matcv::cvtColor(CV_RGB2HSV)
Detect
43Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
convert to cv::Matcv::cvtColor(CV_RGB2HSV)for (int hSlice;;) { ! ! for (int vSlice;;) {
Detect
43Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
convert to cv::Matcv::cvtColor(CV_RGB2HSV)for (int hSlice;;) { ! ! for (int vSlice;;) {cv::Rect r()
Detect
43Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
convert to cv::Matcv::cvtColor(CV_RGB2HSV)for (int hSlice;;) { ! ! for (int vSlice;;) {cv::Rect r()Mat cv::roi(r)
Detect
43Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
convert to cv::Matcv::cvtColor(CV_RGB2HSV)for (int hSlice;;) { ! ! for (int vSlice;;) {cv::Rect r()Mat cv::roi(r)avgColor=cv::mean(roi)
Detect
43Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
convert to cv::Matcv::cvtColor(CV_RGB2HSV)for (int hSlice;;) { ! ! for (int vSlice;;) {cv::Rect r()Mat cv::roi(r)avgColor=cv::mean(roi)//Check Saturation and Hue
Detect
43Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
for (int hSlice;;) { ! ! for (int vSlice;;) {
Debugging
44Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
for (int hSlice;;) { ! ! for (int vSlice;;) { roi = avgColor //Fill
Debugging
44Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
for (int hSlice;;) { ! ! for (int vSlice;;) { roi = avgColor //FillputText(roi, [color cString]
Debugging
44Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
for (int hSlice;;) { ! ! for (int vSlice;;) { roi = avgColor //FillputText(roi, [color cString]putText(roi, stringWithFormat:@"%.0f/%.0f/%.0f")
Debugging
44Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractColorsPressed:
for (int hSlice;;) { ! ! for (int vSlice;;) { roi = avgColor //FillputText(roi, [color cString]putText(roi, stringWithFormat:@"%.0f/%.0f/%.0f")
convert to UIImage //Display
Debugging
44Monday, September 9, 13
#360iDev@CarlBrwn
ExtractColorsextractColorsPressed:
45Monday, September 9, 13
#360iDev@CarlBrwn
SudokuGrabberDemo 3/3
46Monday, September 9, 13
#360iDev@CarlBrwn
FindEdgesfindPuzzlePressed:
47Monday, September 9, 13
@CarlBrwn #360iDev
Just like Cube
48Monday, September 9, 13
#360iDev@CarlBrwn
UnSkewPuzzlesquarePressed:
49Monday, September 9, 13
@CarlBrwn #360iDev
Just like Cube
50Monday, September 9, 13
#360iDev@CarlBrwn
ExtractTextextractPressed:
51Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
Tesseract* tesseract =Setup
52Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
Tesseract* tesseract =initWithDataPath:@"tessdata" language:@"eng"
Setup
52Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
Tesseract* tesseract =initWithDataPath:@"tessdata" language:@"eng"tesseract setVariableValue:@"0123456789"forKey:@"tessedit_char_whitelist"
Setup
52Monday, September 9, 13
@CarlBrwn #360iDev
tesseract-ocrhttp://code.google.com/p/tesseract-ocr/
53Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
convert to cv::MatSlice
54Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
convert to cv::Matfor (int hSlice;;) { ! ! for (int vSlice;;) {
Slice
54Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
convert to cv::Matfor (int hSlice;;) { ! ! for (int vSlice;;) {cv::Rect r()
Slice
54Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
convert to cv::Matfor (int hSlice;;) { ! ! for (int vSlice;;) {cv::Rect r()Mat cv::roi(r)
Slice
54Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
convert to cv::Matfor (int hSlice;;) { ! ! for (int vSlice;;) {cv::Rect r()Mat cv::roi(r)cv::cvtColor(CV_RGB2GRAY);
Slice
54Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
convert to cv::Matfor (int hSlice;;) { ! ! for (int vSlice;;) {cv::Rect r()Mat cv::roi(r)cv::cvtColor(CV_RGB2GRAY);cv::threshold(roi_gray)
Slice
54Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
convert slice to UIImage Detect
55Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
convert slice to UIImage tesseract setImage:imageToParse
Detect
55Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
convert slice to UIImage tesseract setImage:imageToParsetesseract recognize
Detect
55Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
[[UILabel alloc]initWithFrame:CGRectMake()]
Debugging
56Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
[[UILabel alloc]initWithFrame:CGRectMake()] labelsetText:recognizedString
Debugging
56Monday, September 9, 13
@CarlBrwn #360iDev
ViewController.mm- extractPressed:
[[UILabel alloc]initWithFrame:CGRectMake()] labelsetText:recognizedStringself.viewaddSubview:recognizedLabel
Debugging
56Monday, September 9, 13
#360iDev@CarlBrwn
ExtractTextextractPressed:
57Monday, September 9, 13
#360iDev@CarlBrwn
Based onCode From:github.com/carlbrown/OpenCVDemo360iDev2013
Questions?
58Monday, September 9, 13