introduction to skin and face detection

INTRODUCTION TO SKIN AND FACE DETECTIONAleksey Deykin

Introduction What it is

Automatic computer recognition of faces and skin tone

Applications Anything from security and law

enforcement to assisting the elderly and visually impaired

Requirements Fast, accurate, and lighting and ethnicity

invariant

Skin Color Detection Provide a sample of skin

tone Calculate average color

(RGB) Scan images pixel by

pixel If color matches, color

pixel red

RGB The most commonly used color space in

digital images. It encodes colors as an additive combination of 3 primary colors: red (R), green (G) and blue (B)

Red: rgb(255,0,0) Blue: rgb(0,255,0) Green: rgb(0,0,255) Black: rgb(0,0,0) White: rgb(255,255,255)

Simple Skin Detection

Improved Skin Detection

Improved Skin Detection

Improved Skin Detection

Algorithm Loop through every pixel of the sample

rectangle Add pixel’s RGB channels to a vector

Calculate average RGB value (skin tone) Loop through every pixel of the image If R±40 and G±40 and B±40 for rectangle

1, or If R±40 and G±40 and B±40 for rectangle 2

Color the pixel red (skin detected)

Challenges & Limitations Slow

O(xy) 80 seconds per 100 skin detections, or 0.8

seconds per image (400x608) As resolution doubles, computing time

quadruples Color-dependent

Black & white pictures problematic Ethnicity dependent Needs contrasting background

Challenges & Limitations

Further Research Different color space? YCbCr

Used in video and digital photography systems due to its ability to encode and compress RGB information. Stores luminance separately.

Face Detection Viola-Jones algorithm Feature-based vs pixel-based Detector scans input at multiple scales, starting with

a base of 24x24 pixels, such that a 384 by 288 pixel image is scanned at 12 scales with a 1.25x step

AdaBoost learning algorithm (thousands of faces to train)

First selected feature is usually around the eyes (usually darker area) - if eyes are not visible, algorithm usually fails

95% detection (1 in 14084 falsepositive) – 15 fps

Face Detection Results

Challenges & Limitations Trained on front-facing upright faces and

is only reliable for faces rotated around ±15 degrees in plane and ±45 degrees out of place (toward a profile view)

Fails for overexposed (bright) backgrounds

Heavily occluded faces not detected

Further Research Combine skin and face detection?

Pre-screen images for skin, then run face detection over skin regions

Run both algorithms, one is bound to find a face

Extend skin detection?

Detect skin… And faces

Conclusion Simple algorithm to detect skin Slow and highly dependent on lighting Possible to improve results with different

color space Faces naturally form detectable ovals Wear shades to protect privacy

References Elgammal, A., Muang, C., and Hu, D. 2009. Skin Detection - a

Short Tutorial. Rutgers University, Piscataway, NJ. http://www.cs.rutgers.edu/~elgammal/pub/skin.pdf. May 17, 2012.

Shah, M. A. An Introduction to Wavelets and the Haar Transform. http://www.cs.ucf.edu/~mali/haar/. May 17, 2012.

Soetedjo, A., Yamada, K. 2008. Skin Color Segmentation Using Coarse-to-Fine Region on Normalized RGB Chromaticity Diagram for Face Detection. IEICE Trans. Inf. & Syst., Vol.E91-D, No.10 October 2008.

Szeliski, R. 2010. Computer Vision: Algorithms and Applications. http://szeliski.org/Book/. May 17, 2012. pp. 664-665.

Viola, P., Jones, M. J. 2003. Robust Real-Time Face Detection. International Journal of Computer Vision 57(2), pp. 137-154, 2004.