1 track kinect@bicocca - intro
TRANSCRIPT
KINECT Programming
Gesture based interaction: Kinect programming
Ing. Matteo Valoriani [email protected] 26/03/2012
Università degli Studi di Milano - Bicocca
KINECT Programming
Agenda
Parte 1 (60 min):
Introduzione a Kinect: cosa è e come funziona l'oggetto delle meraviglie.
Introduzione allo sviluppo con Kinect
Parte 2 (90 min):
Creazione di un modello 3d dell'utente
Gesture Design
Creazione di un semplice riconoscitore di gesti
Tecniche avanzate di riconoscimento
Conclusione (15 min):
Iniziative per gli studenti
KINECT Programming
Lots of words…
Ambient Intelligence
Internet of Things
Pervasive Computing
Physical Computing
Ubiquitous computing
Augmented reality
Human-centered computing
Smart device
KINECT Programming
• No more desktop-centered computation, but
distributed computation(“ubiquitous”)
• Objects become more “intelligent” and “smart”
• New information’s model
• New possibility of interaction with information
• Machines fit the human environment instead
of forcing humans to enter theirs
… One concept
KINECT Programming
Interface Evolution
Command Line
Interface Graphical User
Interface
CLI GUI
NUI
KINECT Programming
Natural User Interface
Computer Vision
Facial Recognition
Spatial Recognition
Augmented Reality
Gesture Sensing
Audio Recognitio
n
Voice Command
Natural Speech
Ambient Noise
Touch
Single Touch
Multi-Touch
Pen Input
Sensors
Geospatial Sensing
Accelerometers
Biometrics
Ambient Light
Brain Waves
Mind control
Mood Recognition
KINECT Programming
Computer vision is the field concerned with the processing of images from the real world to extract and interpret information in real time.
It is the science and technology of machines that see. Here see means the machine is able to extract information from an image, to solve some task, or perhaps "understand" the scene in either a broad or limited sense.
Computer Vision
KINECT Programming
• Controlling processes (e.g., an industrial robot).
• Navigation (e.g. by an autonomous vehicle or mobile robot).
• Detecting events (e.g., for visual surveillance or people counting).
• Organizing information (e.g., for indexing databases of images and
image sequences).
• Modeling objects or environments (e.g., medical image analysis or
topographical modeling).
• Interaction (e.g., as the input to a device for computer-human
interaction).
• Medicine, ….
Applications
KINECT Programming
• Hardware:
• one or more image sensors
• Software:
• Complex algorithms of image analysis
• Specific user interfaces
• Interactive applications
Technological aspects
KINECT Programming
KINECT Programming
• It is a motion sensing input device by Microsoft for the Xbox
360 console.
• It enables users to control and interact with the Xbox without
the need to touch a game controller, through a natural user
interface using gestures and spoken commands.
• Launched in November 2010, sold 8 million units in 60 days
Kinect
KINECT Programming
=
Immersive user experience
Kinect’s magic
“Any sufficiently advanced technology is indistinguishable
from magic” (Arthur C. Clarke)
KINECT Programming
Power Comes from the Sum
• Any single technology on its own – can create
good experiences
• The sum: This is where
the magic is
• Tons of opportunities ahead
KINECT Programming
• In November 2010, is released the first Linux driver that allows the use of both the RGB camera and depth sensitivity functions of the device.
• In December 2010, PrimeSense, that produces the camera, released their open source drivers with motion tracking middleware called NITE
• Microsoft released a non-commercial Kinect beta SDK for Windows on June 16, 2011(updated in November)
• Microsoft released commercial Kinect SDK for Windows on February 1st, 2012
Kinect programming
KINECT Programming
Videos
• http://www.xbox.com/en-US/Kinect/Kinect-Effect (rehabilitation)
• http://www.youtube.com/watch?v=id7OZAbFaVI&feature=related (Medical suse)
• http://www.kinecthacks.com/kinect-interactive-hopscotch/ (game for children)
• http://www.youtube.com/watch?v=9xMSGmjOZIg&feature=related (Holographic
projection)
• http://www.youtube.com/watch?v=1dnMsmajogA&feature=related (Kinect Glasses-
less 3D)
• http://www.youtube.com/watch?v=s0Fn6PyfJ0I&feature=related (Kinect Virtual
Fashion, the Future of Shopping at Home)
• http://www.youtube.com/watch?v=4V11V9Peqpc&feature=related (Kinect
Projection mapping with box2D physics)
KINECT Programming
Videos (2) • http://www.youtube.com/watch?v=oALIuVb0NJ4 (Multidevice computing)
• http://www.youtube.com/watch?v=-yxRTn3fj1g&feature=related (Kinect Touch
wall )
• http://www.youtube.com/watch?v=KBHgRcMPaYI&feature=related (Kinect
window Bank of Moscow)
• http://kinecthacks.net/motion-control-banking-is-so-easy-even-your-pet-can-
do-it/ (Kinect Banking App Video)
• http://www.youtube.com/watch?v=FMCIO0KNjrs (Art)
• http://www.youtube.com/watch?v=g6N9Qid8Tqs&feature=related (Interactive
LED Floor)
• http://www.youtube.com/watch?v=c6jZjpvIio4 (Kinect in Education)
• http://www.youtube.com/watch?v=_qvMHAvu-yc&feature=related (explore
universe)
KINECT Programming
Resources and tools
• http://channel9.msdn.com/Search?term=kinect&type=All (Others projects)
• http://kinecthacks.net/ (Others projects)
• http://www.modmykinect.com (Others projects)
• http://kinectforwindows.org/resources/ (Microsoft SDK)
• http://www.kinecteducation.com/blog/2011/11/13/9-excellent-programming-
resources-for-kinect/ (resources)
• http://kinectdtw.codeplex.com/ (gesture recognition library)
• http://kinectrecognizer.codeplex.com/ (gesture recognition library)
• http://projects.ict.usc.edu/mxr/faast/ (gesture recognition library)
• http://leenissen.dk/fann/wp/ (gesture recognition library)