kinect in a.i. - university of...
Post on 04-Sep-2019
13 Views
Preview:
TRANSCRIPT
Kinect in A.I.
Akihiro Eguchi
What is Kinect
a motion sensing input device by Microsoft for the Xbox 360 video game console, which has
◦ RGB Camera
◦ 3D Depth Sensors
◦ Motorized Tilt
◦ Multi-Array Microphone
Kinect Hacks
3D Video Capture with Kinect
http://www.youtube.com/watch?v=7QrnwoO1-8A
KinectFusion - 3D reconstruction
and Augmented Reality
http://www.youtube.com/watch?v=quGhaggn3cQ
Kinect Enabled Autonomous
Mini Robot Car Navigation
http://www.youtube.com/watch?v=NmnepqAQIlk
Some More Links
Top ten Kinect hacks
◦ http://spectrum.ieee.org/automaton/robotics/diy/top-10-robotic-kinect-hacks
Robots use Kinect to understand our world (object recognition)
◦ http://www.newscientist.com/article/dn20723-robots-use-kinect-to-understand-our-world.html
3d models from a Kinect on a car
◦ http://blog.decoratorpattern.com/2011/01/23/real-world-mapping-with-the-kinect/
Kinect RGB Demo v0.3.1
◦ http://nicolas.burrus.name/index.php/Research/KinectRgbDemoV3
Drawing in the air with Kinect
◦ http://vimeo.com/16818988
Teaching Kinect to recognize objects
◦ http://www.youtube.com/watch?v=fQ59dXOo63o
Microsoft Official SDK
Microsoft has released of a non-
commercial Kinect software development
kit for Windows on June 16, 2011.
C++/CLI, C#, or Visual Basic .NET.
Tutorial of how to use the SDK
◦ Kinect for Windows SDK Quickstarts
Object learning
How human learn name of objects
Human children use mainly two biases
when learning name of objects:
◦ Shape bias (Landau, Smith, and Jones (1988))
Generalize name of the object if the shape is similar
◦ Function bias (Nelson, Russsel, Duke, and Jones (2000))
Generalize name of the object if the function of the
use seem to be the same
Previous work in object name
learning Simulation of biases (Grabner1, Gall, Gool (2011))
◦ Shape bias:
Grab 3D models that are labeled with a same name (e.g., chair) and run a machine learning algorithm to train the classifier.
◦ Function bias:
First define the use of the object; e.g., chair is a object to sit on.
Then, let the program learn the posture of sitting.
If the object is sittable, the object is more likely to be a chair.
Shape Bias Implementation with
Kinect Take the depth info. of the environment.
Use RANSAC algorithm to remove the ground the object is placed.
Name the objects and run machine learning to train the classifiers.
Google 3D warehouse should enhance the learning (Lai and Fox (2010))
E.g., of machine learning techniques
K-NN algorithm
◦ Simplest machine learning technique
◦ Store all the input associated with a name.
◦ To identify a new object, take k closest match and take a majority vote to find the most probable name of the object.
Problem:
◦ Memory consumption and speed limitation
One solution:
◦ Use SVM, which is a lot better in the way it does not store every exact data of the input.
Function Bias implementation with
Kinect Kinect has a feature to track 20 different
joints of human body.
◦ Easily be able to draw skeletal body image
So, program can learn a posture of human (like sitting) easily.
Like Grabner1, Gall, Gool (2011), we can add a simple function bias with this way.
Activity learning
How human learn name of action
Children distinguish actions based on
◦ Manner bias
Based on the manner the action includes
Shrink-expand, movement of the hands, and etc.
◦ Path bias:
Based on the path the action takes
E.g., left to right, up to down, and etc.
Pulverman, Hirsh-Pasek, Golinkoff, Pruden and Salkind (2006)
Previous work in action learning
Laptev, and Caputo (2004)
◦ used video recorded action data like walking,
jogging, and running, which is four seconds in
average, and run SVM to train the classifier to
recognize those actions.
Implementation of the action
learning with Kinect records 10 seconds of each action and name
it.
motion data can be represented as a relative distances between all of 20 joints tracked by Kinect.
By running SVM on the cumulated data, program can recognize what kind of action humans are performing in front of the Kinect sensor based on a manner bias.
Path bias can be implemented easily as well.
Activity learning demo on youtube
http://www.youtube.com/watch?v=AxCn0eKWkiQ
top related