itcamp 2012 - tim huckaby - keynote
TRANSCRIPT
itcampro @ itcamp12 # Premium conference on Microsoft technologies
The Future of Software Now: NUI - A New Genre of User
Experience
Tim Huckaby Microsoft MVP & RD
InterKnowlogy Chairman/Founder
Actus Interactive Software CEO
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Mobile &
Development ITCamp 2012 sponsors
itcampro @ itcamp12 # Premium conference on Microsoft technologies
NUI: The Natural User Interface
• Common Gestures of Touch
• Physical Object Interaction
• Virtual Object Interaction
Multi-Touch Capable
• Motion Based Interface
• Tracking of full-body movement, facial expression, and voice with precision
Gesture Capable
• Multi-array Microphone
• Uniquely Distinguish Audio & Voices Voice Capable
• Conscious and non-conscience interfaces
• Non Invasive
Neural Capable
itcampro @ itcamp12 # Premium conference on Microsoft technologies
itcampro @ itcamp12 # Premium conference on Microsoft technologies
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
itcampro @ itcamp12 # Premium conference on Microsoft technologies
A little bit about me and why I am here…
itcampro @ itcamp12 # Premium conference on Microsoft technologies
About Tim, InterKnowlogy & Actus…
InterKnowlogy (www.InterKnowlogy.com) • Tim Huckaby, Founder/Chairman ([email protected])
– 13+ year old services firm – Custom Application Development
– focused on building NUI solutions
– Web, mobile, and desktop
– windows 7/8, Surface, Windows phone 7, azure, Kinect,
•Largest Client: Microsoft
Actus Software (www.Actus-Software.com) • Tim Huckaby, Founder/CEO ([email protected])
•Software Product Company
– Interactive Kiosk and Active Digital Signage
– Multi-Touch, Gesture, and Voice Driven Solutions
itcampro @ itcamp12 # Premium conference on Microsoft technologies
We have come a long way…
The “Green-Screen”
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Image courtesy of Computer History Museum
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Touch Capable Computing Devices Are Not New
Past Future (Now) Present
1972: PLATO IV Touch Screen Terminal
The Samsung SUR40 for Microsoft Surface
Windows 8 - Metro
Apple iPad
itcampro @ itcamp12 # Premium conference on Microsoft technologies
What is Relatively New is:
Inexpensive Touch Capable, Non
Proprietary Hardware
A Windows OS designed for Multi-
Touch
A Consistent Multi-Touch API for .NET
across all Windows Hardware/Software
Platforms
Developer Productivity in Touch
Capable Apps
The Forecast of Multi-Touch Hardware
Capability in Every New Computing
Device in the Short Term
itcampro @ itcamp12 # Premium conference on Microsoft technologies
• HP TouchSmart TM2
• Rates in Windows as 5.0
• Price in October-2010: Starting from: $1499.99*
• Price in March-2011: Starting from: $799.99*
• Price Today (HP TouchSmart 320m ): $579.99
About the Windows 7 Computer I am Using Today…
itcampro @ itcamp12 # Premium conference on Microsoft technologies
HTML 5 Touch Events Spec, Version 2
W3C Editor's Draft 14, November 2011
Defines a set of low-level events that represent
one or more points of contact with a touch-
sensitive surface
Changes of those points with respect to the
surface and any DOM elements displayed upon
it (e.g. for touch screens) or associated with it
(e.g. for drawing tablets without displays).
Addresses pen-tablet devices, such as drawing
tablets, with consideration toward stylus
capabilities.
Source: https://dvcs.w3.org/hg/webevents/raw-file/tip/touchevents.html
itcampro @ itcamp12 # Premium conference on Microsoft technologies
HTML 5 Touch Examples
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
itcampro @ itcamp12 # Premium conference on Microsoft technologies
What is also Relatively New: Multi-Touch User Interfaces Combined with Object
recognition of Physical Objects
The Microsoft Surface is expensive
because it has 5 cameras looking up
that do Object Recognition
Based on a “byte tag” (not a QR
code)
Object Recognition in Windows
The Use Cases are limitless…
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Surface Gaming - Craps
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Windows 8 - Designed for Multi-Touch
Metro - Perfect for Gesture
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Windows 8:
• Metro mode / Desktop Mode
• Actus Interactive Kiosk – Metro
• Semantic Zoom
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
itcampro @ itcamp12 # Premium conference on Microsoft technologies
• Learning made Fun / Entertaining
• Proven as a learning tool beyond just
Children
• Computer based Games are a significant part
of popular culture
• Educational Video Game Curriculums
– MIT, UC Irvine, Minnesota, Mich State, etc.
NUI: Games as a Tool for Learning
itcampro @ itcamp12 # Premium conference on Microsoft technologies
NASA: Mars Bound
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
itcampro @ itcamp12 # Premium conference on Microsoft technologies
World's Fastest-Selling Consumer Electronics Device
… a Future Beyond Gaming
Combination of RGB camera, depth sensor and multi-array microphone
RGB camera delivers three basic color components
Depth sensors “sees” in 3-D
Microphone locates voices by sound and excludes ambient noise
The sensor provides eyes and ears - but it needs a brain
Raw data from the Kinect is just “a whole bunch of noise that someone needs to
take and turn into signal”
Our software finds the signal
Software makes all the magic possible
Skeletal Tracking
Face, Gesture Recognition
Audio Echo cancellation
Audio Beam Forming
Speech Recognition
Machine Learning
What is Kinect?
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Kinect for Windows
Combination of RGB camera, depth sensor and multi-array microphone
RGB camera delivers three basic color components
Depth sensors “sees” in 3-D
Microphone locates voices by sound and excludes ambient noise
Consumer Priced: $249.99
Tracks full-body movement, facial expression, and voice with precision
Kinect sensor bar scans and monitors 1 million points on the body
Kinect for Windows SDK
Skeletal Tracking
Face, Gesture Recognition
Audio Echo cancellation
Audio Beam Forming
Speech Recognition
Machine Learning
itcampro @ itcamp12 # Premium conference on Microsoft technologies
The Good News: • The Kinect SDK for Windows
The Not So Good News: • Windows Only Device; K4W does not work on your
Xbox; you cannot build apps for 18 million Xbox Kinect owners with it.
XBOX Kinect
itcampro @ itcamp12 # Premium conference on Microsoft technologies
The XBOX Kinect for Windows
Touch-less Operating Room
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Neural Interfaces
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Conscious and Non-Conscious Interfaces
• High fidelity EG acquisition device • No conductive gel, etc. • Non-invasive • Use Case Examples:
• Conscious: mind manipulation of objects • Non-Conscious: Ranking music
• First target market: gaming • $300!!!
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Direct API access to all Emotiv detection suite inputs and controls,
including training, profile management and gyro outputs
Program in C++ and compatible languages, .NET (VB.NET, C#), Java
Developer plug-in available for Unity3D
Active developer community (exclusive Developer user forum)
C++ Header files and Import Libraries
…which means…
Wrap the SDK in managed C++ thereby exposing a the API to a C#
application
That is very powerful…
Emotiv Software Development Kit (SDK)
itcampro @ itcamp12 # Premium conference on Microsoft technologies
We are within a decade of building the Holodeck
“The Ultimate NUI Application”
itcampro @ itcamp12 # Premium conference on Microsoft technologies
• We are at the cusp of a revolution in software & hardware
• Hardware and software platforms will ultimately synchronize
• Using Natural User Interface (NUI) technologies improve the User Experience
• The hardware, tools, plumbing and platforms for NUI computing are powerful and available now.
– …and will just continue to get better, cheaper, powerful and more available
• Within a realistic time frame NUI computing will be the norm
• The concepts of NUI change the way we design and build software
Summary
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP
HoloDesk - Direct 3D Interactions with a Situated See-Through Display
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Tim Huckaby
• More info on InterKnowlogy:
www.InterKnowlogy.com Silverlight.InterKnowlogy.com www.vimeo.com/InterKnowlogy
• More info on Actus Software:
www.Actus-Software.com
• About Tim Huckaby… – Founder/Chairman, InterKnowlogy – Founder/Chairman, Actus Software – Microsoft® Regional Director – Southern
California – Microsoft® .NET Partner Advisory Council
Founder / Member – Microsoft® MVP - .NET – INETA Speaker – International .NET Users
Group Association – Board Member: Association for Competitive
Technology – Board Member: Dev Connections – Author / Speaker
• Contact me: Tim Huckaby
E-mail: [email protected]
• Phone:
Office: 760-444-8640 Mobile: 619 990 9200
Blog: http://team.interknowlogy.com/blogs/timhuckaby
FaceBook: http://www.facebook.com/Tim.Huckaby.Carlsbad
LinkedIn: http://www.linkedin.com/in/timhuckaby
Twitter: @TimHuckaby
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Mobile &
Development ITCamp 2012 sponsors
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Mobile &
Development
Agenda
itcampro @ itcamp12 # Premium conference on Microsoft technologies
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Mobile &
Development
itcampro @ itcamp12 # Premium conference on Microsoft technologies
Q & A