GoogleIO @ Java Meetup Ahmedabad

Build next generation apps with eyes and ears using Google Chrome

25th Jun, 2014

Dhruv GohilFrom Ishi systems

Treat your users as human!

UX - 101

Tools should fit human not the other way around!

Now we have books on UX!

• With computers, we lost 'human touch' in interaction

• Touch screens brought it back, but frankly its not the natural way to do 'things'

Let's get back to basics..

• I have to tell you something.. I will speak and you will listen.

• So why it has to be different for computers?!

Enter the new era!

• Now browsers can speak to you.

• And browsers can hear you too!

http://updates.html5rocks.com/2013/01/Voice-Driven-Web-Apps-Introduction-to-the-Web-Speech-API

Demo time!

https://www.google.com/intl/en/chrome/demos/speech.html

Do it yourself!

You talk too much...

● It's time to listen..

● Yes! That's all we need!

Off-line mode too!

http://syl22-00.github.io/pocketsphinx.js/

This is how translation should work!

http://www.moreawesomeweb.com/demos/speech_translate.html

Command mode!

● I am married.... ammm.. Can I command my computer to do things?

https://www.talater.com/annyang/ https://github.com/TalAter/annyang

But can you see me?

http://wolframhempel.github.io/photobooth-js/

Ok but how its useful?

http://revealjs.herokuapp.com/

Time to demo!, again.! :-)

https://github.com/willy-vvu/reveal.js

What was that?

● Magic of 'Motion detection algorithms'

● Over simplified How To:

– Capture video [getUserMedia()]

– Draw frames [ html5 <canvas> ]

– Run Algorithm to detect change between frames [ .js]

● If you are deeply interested

– Read : http://labs.littlemissrobot.com/behind-the-mirror/

http://www.adobe.com/devnet/html5/articles/javascript-motion-detection.html

More!

● https://github.com/foo123/HAAR.js#live-examples

– Interactive Face Detection

– Many Faces Detection

– Interactive Mouth Detection

● http://www.soundstep.com/blog/experiments/jstracking/

– Motion tracking ● http://www.soundstep.com/blog/experiments/jsdetection/

– The Magical Xylophone

But Why chrome?

Its time to...

WebSpeech API fundametals

● Audio capture

● Request authorization

● Audio processing pip line

– Nodes (FFT , Gain, Filter...)

– http://chimera.labs.oreilly.com/books/1234000001552/● Join to destination (Speakers)

● + Visualizations

GetUserMedia API fundamentals

● Audio + Video capture

● Request authorization

● Capture

● Draw (Hidden)

● Analyze + Process

● Re-Draw

And finally...

लगता ह ै बर्फर िगर रही है!

https://github.com/IonicaBizau/WebcamSnowhttp://ionicabizau.net/WebcamSnow/