hicss17 asakawa
TRANSCRIPT
![Page 1: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/1.jpg)
Cognitive Assistant for The Real-world Accessibility
Chieko Asakawa IBM Fellow, IBM ResearchIBM Distinguished Service Professor, Carnegie Mellon University
![Page 2: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/2.jpg)
© 2011 IBM Corporation2
Cognitive Ability of Visually Impaired People
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0.8
1.0
1.2
1.4
1.6
1.8
2.0
2.2
2.4
2.6
2.8
3.0
3.2
3.4
3.6
音声速度(倍速)
再生
率 (%)
A
B
C
D
E
F
G
Reca
ll Rat
e (%
)
Speech speed
X X X X X X X X X X X X X XX
![Page 3: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/3.jpg)
3 ©2016IBMCorporation
![Page 4: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/4.jpg)
4 ©2016IBMCorporation
Information accessibility Mobility
![Page 5: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/5.jpg)
Braille Digitalization
Paper Braille TypewriterComparison:
Braille and Printed Japanese Dictionary
Digital Braille System
![Page 6: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/6.jpg)
Home Page Reader
6 © 2016 IBM Corporation
nThe first practical voice browsers in the world
nBecame a product in 1997 and translated into 11 languages
IBM Home Page Reader (1999) Japanese, Italian, French, German, Spanish, English (U.S.,
U.K.)
“Home Page Reader is my small window to the world. I can read, write, and access information.I can do everything to participate in society.”
— comment from a user, 1997
![Page 7: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/7.jpg)
©2016IBMCorporation
![Page 8: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/8.jpg)
Smart phone demo
©2016IBMCorporation
![Page 9: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/9.jpg)
Exponential Increase of Accessible Information (Conceptual Chart)
©2016IBMCorporation
Paper BrailleDigital Braille
Voice Web access
1980 20001990 2010
Amou
nt o
f ac
cess
ible
info
rmat
ion
9
![Page 10: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/10.jpg)
https://www.youtube.com/watch?v=cTHa_5Y5IA4
10
https://theses.lib.vt.edu/theses/available/etd-05072012-161514/unrestricted/DAngio_PC_D_2012_1.pdf
http://www.ted.com/talks/dennis_hong_making_a_car_for_blind_drivers
Blind Driver Challenge at the Rolex 24 At Daytona with Audio Description
![Page 11: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/11.jpg)
Uber Self-driving Car Pilot
11 ©2016IBMCorporation
![Page 12: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/12.jpg)
Walk by myself Finding discount signsFinding restaurantsShopping
The Second Challenge: Real World Accessibility
©2016IBMCorporation12
![Page 13: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/13.jpg)
Cognitive Assistant
13
Augment missing or weakened abilities by the power of cognitive computing.
©2016IBMCorporation
![Page 14: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/14.jpg)
“甦るヒーローライブラリー第3集 光速エスパー”, TCエンタテインメントより
Chika
14 ©2016IBMCorporation
![Page 15: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/15.jpg)
©2016IBMCorporation
https://www.ted.com/talks/chieko_asakawa_how_new_technology_helps_blind_people_explore_the_world
![Page 16: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/16.jpg)
Foundation of Cognitive Assistant
©2016IBMCorporation16
LocalizationRadio-wave
VisionDead reckoning
RecognitionObjectsPeople
Environment
KnowledgePersonal
Social mediaEncyclopedic
InteractionSpeechGesture
Sonification
![Page 17: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/17.jpg)
Localization
Quasi-zenith satellitesfor Japan
2018
http://qzss.go.jp/technical/technology/tech01_orbit.html
Accurate indoor localizationBLE Beacons + Dead reckoning
WIFI
![Page 18: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/18.jpg)
Recognition of Environment
Vision-based EnvironmentRecognition
Infrared Depth Sensor on smartphonesGoogle Tango
https://youtu.be/Qe10ExwzCqk
![Page 19: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/19.jpg)
Recognition of Things
Watson Vision API
coffeeespresso
wineglass
glasses potterydish
![Page 20: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/20.jpg)
Oil filterChicken soup
Pen sharpener Spice bottle
Cristmas socks Cheetos
Beer bottle Diet coke
Photos for learning taken by blind subjects
Incorrect resultsBefore leaning
Correct resultsAfter learning
Personal Object Recognition
Honorable mention at CHI2017
![Page 21: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/21.jpg)
©2016IBMCorporation
Knowledge – Importance of Open data
Detailed pedestrian maps
Crowd prediction
RestaurantShops
Items
Temporal changes
21
Accessibility information
![Page 22: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/22.jpg)
Interaction
©2016IBMCorporation
Approaching to human accuracy
http://blogs.microsoft.com/next/2016/10/18/historic-achievement-microsoft-researchers-reach-human-parity-conversational-speech-recognition/#sm.0019gj85a12umd89ys32kew39p8d6
Autonomous Bus with Voice Dialog
https://www-03.ibm.com/press/jp/ja/pressrelease/50250.wss
![Page 23: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/23.jpg)
デモ
©2016IBMCorporation
![Page 24: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/24.jpg)
Cognitive Assisting Robot
Manuela Veloso is leading the CoBot project. On November 18, 2014, the CoBot robots jointly reached 1,000km of autonomous localization and navigation.
![Page 25: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/25.jpg)
©2016IBMCorporation
Sep 2016 - August 2019 Awarded Amount: $1millionInvestigator:• Kris Kitani, Computer Vision, Robotics Institute, CMU• Manuela Veloso, Robotics, Robotics Institute, CMUAdvisor:• Chieko Asakawa, IBM Fellow, Robotics Institute, CMU
National Robotics InitiativeA Cognitive Navigation Assistant for the Blind
25
Implementation of a navigation assistant that uses a collection of sensing modalities and algorithms to guide a blind person through the knowledge landscape of an unfamiliar environment.
![Page 26: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/26.jpg)
World Robot Summit
©2016IBMCorporation
Industry robots Services robots Disaster response robots
26
Japanese government is planning a robot competition event in 2020, the same year, Olympics and Paralympics will be held in Tokyo.
http://www.nedo.go.jp/robot/about.html http://www.miraikan.jst.go.jp/exhibition/future/robot/robotworld.html#slide1
https://www-robotics.jpl.nasa.gov/tasks/taskImage.cfm?TaskID=236&tdaID=700043&Image=501
![Page 27: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/27.jpg)
18th century, the mathematician Euler authored a leading textbook on mathematics after he became blind.
Captions for TV programs were invented for deaf people, but are now widely used in sports bars and to help immigrants, etc.
Based on Self-standing wheelchair.
2011, voice dialogtechnologies for smartphones
TelephoneKeyboard
Voice recognition
Voice dialogTV Captions
Voice synthesis and OCR
Modern Mathematics
Autonomic vehicles
Segway
“Blind driving challenges”
Early use of keyboards was a to help people with hand-and-motor disabilities in writing
First consumer product was a reading assistant for the blind in the 1970s.
1876, Alexander Graham Bell invented the phone after teaching hearing impaired people.
Innovations Flourish From Accessibility NeedsSmart
Machines
27
17th century, Dom Perignon. Champagne
©2016IBMCorporation
![Page 28: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/28.jpg)
1960sTalking Typewriter
1960s Talking Typewriter
1975 1403 Braille Printer
1984 Talking 3270 Terminal
1990 VoiceType™
2008 Social Accessibility
1998 ViaVoice®
2000 Accessibility Center
1997 Home Page Reader
1988 ScreenReader/DOS
2004 aDesigner
2007 Eclipse Accessibility Tools Framework
2009 Rich Internet Accessibility Technologies
1914 First IBMer with disabilities
IBM History of Accessibility
28 ©2016IBMCorporation
2012 Brazil Accessible Vocational Training Project
2011 Senior Workforce Project 1984Talking 3270 Terminal
![Page 29: Hicss17 asakawa](https://reader033.vdocuments.net/reader033/viewer/2022051709/587587711a28ab901c8b5079/html5/thumbnails/29.jpg)
©2016IBMCorporation
2010 20302020
Amou
nt o
f Ac
cess
ible
real
wor
ld in
form
atio
n(c
once
ptua
l num
ber)
Future: Real-world Accessibility – Exponential Growth
Cognitive Assistant Lab @ CMU
Smartphone accessibility
Standardization?Core technology development
Mobile Cognitive ?
Apple Siri
Microsoft Seeing AI project
Google Tango Phone
Tokyo NihonbashiDemo
Watson Vision API
Adoption?Early days
2016Dec.
Tokyo OlympicsParalympics