aibo companion doas – group 1 aitor azcarate onaindia abeer mahdi zhiwei zhan ning yang...
TRANSCRIPT
![Page 1: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/1.jpg)
Aibo companion
DOAS – group 1Aitor Azcarate Onaindia
Abeer Mahdi
Zhiwei Zhan
Ning Yang
Supervisor: Frans Groen
![Page 2: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/2.jpg)
Introduction
In the future more help is needed for elder care. The solution is to make use of intelligent home with
intelligent agents for interaction : for example Aibo We looked at the possibilities of Aibo as elder
companion We implemented some interesting tasks. (mentioned
later)
![Page 3: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/3.jpg)
Tasks that should be realized by Aibo for elderly care
1.Sensing intelligence-Follow the elder-Detect when a person is falling down-Detect location
2. Communication intelligence -interaction with persons and nurse-interface with home equipments like tv.
3.Emotional intelligence-Face feature recognition-Initiate a conversation -Inactivate mode -Pro-active suggestions-Reaction to sound
![Page 4: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/4.jpg)
System Architecture
![Page 5: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/5.jpg)
Literature study
Following the elder, algorithms that can be used are:– Mean-shift algorithm– Optical flow motion– Sound direction detection– A combination of the above
Localization of Aibo– ‘map using’ methods– Appearance-based methods
Face recognition– Performance being evaluated
Aibo in the future – As the universal interface to the home equipment
![Page 6: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/6.jpg)
Tasks implemented
We chose basic functions that can be extended in the future.
Reaction to sound– Make Aibo react to voice commands, by performing specific
tasks
Face detection– When seeing a face Aibo walks to the face and sits down
Pink Ball tracking– Make Aibo look for a pink ball and walk to it.
![Page 7: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/7.jpg)
Available tools
Development tools– Openr SDK (c++ programming)– Tekkotsu (c++, java and matlab) – Remote framework (combined with visual c++)– R-code (script language, which we used)
Graphic user interface– Motion editor (creating new motions for Aibo)
Control panel– Sony entertainment player (control Aibo)
![Page 8: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/8.jpg)
Reaction to sound
Aibo has a predefined voice command recognition list of 54 voice commands
We implemented some reactions to certain voice commands, some of these reactions are motions and others are voices that we recorded.
We experimented with Sony motion editor to create new motions, but we noticed that it doesn’t support Aibo ERS-7 very well.
![Page 9: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/9.jpg)
Results
We noticed that Aibo sometimes cannot recognize the voice command correctly, despite the distance of the speaker.
--Demo
![Page 10: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/10.jpg)
Face detection
R-Code has a predefined function for face detection
We implemented Aibo to detect a face, if found, Aibo will walk to it and sit down
![Page 11: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/11.jpg)
Results
Aibo can detect faces correctly. However, in this stage it can only detect the
front side of the face. --Demo
![Page 12: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/12.jpg)
Pink Ball tracking
R-Code has a predefined function for searching the Pink ball
This can be extended to follow the elder in the intelligent home
![Page 13: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/13.jpg)
Result
The results are not accurate, this is because the basic functions are not accurate, for example the walk function. This can be caused by the physical construction of Aibo
Aibo can follow the pink ball correctly for an average of 4 times, after that it losses the ball.
The closer the ball is, the less chance of finding the ball.
![Page 14: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/14.jpg)
Result (2)
--Demo pink ball
![Page 15: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/15.jpg)
Conclusions
why we chose R-code– Simple script language to learn in a short time – Compact compared with for example Java, which take 400
lines of code to make Aibo nod it’s head (with R-code it is a few lines)
After experimenting with R-code, we noticed the following
– It is not an open source, that make it not extendable– It is not suitable for low level programming or academic use – The predefined functions are not very accurate
![Page 16: Aibo companion DOAS – group 1 Aitor Azcarate Onaindia Abeer Mahdi Zhiwei Zhan Ning Yang Supervisor: Frans Groen](https://reader035.vdocuments.net/reader035/viewer/2022062222/5697bf7d1a28abf838c84c54/html5/thumbnails/16.jpg)
Conclusions (2)
Aibo is currently not capable of being the companion for the elders
– It is a nice entertainer but not really suitable for critical circumstances
– it would be more capable if aibo can interact with medical aids like nurses in the hospital
The things that should be done to make Aibo more suitable are
– Extend the voice recognition list– Improve the basic functions (e.g. walking)– Make a good API