behavior analysis based on coordinates of body tags
DESCRIPTION
Behavior analysis based on coordinates of body tags. Mitja Luštrek, Boštjan Kaluža, Erik Dovgan, Bogdan Pogorelc, Matjaž Gams. Jožef Stefan Institute, Department of Intelligent Systems Špica International, d. o. o. Slovenia. Introduction. Problem: Number of elderly increasing - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/1.jpg)
Behavior analysis based on coordinates of body tags
Mitja Luštrek,Boštjan Kaluža, Erik Dovgan,
Bogdan Pogorelc, Matjaž Gams
Jožef Stefan Institute,Department of Intelligent Systems
Špica International, d. o. o.
Slovenia
![Page 2: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/2.jpg)
Introduction
• Problem:– Number of elderly increasing– Too few young people to care for them
![Page 3: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/3.jpg)
Introduction
• Problem:– Number of elderly increasing– Too few young people to care for them
• Solution:– Ambient assisted living technology– Confidence project:
detect falls, monitor well-being
![Page 4: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/4.jpg)
Work presented
• Input: coordinates of tags attached to body
![Page 5: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/5.jpg)
Work presented
• Input: coordinates of tags attached to body
• Task 1: activity recognition (falls)• Task 2: recognition of abnormal walking
![Page 6: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/6.jpg)
Work presented
• Input: coordinates of tags attached to body
• Task 1: activity recognition (falls)• Task 2: recognition of abnormal walking
Machine
learning
![Page 7: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/7.jpg)
Work presented
• Input: coordinates of tags attached to body
• Task 1: activity recognition (falls)• Task 2: recognition of abnormal walking
• Analyze how recogntion is affected by:– Number of tags– Quality of tag coordinate measurements
Machine
learning
![Page 8: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/8.jpg)
Presentation overview
• Sensing hardware• Task 1: activity recognition (falls)• Task 2: recognition of abnormal walking
![Page 9: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/9.jpg)
Sensing hardware
Infraredmotioncapture
Volunteer wearing 12 tagsperforming an activity
![Page 10: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/10.jpg)
Sensing hardware
(x, y, z)for all tags
at 10 Hz
Infraredmotioncapture
Volunteer wearing 12 tagsperforming an activity
![Page 11: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/11.jpg)
Sensing hardware
(x, y, z)for all tags
at 10 Hz
Infraredmotioncapture
Volunteer wearing 12 tagsperforming an activity
Add noise to simulate realistic
hardware
![Page 12: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/12.jpg)
Task 1: activity recognition
![Page 13: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/13.jpg)
FeaturesReference coordinate system
• z coordinates• absolute velocities, z velocities• absolute distances between tags,
z distances
![Page 14: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/14.jpg)
FeaturesReference coordinate system
Body coordinate system
• z coordinates• absolute velocities, z velocities• absolute distances between tags,
z distances
• x, y, z coordinates• absolute velocities,
x, y, z velocities
![Page 15: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/15.jpg)
Features
• z coordinates• absolute velocities, z velocities• absolute distances between tags,
z distances
• x, y, z coordinates• absolute velocities,
x, y, z velocities
Reference coordinate system
Body coordinate system
Angles
![Page 16: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/16.jpg)
Feature vectorst
Snapshot
Snapshot
![Page 17: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/17.jpg)
Feature vectorst-1 tt-2t-9 ...
Activity
Feature vector
![Page 18: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/18.jpg)
Feature vectorst-1 tt-2t-9 ...
t+1
t+2
t+3
...
![Page 19: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/19.jpg)
Feature vectorst-1 tt-2t-9 ...
t+1
t+2
t+3
...
Machine learning with SVM
![Page 20: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/20.jpg)
Experimental setup
• 6 activities:– Walking– Sitting down– Sitting
– Lying down– Lying– Falling
![Page 21: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/21.jpg)
Experimental setup
• 6 activities:– Walking– Sitting down– Sitting
– Lying down– Lying– Falling
• Number of tags: 1 to 12• Noise level: none to Ubisense × 2
![Page 22: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/22.jpg)
Recognition accuracy
![Page 23: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/23.jpg)
Fall detection
• Simple rule:if 3 × recognized fallingfollowed by 1 × recognized lyingthen fall
![Page 24: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/24.jpg)
Fall detection
• Simple rule:if 3 × recognized fallingfollowed by 1 × recognized lyingthen fall
• Fall detection accuracy:– Mostly independent of noise– 93–95 %
![Page 25: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/25.jpg)
Summary of activity recognition
• SVM to train a classifier for activity recognition
• Accuracy: 91 % with Ubisense noise and 4–8 tags
![Page 26: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/26.jpg)
Summary of activity recognition
• SVM to train a classifier for activity recognition
• Accuracy: 91 % with Ubisense noise and 4–8 tags
• Simple rule for fall detection• Accuracy: 93–95 %
![Page 27: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/27.jpg)
Task 2: recognition of abnormal walking
![Page 28: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/28.jpg)
Feature vectors
• 1 feature vector = 1 left + 1 right step
![Page 29: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/29.jpg)
Feature vectors
• 1 feature vector = 1 left + 1 right step
• Features from medical literature on gait analysis
![Page 30: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/30.jpg)
Features
Double support time
![Page 31: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/31.jpg)
Features
Swing time
![Page 32: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/32.jpg)
Features
Support time
![Page 33: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/33.jpg)
Features
Distance of the footfrom the ground
![Page 34: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/34.jpg)
Features
Ankle angle
Knee angle
Hip angle
![Page 35: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/35.jpg)
Features
Ankle angle
Knee angle
Hip angleAnd others...
![Page 36: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/36.jpg)
Machine learning = outlier detectionLocal outlier factor algorithm
![Page 37: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/37.jpg)
Machine learning = outlier detectionLocal outlier factor algorithm
Normal walking
Abormalwalking
![Page 38: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/38.jpg)
Experimental setup
• Normal walking• Abnormal walking:– Limping– Parkinson’s disease– Hemiplegia
![Page 39: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/39.jpg)
Experimental setup
• Normal walking• Abnormal walking:– Limping– Parkinson’s disease– Hemiplegia
• Number of tags: 2, 4, 6, 8• Noise level: none to Ubisense × 2
![Page 40: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/40.jpg)
Recognition accuracy
![Page 41: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/41.jpg)
Summary of recognition of abnormal walking
• Medically relevant features• Outlier detection to recognize
abnormal walking• Accuracy:
92 % with Ubisense noise and 6 tags
![Page 42: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/42.jpg)
Conclusion
• Tag localization + machine learningsuitable for ambient assised living
• Results comparable to competitive approaches (inertial sensors)
![Page 43: Behavior analysis based on coordinates of body tags](https://reader036.vdocuments.net/reader036/viewer/2022062410/56815c37550346895dca26b5/html5/thumbnails/43.jpg)
Conclusion
• Tag localization + machine learningsuitable for ambient assised living
• Results comparable to competitive approaches (inertial sensors)
• Future work:– Test with realistic hardware– Analyze other activities (besides walking)