better motion control using accelerometer/gyroscope sensor fusion
DESCRIPTION
This is my Droidcon Tunis 2012 presentation about sensor fusion between gyroscope and accelerometer sensors.TRANSCRIPT
![Page 1: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/1.jpg)
Better motion control using accelerometer/gyroscope sensor fusion
Gabor [email protected]
Sfonge Ltd.http://www.sfonge.com
![Page 2: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/2.jpg)
Where were we?
● Droidcon 2011, London: Motion recognition on Android devices● http://mylifewithandroid.blogspot.com/2011/10/my-
presentation-about-motion.html
● Processing only the accelerometer for motion recognition
![Page 3: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/3.jpg)
Acceleration
Acceleration caused bythe change of direction
v1
v2dV
Acceleration caused by the change of velocity
v1
v2
dV
a=ΔVΔ t
![Page 4: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/4.jpg)
Extract motion information from accelerometer data
● Accelerometer data is a vector, having 3 axes (x,y,z)● This vector has the following components:
● Gravity acceleration– Pointing toward the center of the Earth
– Value of about 10 m/s2
– That's what we measure when the accelerometer is used to calculate tilt
● Any other acceleration the device is subject to– Added to the gravity acceleration
– “Disturbs” tilt measurement in gaming (swift movements cause acceleration) – hence the reason for gyroscopes
– Can be used for movement detection
![Page 5: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/5.jpg)
Measured acceleration
![Page 6: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/6.jpg)
Absolute value
● x, y, z: acceleration vector components● g – value of the gravity acceleration (can be
approximated as 10)
a=√x2+ y2+ z2−g
![Page 7: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/7.jpg)
Snap – one wayMovement starts: accelerating
Movement ends: decelerating
![Page 8: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/8.jpg)
Droidcon 2011 flashback
● Conclusions:● Power consumption is a problem● Some neat functionality can be implemented by doing
pattern recognition on the acceleration vector's absolute value
● In general case the gravity and motion acceleration components cannot be separated
● You can try to use an additional sensor like the gyro to help the separation
![Page 9: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/9.jpg)
Gyroscope
● Very new phenomenon as gyroscopes suitable for consumer electronic devices appeared very recently
● First appearance: Wii Motion Plus accessory, 2009 June
● First Android smart phone: Nexus S (end of 2010)
● Pros:
● Not sensitive to gravity● Cons:
● Currently supported only by high-end Android phones● Drift problems (more about that later)
![Page 10: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/10.jpg)
Compass
● Measures the device orientation wrt. the magnetic vector of the Earth
● This vector points toward the magnetic center of the Earth
– It has a component that points to the magnetic North pole – that's what we use for orientation
– Beware of the z component! (also called magnetic inclination). If the device is not held horizontally, the downward vector element influences the measurement
● Pros:
● Can be used to deduce gravity, not sensitive to motion acceleration
● Widely available in Android devices
● Cons:
● Requires calibration
● Sensitive to metal objects, magnetic fields (e.g. electric motors)
![Page 11: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/11.jpg)
This time it is gyroscope only
![Page 12: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/12.jpg)
Gyroscope
![Page 13: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/13.jpg)
Gyroscope measurement data
● Measures rotation around 3 axes● More exactly: measures rotation speed (angular
velocity) around the axes
v x=Δφ
Δ t
![Page 14: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/14.jpg)
Getting the rotation angle
● Get the angle difference
● Get the absolute angle
Δφ=v x Δ t
φ '=φ+Δφ
![Page 15: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/15.jpg)
Drift
![Page 16: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/16.jpg)
Noise
![Page 17: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/17.jpg)
Gyro as support sensor
● Because of accumulating error, gyro alone can be rarely used
● But● The accelerometer has no accumulated error but
has the gravity component problem● The gyro has accumulated error but is not sensitive
to gravity
● Sensor fusion: the use of multiple sensors so that they compensate each other's weaknesses
![Page 18: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/18.jpg)
Accelerometer-gyro fusion
● The easy way● Use the virtual sensors that calculate gravity and
linear acceleration from multiple sensors
● The hard way● Process raw accelerometer and gyroscope data to
yield the motion information you need
![Page 19: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/19.jpg)
Virtual sensors
Gravity and motion accelerationdeduced from the accelerometerand the gyroscope
Roll/pitch/yaw from the compass
Drift-compensated gyroscope
![Page 20: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/20.jpg)
Drift-compensated gyroscope
![Page 21: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/21.jpg)
The hard way
● Why would you go the hard way?● Sensor fusion co-processing provided by the phone
is not precise enough or can have undesirable properties (like auto-calibration in Nexus S)
● Virtual sensors are not available (is there any such case with gyro-equipped phone?)
● You would like to understand how it works and what to expect from built-in sensor fusion
● Just for the fun of it :-)
![Page 22: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/22.jpg)
What we want
● Remember: accelerometer measures the sum of gravity and motion acceleration
● Kills two use cases:● If you need device tilt, the motion acceleration
component corrupts the measurement● If you want motion acceleration, it is impossible to
subtract the gravity acceleration in a general case
● Separate gravity and motion acceleration with the help of the gyroscope
![Page 23: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/23.jpg)
Idea
![Page 24: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/24.jpg)
Idea in words
● Pick a reliable gravity vector measurement (make sure that there's no motion then)
● If you detect motion (more about later), rotate the previous gravity vector using the gyroscope data and use it as gravity vector estimation
● Subtract this gravity vector estimation from the measured acceleration – this yields the motion acceleration
![Page 25: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/25.jpg)
Updating the gravity vector estimation
● The gravity vector estimation has to be updated time to time as rotation angle errors accumulate
● If we detect an acceleration measurement where there is no motion acceleration, we can take it as new reliable gravity vector estimation
● Remember slide #7: if the absolute value of the accelerometer output is close to the Earth's gravity, we can assume that there's no motion → the gravity vector estimation can be updated with the current accelerometer output
![Page 26: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/26.jpg)
Implementation
● Example program: http://www.sfonge.com/forum/topic/example-application-accelerometergyroscope-processing-android
![Page 27: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/27.jpg)
Now what?
3D linear acceleration signal of a well-known motion
![Page 28: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/28.jpg)
Recognizing motion
● 3D linear acceleration signals are not so intuitive
● Motion recognition:● Record acceleration pattern of reference motion
and compare with these references● Convert from acceleration domain to something
more intuitive like velocity– Accelerometer/gyroscope bias will become linearly
growing drift after you integrate the acceleration signal!
![Page 29: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/29.jpg)
Walking with swinging hand
![Page 30: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/30.jpg)
Walking with steady hand
![Page 31: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/31.jpg)
Cutting corners
![Page 32: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/32.jpg)
Conclusions
● Each sensor has strengths and weaknesses● Combine them and they compensate each
other● Some sensor fusion is already built-in● If not → don't worry, come up with your own, it's
fun!● Motion recognition based on 3D linear
acceleration signal is much more exact than doing the same from 1D signal
![Page 33: Better motion control using accelerometer/gyroscope sensor fusion](https://reader034.vdocuments.net/reader034/viewer/2022042512/5558c5f4d8b42a995d8b4726/html5/thumbnails/33.jpg)
Questions?