![Page 1: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/1.jpg)
Muhammad Al-NasserMohammad Shahab
Stochastic Optimization of Bipedal Walking using Gyro Feedback and
Phase Resetting
King Fahd University of Petroleum and Minerals
March 2008 COE584: Robotics
COE 584/484: Robotics
![Page 2: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/2.jpg)
Outline1. Problem Definition2. Physical Description3. Humanoid Walking System4. Feedback
1. Gyroscope2. Phase Resetting
5. Stochastic Optimization1. PGRL
6. Experimentation7. Comments
![Page 3: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/3.jpg)
Problem Definition
Authors Felix Faber & Sven Behnke, Univ. of
Freinbrg, Germany
Problem Statement: “to optimize the walking pattern of a
humanoid robot for forward speed using suitable metaheuristics”
![Page 4: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/4.jpg)
First Humanoid Robot!
• 1206 AD
• Ibn Ismail Ibn al-Razzaz Al-Jazari
• A boat with four programmable automatic musicians that floated on a lake to entertain guests at royal drinking parties!!
![Page 5: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/5.jpg)
Problem Definition
• Problems?
Nonlinear Dynamics: i.e. complex system to control
Sensor Noise:CameraGyroscopeUltrasonicForce…
Environment Disturbances:Unknown surface…
Inaccurate Actuators:Motors…
![Page 6: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/6.jpg)
Physical Description
• Jupp, team NimbRo
• 60 cm, 2.3 kg
• Pocket PC
![Page 7: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/7.jpg)
Physical Description
• Pitch joint to bend trunk
• Each leg• 3DOF hip• Knee• 2DOF ankle
• Each arm• 2DOF shoulders• elbow
![Page 8: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/8.jpg)
Humanoid Walking System• One Approach
• Model-Based (Geometric Model)• Accurate Model• Solving motion equations for all joints (offline)• 19 Degrees of Freedom• Nonlinear model equations• Computational complexity
ControllerLeg Motion
Trajectory
Joints motor positions
’s
Robot walks!
![Page 9: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/9.jpg)
Humanoid Walking System• 2nd Approach
Controller
Joints motor positions
’s
• Central Pattern Generators (CPG)• Sinusoid joint trajectory generated• Bio-Inspired• no need for model
![Page 10: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/10.jpg)
Humanoid Walking System• Open-loop (no feedback) Gait
• Mechanism1. Shifting weight from one leg to the other2. Shortening the leg not needed3. Leg motion in forward direction
![Page 11: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/11.jpg)
Humanoid Walking SystemOpen-loop GaitClock-driven, Trunk phase being central
clockTrunk Phase (with ‘foot step frequency’ )
Right leg motion phase = Trunk + /2Left leg motion phase = Trunk - /2
time
-
![Page 12: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/12.jpg)
Humanoid Walking System
• (continued)
Kinematic Mapping
Left
Right
Leg
Foot
yLeg
pLeg
rLeg
Leg
pFoot
rFoot
Foot
r: Rollp: Pitchy: Yaw
“Human-Like Walking using Toes Joint and Straight Stance Leg” by Behnke
Swing
Swing is leg swing amplitude
Is leg extension
![Page 13: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/13.jpg)
Feedback•Overall Control System
Joints motor positions
’s
Mapping
Controller
1. Gyroscope: Gyro = Inclination (Balance) Angular Velocity
2. Force Sensing Resistors: foot touch ground trigger (‘High’ or ‘Low’)
![Page 14: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/14.jpg)
Feedback•Gyroscope
– device for measuring orientation, based on the principles of conservation of angular momentum
– Remember Physics 101!
![Page 15: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/15.jpg)
Feedback P-Control
Gyro increase = robot fall
• Proportional Control • reactive action proportionate to ‘error’ (Error = sensor value –
desired value)• Desired values = zero (i.e. no inclination)
• Other: Proportional-Integral Control• action proportionate to ‘error’ and proportionate to
accumulation of ‘error’
Joints motor positions
’s
Gyro
pGyro
p
rGyro
r
FootOldFootNew K
K
![Page 16: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/16.jpg)
Feedback• Overall System
Joints motor positions
’s
Mapping
P-Control
![Page 17: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/17.jpg)
Feedback• Overall System
Controller
Joints motor positions
’s
Online Adaptation(Stochastic Optimization)
• Adaptive Control• Online tuning of ‘parameters’ of the
controller
![Page 18: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/18.jpg)
Stochastic Optimization Approach
• Goal:– Adjust parameters to achieve faster and
more stable walk.
• Fitness function (cost function) is used to express optimization goals (i.e. speed & robustness)
f (.): RN--->RN: number of parameters of interest
)(xf
![Page 19: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/19.jpg)
Stochastic Optimization Approach
• The parameters are
Kinematic Mapping
(Behnke paper)
![Page 20: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/20.jpg)
Stochastic Optimization Approach
• We evaluate f in a given set of parameters• x = [x1 , x2 , ... , xN] (Table 1)
• Now, how to find the values of the parameters that will result in the highest fitness value?– use a metaheuristic method called PGRL
?+1
d <dexp
![Page 21: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/21.jpg)
Policy Gradient Reinforcement Learning (PGRL)
• An optimization method to maximize the walking speed
• It automatically searches a set of possible parameters aiming to find the fastest walk that can be achieved
![Page 22: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/22.jpg)
Policy Gradient Reinforcement Learning
• How dose PGRL work?1st : generates randomly B test polices {x1, x2,…,
xB} • around an initially given set of parameter vector
xπ
• (where x = [x1 , x2 , … , xN])
– Each parameter in a given test policy xi is randomly set to
• where 1≤i ≤B and 1 ≤j ≤N• ε is a small constant value
jjj xorxx ,
![Page 23: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/23.jpg)
Policy Gradient Reinforcement Learning
• 2nd: – the test policy is evaluated by ‘fitness
function’.
• For each parameter j is grouped into 3 categories
• Which are• depending on where the jth parameter is
modified by –ε, 0, +ε
jjj SorSS 0,
![Page 24: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/24.jpg)
Policy Gradient Reinforcement Learning
• Next 3rd , construct vector a=[a1, a2, …, aN]
• As are average of each category
![Page 25: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/25.jpg)
Policy Gradient Reinforcement Learning
• Then 4th (finally), adjust xπ as follows
where η is a scalar step size
![Page 26: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/26.jpg)
Extension to PRLG
• Adaptive step sizeafter g steps:
where s: the number of fitness functions
evaluationsS: maximum allowed number of s
![Page 27: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/27.jpg)
Overall
• Overall System
Controller
Joints motor positions
’s
PGRLxπ
![Page 28: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/28.jpg)
Experiment
![Page 29: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/29.jpg)
Results
![Page 30: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/30.jpg)
Results
• speed is 21.3 cm/s
• fitness is 1.36
• Speed is 34.0 cm/s
• Fitness is 1.52
After 1000 iteration
Initial
60%
![Page 31: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/31.jpg)
Parameters
![Page 32: Stochastic Optimization of Bipedal Walking using Gyro Feedback and Phase Resetting](https://reader036.vdocuments.net/reader036/viewer/2022062322/568145f5550346895db2fe98/html5/thumbnails/32.jpg)
Glossary
• Stance leg: – the leg which is on the floor during the walk.
• Swing leg:– the leg which moving during the walk.
• Single support:– The case where robot is touching the floor with one
leg.
• Double support:– The case where robot is touching the floor with both
legs.