cs391r - physical simulation environment tutorial
TRANSCRIPT
![Page 1: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/1.jpg)
cs391R - Physical Simulation EnvironmentTutorial
Yifeng Zhu
Department of Computer ScienceThe University of Texas at Austin
September 28, 2020
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 1 / 35
![Page 2: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/2.jpg)
Overview
1 Pybullet - Robovat (Fang, Zhu, Garg, Savarese, et al., 2019)RPL robovat: [Link]Original version - Stanford robovat: [Link]
2 Mujoco - Robosuite (Zhu et al., 2020) [Link]
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 2 / 35
![Page 3: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/3.jpg)
Pybullet vs Mujoco?
Comparison between Mujoco and Pybullet (Erez, Tassa, and Todorov,2015)
Figure: Grasping
Figure: Number of bodies
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 3 / 35
![Page 4: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/4.jpg)
Robovat vs Robosuite?
Robovat:
Designed for grasping, manipulation research, better support of 3Dobjects.
Slower than Mujoco, Fewer options of controllers.
Robosuite:
Designed for Reinforcement Learning / Imitation Learning.
Efficient simulation of objects with simple geometry.
Easier to create procedurally generated scene.
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 4 / 35
![Page 5: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/5.jpg)
Past research using robovat
Figure: HierarchicalPlanning (Fang, Zhu, Garg,Savarese, et al., 2019)
Figure: Grasping (Fang, Zhu,Garg, Kuryenkov, et al., 2018)
Figure: Grasping (Qin et al., 2019)Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 5 / 35
![Page 6: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/6.jpg)
Past research using robosuite
Figure: Reinforcement Learning (Fan et al., 2018)
Figure: Teleoperation for data collection (Mandlekar et al., 2018)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 6 / 35
![Page 7: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/7.jpg)
Simulator
Pros
Cost-efficientEasy to prototype robot experiments
Cons
Not perfectLots of artifacts
How constraint is implemented?Impulse acts as a constraint.
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 7 / 35
![Page 8: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/8.jpg)
Description File
Pybullet - URDF file (Universal Robot Description File)http://wiki.ros.org/urdf/Tutorials
Mujoco - MJCF filehttp://www.mujoco.org/book/XMLreference.html
Both of them are xml data files. Any parameters regarding robots /objects can be defined in URDF/MJCF files.
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 8 / 35
![Page 9: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/9.jpg)
Pybullet
Documentation: [Link]Pybullet is a python wrapper of Bullet physics.
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 9 / 35
![Page 10: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/10.jpg)
Robovat
Robots:
1 Panda
2 Sawyer
Grippers:
1 Panda Gripper
2 Rethink Gripper
Controllers:
1 Joint Position Control
2 Joint Velocity Control
3 Joint Torque Control
4 Inverse Kinematics
A lot of object meshes (including YCB)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 10 / 35
![Page 11: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/11.jpg)
Robovat - Create Environment I
Create Simulator
simulator = Simulator(worker_id=args.worker_id,
use_visualizer=bool(args.debug),
assets_dir=args.assets_dir)
Parse configs for env and policy
env_config, policy_config =
parse_config_files_and_bindings(args)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 11 / 35
![Page 12: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/12.jpg)
Robovat - Create Environment II
Create Environment
env = eval(env_name)(simulator=simulator,
config=env_config,
debug=args.debug)
obs = env.reset()
Plot visual observation
plt.imshow(np.squeeze(obs[env.config.OBSERVATION.TYPE]))
plt.show()
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 12 / 35
![Page 13: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/13.jpg)
Robovat - Create Environment III
Move the gripper
# You need to deep copy pose objects
target_end_effector_pose = env.robot.end_effector.pose.copy()
target_end_effector_pose.x = 0.5
target_end_effector_pose.y = 0.0
# look at robovat/envs/franka_panda_grasp_env.py
env.execute_moving_action(target_end_effector_pose)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 13 / 35
![Page 14: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/14.jpg)
Robovat - Basic Functionality I
Add body from URDF
simulator.add_body(URDF_FILE_PATH,
pose, scale=scale, name=OBJECT_NAME)
Add body from obj file
simulator.add_body(OBJ_FILE_PATH,
pose, scale=scale, name=OBJECT_NAME,
collisionFrameOrientation=[0, 0, 0, 1],
visualFrameOrientation=[0, 0, 0, 1],
baseMass=0.1)
Change dynamics of an object
object.set_dynamics(lateral_friction=1.0,
contact_damping=1.0)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 14 / 35
![Page 15: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/15.jpg)
Robovat - Basic Functionality II
Control a robot (Franka example)
robot.move_to_joint_positions(position_sequence)
robot.move_to_gripper_pose(target_gripper_pose)
robot.move_along_gripper_path(gripper_pose_array)
robot.grip(grip_pos) # grip_pos \in [0, 1]
robot.stop_l_finger() # stop left finger
robot.stop_r_finger() # stop right finger
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 15 / 35
![Page 16: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/16.jpg)
Robovat - Create an environment class
For more details on how to create an environment class, look at theexample robovat/envs/franka panda grasp env.py .
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 16 / 35
![Page 17: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/17.jpg)
Robovat - Contact-rich tasks
By contact-rich, we mean motions that involve interaction with objectsinstead of purely robot arm movements. Contact-rich motions are notperfect in simulation, so we need to keep this fact in mind all the time. Asa consequence, you need to tune some physical parameters in simulation inorder to obtain a more realistic execution.There are several important coefficients that you need to tune:
1 contact damping and contact stiffness
2 lateral friction
3 spinning friciton
4 rolling friction
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 17 / 35
![Page 18: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/18.jpg)
Robovat - Config files
Config files in robovat are Yaml files. For franka panda robots, look atconfigs/robots/franka panda.yaml . You can specify which URDF file to
be loaded in ARM URDF argument.
For environment configurations, please look atconfigs/envs/franka panda envs/franka panda grasp env.yaml
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 18 / 35
![Page 19: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/19.jpg)
Robovat Example - Grasping I
Before grasping - change parameters
robot.l_finger_tip.set_dynamics(
lateral_friction=0.001,
spinning_friction=0.001)
robot.r_finger_tip.set_dynamics(
lateral_friction=0.001,
spinning_friction=0.001)
table.set_dynamics(
lateral_friction=100)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 19 / 35
![Page 20: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/20.jpg)
Robovat Example - Grasping II
After releasing - change parameters back
robot.l_finger_tip.set_dynamics(
lateral_friction=100,
rolling_friction=10,
spinning_friction=10)
robot.r_finger_tip.set_dynamics(
lateral_friction=100,
rolling_friction=10,
spinning_friction=10)
table.set_dynamics(
lateral_friction=1)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 20 / 35
![Page 21: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/21.jpg)
Robovat Example - Grasping
Suppose we have object’s pose information in a variable object pose , ifwe want to move two tips of the gripper symmetrically, run the command:
env.execute_grasping_action(object_pose)
Or if you want to move fingers of a gripper asymmetrically (To createmore stable grasp in simulation), run the command:
env.execute_gentle_grasping_action(object_pose)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 21 / 35
![Page 22: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/22.jpg)
Robovat Example - Pushing
Before pushing - change parameters
table.set_dynamics(
lateral_friction=0.1)
After pushing - change parameters back
table.set_dynamics(
lateral_friction=1.0)
And for more details of executing pushing, please look at the functionexecute action in robovat/envs/franka panda push env.py .
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 22 / 35
![Page 23: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/23.jpg)
Mujoco
Documentation: [Link]What robosuite uses is the python wrapper of Mujoco.
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 23 / 35
![Page 24: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/24.jpg)
Robosuite
Robots:
1 Panda
2 Jaco
3 Kinova3
4 IIWA
5 UR5e
6 Sawyer
7 Baxter
Grippers:
1 Panda Gripper
2 Jaco ThreeFinger
3 Wiping Gripper
4 Robotiq85
5 Rethink Gripper
6 ...
Controllers:
1 Joint Position Control
2 Joint Velocity Control
3 Joint Torque Control
4 Operational Space Control
5 Operational Space Control(Position only)
6 Inverse Kinematics
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 24 / 35
![Page 25: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/25.jpg)
Robosuite - Create a robot
Create the world
from robosuite.models import MujocoWorldBase
world = MujocoWorldBase()
Create a robot
from robosuite.models.robots import Panda
mujoco_robot = Panda()
Add a gripper
from robosuite.models.grippers import gripper_factory
gripper = gripper_factory(’PandaGripper’)
mujoco_robot.add_gripper(gripper)
Add the robot to the world
mujoco_robot.set_base_xpos([0, 0, 0])
world.merge(mujoco_robot)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 25 / 35
![Page 26: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/26.jpg)
Robosuite - Create a tabletop environment
Create a table
from robosuite.models.arenas import TableArena
mujoco_arena = TableArena()
mujoco_arena.set_origin([0.8, 0, 0])
world.merge(mujoco_arena)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 26 / 35
![Page 27: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/27.jpg)
Robosuite - Create a tabletop environment
Add an object which can move around (thus needs a free joint)
from robosuite.models.objects import BallObject
from robosuite.utils.mjcf_utils import new_joint
sphere = BallObject(
name="sphere",
size=[0.04],
rgba=[0, 0.5, 0.5, 1]).get_collision()
sphere.append(new_joint(
name=’sphere_free_joint’,
type=’free’))
sphere.set(’pos’, ’1.0 0 1.0’)
world.worldbody.append(sphere)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 27 / 35
![Page 28: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/28.jpg)
Robsuite - Run simulation
Obtain a MjModel instance which can be used for Mujoco Simulation.
model = world.get_model(mode="mujoco_py")
from mujoco_py import MjSim, MjViewer
sim = MjSim(model)
viewer = MjViewer(sim)
# disable visualization of collision mesh
viewer.vopt.geomgroup[0] = 0
for i in range(10000):
sim.data.ctrl[:] = 0
sim.step()
viewer.render()
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 28 / 35
![Page 29: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/29.jpg)
Robsuite - Miscellaneous I
Get link / joint
# Get Joint indices
jnt_idx = env.robots[0]._ref_joint_indexes
n = len(jnt_idx) # Number of dof
# Get a body’s name
env.sim.model.body_id2name(idx)
# Set to zero pose
env.robots[0].set_robot_joint_positions(np.zeros(n))
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 29 / 35
![Page 30: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/30.jpg)
Robsuite - Miscellaneous II
Get screw axisUnit Screw axis: S = [ω, v]
w0 = np.array(env.sim.data.xaxis)[jnt_idx]
p0 = np.array(env.sim.data.xaxis)[jnt_idx]
v0 = -np.cross(w0, p0)
S = np.hstack([w0, v0])
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 30 / 35
![Page 31: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/31.jpg)
Robsuite - Miscellaneous III
Copmute Spatical Jacobian (a.k.a. Fixed frame)Jacobian: J = AdT (S)J = []
Ts = np.eye(4)
for i in range(n):
row = adjoint(Ts).dot(S[i])
J.append(row)
T = exp2mat(w0[i], v0[i], theta[i])
Twist of end-effector V = J θ̇V = J.dot(theta_dot)
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 31 / 35
![Page 32: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/32.jpg)
Robsuite - Miscellaneous IV
Apply torque (Gravity compensation)
joint_names = ["gripper_z_joint"]
indices = [sim.model.get_joint_qvel_addr(x)
for x in joint_names]
sim.data.qfrc_applied[indices] = sim.data.qfrc_bias[
indices]
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 32 / 35
![Page 33: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/33.jpg)
Other simulation options
1 OpenAI Gym
2 AI2-THOR
3 RLBench
4 CARLA
5 AirSim
6 Interactive Gibson
7 AI Habitat
8 ...
For more details and links, please at the course webpage:https://www.cs.utexas.edu/∼yukez/cs391r fall2020/project.html.
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 33 / 35
![Page 34: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/34.jpg)
References I
Erez, Tom, Yuval Tassa, and Emanuel Todorov (2015). “Simulation toolsfor model-based robotics: Comparison of bullet, havok, mujoco, ode andphysx”. In: 2015 IEEE international conference on robotics andautomation (ICRA). IEEE, pp. 4397–4404.
Fan, Linxi et al. (2018). “SURREAL: Open-Source Reinforcement LearningFramework and Robot Manipulation Benchmark”. In: Conference onRobot Learning.
Fang, Kuan, Yuke Zhu, Animesh Garg, Andrey Kuryenkov, et al. (2018).“Learning Task-Oriented Grasping for Tool Manipulation fromSimulated Self-Supervision”. In: Robotics: Science and Systems (RSS).
Fang, Kuan, Yuke Zhu, Animesh Garg, Silvio Savarese, et al. (2019).“Dynamics Learning with Cascaded Variational Inference for Multi-StepManipulation”. In: Conference on Robot Learning (CoRL).
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 34 / 35
![Page 35: cs391R - Physical Simulation Environment Tutorial](https://reader031.vdocuments.net/reader031/viewer/2022012517/6191d6afdd549a37f77f3a03/html5/thumbnails/35.jpg)
References II
Mandlekar, Ajay et al. (2018). “Roboturk: A crowdsourcing platform forrobotic skill learning through imitation”. In: arXiv preprintarXiv:1811.02790.
Qin, Zengyi et al. (2019). “Keto: Learning keypoint representations fortool manipulation”. In: arXiv preprint arXiv:1910.11977.
Zhu, Yuke et al. (2020). “robosuite: A Modular Simulation Framework andBenchmark for Robot Learning”. In: arXiv preprint arXiv:2009.12293.
Yifeng Zhu cs391R - Robot Learning Online September 28, 2020 35 / 35