autonomous robots: design and testing using virtual

28 Sept. 2007

1

CERMA 2007

Prof. Ph.D. Prof. Ph.D. Fernando Fernando OsórioOsório -- Applied Computing / Applied Computing / UnisinosUnisinosProfaProfa . Ph.D. . Ph.D. SoraiaSoraia MusseMusse -- Computing Science / PUCComputing Science / PUC --RSRSProf. Prof. M.ScM.Sc. . FarleiFarlei HeinenHeinen -- Computing Eng. / Computing Eng. / UnisinosUnisinosM.ScM.Sc. Milton Roberto . Milton Roberto HeinenHeinen -- Ph.D. Student at UFRGSPh.D. Student at UFRGSProf. Prof. Ph.DPh.D Christian Christian KelberKelber -- Electrical Eng. / Electrical Eng. / UnisinosUnisinosGustavo Gustavo PessinPessin -- M.ScM.Sc. at . at UnisinosUnisinos

GRAPHIT - Computer Graphics and Vision Group (Unisinos/PUC-R S)GPVA - Autonomous Vehicles Research Group (Unisinos)GIA - Artificial Intelligence Research Group (Unisinos)RBV - Rede Brasileira de Visualização [FINEP/Brazil]

IEEE / DVP - Distinguished Visitors Program Latin America

Applied Computing Applied Computing PostPost--grad. Programgrad. Program-- PIPCAPIPCA

UNISINOS University UNISINOS University -- BrazilBrazil

Autonomous robots: Autonomous robots: Design and Testing using Design and Testing using

Virtual Reality and Physical SimulationVirtual Reality and Physical Simulation

Research group:

Fernando Osório, Ph.D. / IEEE CS DVP

28 Sept. 2007

2

CERMA 2007

IEEE / DVP - Distinguished Visitors Program Latin America

Prof. Fernando Santos Osório - IEEE MemberApplied Computing Research Post-Graduation Program - PIPCAUNISINOS University - Brazil (Porto Alegre - Southern Region)IEEE Computer Society DVP Program

Jesuit UniversityWith: - 30.000 Students- 900 Professors- 16 PPGs (post-grad programs)

28 Sept. 2007

3

CERMA 2007

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Robots and Agents - Control

Multi-Agents Systems - Knowledge

5. Applications: Autonomous Robots VR Simulation Tools

6. Conclusions and New TrendsVídeo Demo Web/Java

28 Sept. 2007

4

CERMA 2007

Introduction VR - Virtual Reality

[Paul Milgram et al. 95]

From REAL to VIRTUAL3D + Immersion + Interaction

Virtual Reality

28 Sept. 2007

5

CERMA 2007

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

Virtual RealityVirtual Reality...

* VRML - 3D Worlds (Geometry)

* QTVR - Panorama 3D (Images)

Introduction VR - Virtual Reality

\\VRMLVRML

Virtual Reality

28 Sept. 2007

6

CERMA 2007

2424

IRISA / INRIA IRISA / INRIA -- FranceFrancehttp://www.irisa.fr/lagadic/demo/demohttp://www.irisa.fr/lagadic/demo/demo--ar3/demoar3/demo--ar3ar3--eng.htmleng.html

Real World Integrated with Virtual Objects

Augmented Reality

28 Sept. 2007

7

CERMA 2007

Virtual Reality

Virtual RealityVirtual Reality...

* 3D Virtual Environment

* Interaction => Virtual Reality Devices

* Immersion => Virtual Reality Devices

* Realism => Graphical Realism (photo-realism)

Movements

Interaction Real x Virtual

"Physics Realism"

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

How to do it?

28 Sept. 2007

8

CERMA 2007

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

28 Sept. 2007

9

CERMA 2007

1. Introduction



[Funge 1999]

28 Sept. 2007

10

CERMA 2007

1. Introduction



[Funge 1999]

28 Sept. 2007

11

CERMA 2007

1. Introduction



[Funge 1999]

28 Sept. 2007

12

CERMA 2007

1. Introduction



[Funge 1999]

28 Sept. 2007

13

CERMA 2007

1. Introduction



[Funge 1999]

PhantomPhantom

HaptionHaption

CyberForceCyberForce

OmegaOmega

28 Sept. 2007

14

CERMA 2007

1. Introduction



[Funge 1999]

[Ari Chapiro - Dance]

[CromosLab]

Agents

28 Sept. 2007

15

CERMA 2007

1. Introduction



[Funge 1999]

28 Sept. 2007

16

CERMA 2007

1. Introduction



[Funge 1999]

Knowledge

AutonomousBehaviour

28 Sept. 2007

17

CERMA 2007




[Funge 1999]

Virtual World

xx

Real World

28 Sept. 2007

18

CERMA 2007

Presentation Topics

Agenda:










5. Autonomous Robots VR Simulation Tools

6. Conclusions and New TrendsVídeo Demo Web/Java

28 Sept. 2007

19

CERMA 2007



[Funge 1999]

Virtual World

xx

Real World

Increasing Reality in VR Applications: Physical and Behavioral Simulation


28 Sept. 2007

20

CERMA 2007

1. Introduction

RealisticVR

Increasing Reality in VR Applications: Physical and Behavioral Simulation

Virtual World

xx

Real World

Real WorldReal WorldSimulationSimulation

28 Sept. 2007

21

CERMA 2007

Simulation Tools:

* ODE - Open Dynamics Engine

* OpenSteer

* PhysX AGEIA

* Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods

VR Simulation: Some important questions...


28 Sept. 2007

22

CERMA 2007

Simulation Tools:


* OpenSteer

* PhysX AGEIA


VR Simulation: Some important questions...


Physics:

Physical structure: resistance, mass, density, elasticity;Position and orientation in the 3D space; Kinematics and Dynamics;Linear and angular velocities; Motion (w/ forces and torques), trajectories; Acceleration, deceleration;Attraction and repulsion;Gravity, friction, inertia; Kinetic and potential energy; Laws of energy conservation, linear and angular momentum;Collisions and reaction to collisions;Steering models (wheeled cars, aircrafts, projectiles, boats and ships); Articulated Rigid Bodies Simulation (skeleton, robotic arm);Dynamic Simulation of Deformable Objects: elastic objects; Fluid simulation and Particle Systems (fire, smoke, clouds and liquids).

28 Sept. 2007

23

CERMA 2007

* Deformable Objects and Fluids

- Finite Elements Methods

- Spring-Mass Systems

- CFD (Computational Fluid Dynamics)

- Level Set Methods


28 Sept. 2007

24

CERMA 2007


Simulation of Articulated Rigid Body Dynamics

Open Source Library (C/C++ API)

Used with OSG, Ogre3D, CrystalSpace, ...


Slider jointSlider jointHinge jointHinge jointBall and socket jointBall and socket joint

Universal jointUniversal joint

28 Sept. 2007

25

CERMA 2007


Simulation of Articulated Rigid Body Dynamics


Physics Simulation:

- Gravity, friction, acceleration, deceleration; - Generation of motion: applying forces and torques (motors);- Collision avoidance and treatment (reaction, object bounce);- Kinematics models and rigid body dynamics simulation; - Different types of joints with actuators (motors)

Webbots uses ODE [Cyberbotics]

28 Sept. 2007

26

CERMA 2007



28 Sept. 2007

27

CERMA 2007



28 Sept. 2007

28

CERMA 2007



WebotsCyberbotics

28 Sept. 2007

29

CERMA 2007



Juice[Nate W.]

28 Sept. 2007

30

CERMA 2007


Simulation Tools:


* OpenSteer

* PhysX AGEIA


VR Simulation: Some important questions...REAL TIME SIMULATIONREAL TIME SIMULATION

28 Sept. 2007

31

CERMA 2007

Intelligent Agents:

Agents: Perception, ActionAgent Behaviours

Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning


Comportamento

28 Sept. 2007

32

CERMA 2007

Intelligent Agents:


Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge Integration


Reactive-Deliberative Architecture

Control Architectures: Reactive, Deliberative, Hierarchical, Hybrid

Hybrid Architecture

Architecture BDI (Beliefs-Desires-Intentions)

F. Osório et al.[Virtual Concept 2005]

28 Sept. 2007

33

CERMA 2007

Intelligent Agents:


Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning


Robotic

Teams, Squads, Swarms

Artificial Intelligence Tools

28 Sept. 2007

34

CERMA 2007

Presentation Topics

Agenda:











6. Conclusions and New Trends

28 Sept. 2007

35

CERMA 2007

5. Applications: VR Simulation Tools

Applications @ Unisinos

1. Autonomous Robots in VR Environments

SimRob3D - Mobile Robots Simulator

SEVA 3D - Autonomous Vehicle Parking

LEGGEN - Legged (articulated) Robots Simulator

Robombeiros - Multi-Robots Fire Fighting

28 Sept. 2007

36

CERMA 2007


SimRob3D

- Our Simulation Tools: SimRob2D (Khepera)SimRob3D Seva2D

Autonomous Robots in VR Environments

28 Sept. 2007

37

CERMA 2007


SimRob3D Simulator

> Sensors: Infrared, Sonar, Bumper

> Actuators / Kinematics: Differential, Ackerman

> Realistic Simulation Model: 3D World + noise / error (imprecise sensors and actuators)


28 Sept. 2007

38

CERMA 2007


SimRob3D

Simulator


28 Sept. 2007

39

CERMA 2007


SEVA 3D - "Sistema de Estacionamento de Veículos Autônomos"


- Baja Buggy remotely controlled by a cell phoneC. Kelber - UNISINOS, Brazil

Published at:IEEE WCCIIJCNN 2006


28 Sept. 2007

40

CERMA 2007


SEVA 3D

28 Sept. 2007

41

CERMA 2007

SEVA 3D Simulator

> Vehicle Simulation xx Vehicle Control

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization


5. Applications: SEVA 3D

28 Sept. 2007

42

CERMA 2007

SEVA: FSA - Finite State Automaton

Inputs:

- Sonar Sensors: Stochastic ray-casting / 3D cone)

Outputs:

- Steering Wheel Angle- Gas pedal (car speed + direction: fwd, back)

States:



28 Sept. 2007

43

CERMA 2007

SEVA: NEURAL FSA - Learning the FSA...

Si(t) + Si(t-1) + ...Temporal Window

Adapted Jordan-Net using RProp Learning


28 Sept. 2007

44

CERMA 2007

SEVA3D - Autonomous Vehicle Parking Simulator

3D

SensorsActuatorsKinematicsFSA CtrlANN Ctrl


28 Sept. 2007

45

CERMA 2007







Robombeiros - Multi-Robots Fire Fighting

28 Sept. 2007

46

CERMA 2007


Legged Robots Evolution and Walking Control



[EPFL]

28 Sept. 2007

47

CERMA 2007


Legged Robots Evolution and Walking Control

Sources of Inspiration: LEGGEN - Published at: IEEE WCCI CEC 2006SBIA 2006

33


28 Sept. 2007

48

CERMA 2007

28 Sept. 2007

49

CERMA 2007


LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments

Simulation of Robots: 3D Realistic Virtual Environments- Sensors: infrared, sonar, bumpers, gyro (accelerometers),

GPS, compass, light and vision sensors, etc. - Actuators: legs and arms with angular motors (joints) - Physics: collision, kinematics, rigid body dynamics

Simulation of Legged Autonomous Robots:- Robot ControlArchitectures Implementation

Genetic Evolved Control of Articulated Robots (w/legs)

28 Sept. 2007

50

CERMA 2007



Simulation of 3D Realistic Virtual Legged Robots LEGGEN Simulator - Tools:

1. OSG- Open Scene Graph (OpenGL + Extensions)[ http://www.openscenegraph.org/ ]

2. ODE - Open Dynamics EngineRigid Body Physics Simulation(gravity, inertia, friction, collision, joints, etc)[ http://www.ode.org/ ]

3. GALib - Genetic Algorithms Simulation[ http://www.lancet.mit.edu/ga/ ]

4. Robot Control FSM: Finite State Machine = Sense + Act

28 Sept. 2007

51

CERMA 2007



Simulation main goals:

- Evaluate different Robot Models(hardware configurations)IEEE WCCI / CEC 2006 - Vancouver, Canadá

- Evaluate different Fitness FunctionsIBERAMIA / SBIA - Ribeirão Preto, SP Robot Models

Boston Dynamics

Evaluate different robot models in order to select a better hardware configuration

28 Sept. 2007

52

CERMA 2007

LEGGEN SIMULATOR

Simulation Results:

28 Sept. 2007

53

CERMA 2007

LEGGEN SIMULATOR

Simulation RESULTS:

Tetrapod Video - Distance, Gyro

28 Sept. 2007

54

CERMA 2007

LEGGEN SIMULATOR

Simulation RESULTS: Tetrapod Video - "bloopers"

28 Sept. 2007

55

CERMA 2007







Robombeiros - Fire Fighting

28 Sept. 2007

56

CERMA 2007

Robombeiros - Fire Fighting VR Simulation

Virtual Simulation Environment:* 2D and 3D Simulation* Simulation of fire propagation* Autonomous fire-fighting team* Define: Strategy, Mission, Execution

Fire PropagationSimulation:

- Direction and Speedof wind

- Vegetation type andcoverture density

(speed of propagation)- Terrain

Figure: 2D Simulation using SDL library => http://pessin.googlepages.com/robombeiros

Published at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]

28 Sept. 2007

57

CERMA 2007


Virtual Simulation Environment:

http://pessin.googlepages.com/robombeirosPublished at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]

3D Visualization:- Vegetation, Fire- Autonomous mobile Robots- Stereo 3D - Tools: OSG, ODE, Demeter

28 Sept. 2007

58

CERMA 2007


Virtual Simulation Environment:

3D Simulation:- Fire propagation- Physics- Robot Control

http://pessin.googlepages.com/robombeiros

28 Sept. 2007

59

CERMA 2007

Presentation Topics

Agenda:











6. Conclusions and New Trends

28 Sept. 2007

60

CERMA 2007

New Trends

Claytronics - Nanotechhttp://www.cs.cmu.edu/~claytronics/[Published at IROS2006]

IEEE Intelligent Robot and Systems Conference

28 Sept. 2007

61

CERMA 2007

Conclusions and New Trends

Virtual Reality Environments:

Geometric + Kinematic + Physical + Behavioural + Cognitive=

Realistic VR Environments

New Trends: VR + PhysicsArtificial IntelligenceAR - Augmented RealityHaptic Interfaces

Autonomy IVREIntelligent Virtual Reality Environments

Simulation VR PBSimVR Physical and Behavioral Simulation

More Real VR++

28 Sept. 2007

62

CERMA 2007

CONTACT INFORMATION

UNISINOS University UNISINOS University -- BrazilBrazil

Applied Computing Research PostApplied Computing Research Post --grad Programgrad Program -- PIPCAPIPCAAutonomous Vehicles Research GroupAutonomous Vehicles Research Group -- GPVAGPVA

WebWeb: : GoogleGoogle -- veiculosveiculos autonomosautonomos

GPVA GPVA WebWeb PagePage: : http://www.eletrica.unisinos.br/~autonomhttp://www.eletrica.unisinos.br/~autonom

ContactContact -- WebWeb PagePage: : http://inf.unisinos.br/~osorio/http://inf.unisinos.br/~osorio/

This conference This conference -- Web Page: Web Page: http://inf.unisinos.br/~osorio/palestras/cerma07.ht mlhttp://inf.unisinos.br/~osorio/palestras/cerma07.ht ml

Contact: Prof. Dr. Fernando OsórioE-Mail: [email protected]

autonomous robots: design and testing using virtual

Documents