desenvolvimento de aplicações de robótica inteligente em

28 Sept. 2007

1

Prof. Ph.D. Prof. Ph.D. Fernando Fernando OsórioOsório -- Applied Computing / Applied Computing / UnisinosUnisinosProfaProfa . Ph.D. . Ph.D. SoraiaSoraia MusseMusse -- Computing Science / PUCComputing Science / PUC --RSRSProf. Prof. M.ScM.Sc. . FarleiFarlei HeinenHeinen -- Computing Eng. / Computing Eng. / UnisinosUnisinosM.ScM.Sc. Milton Roberto . Milton Roberto HeinenHeinen -- Ph.D. Student at UFRGSPh.D. Student at UFRGSProf. Prof. Ph.DPh.D Christian Christian KelberKelber -- Electrical Eng. / Electrical Eng. / UnisinosUnisinosGustavo Gustavo PessinPessin -- M.ScM.Sc. at . at UnisinosUnisinos

GRAPHIT - Computer Graphics and Vision Group (Unisinos/PUC-R S)GPVA - Autonomous Vehicles Research Group (Unisinos)GIA - Artificial Intelligence Research Group (Unisinos)RBV - Rede Brasileira de Visualização [FINEP/Brazil]

IEEE / DVP - Distinguished Visitors Program Latin America

Applied Computing Applied Computing PostPost--grad. Programgrad. Program-- PIPCAPIPCA

UNISINOS University UNISINOS University -- BrazilBrazil

Desenvolvimento de Aplicações de Desenvolvimento de Aplicações de Robótica Inteligente em Robótica Inteligente em

Ambientes de Realidade VirtualAmbientes de Realidade Virtual

Research group:

Fernando Osório, Ph.D. / IEEE CS DVP

I JAC - Workshop de I.A. / UNISC 2007 - Palestra Convidada

28 Sept. 2007

2

IEEE / DVP - Distinguished Visitors Program Latin America

Prof. Fernando Santos Osório - IEEE MemberApplied Computing Research Post-Graduation Program - PIPCAUNISINOS University - Brazil (Porto Alegre - Southern Region)IEEE Computer Society DVP Program

Jesuit UniversityWith: - 30.000 Students- 900 Professors- 16 PPGs (post-grad programs)

PPG CAPES PPG CAPES NotaNota 44

28 Sept. 2007

3

Presentation Topics

Agenda:

1. Introduction: VR - Hierarchy of Models

2. VR and Simulation

Geometry, Physics, Behaviour, Knowledge and Cognition

3. Physics Simulation Tools

Opensteer, ODE, PhysX, Deformable/Dynamic

4. Intelligent Behaviour

Agents: Perception, Action, Behaviour

Autonomous Robots and Agents - Control

Multi-Agents Systems - Knowledge

5. Applications: Autonomous Robots VR Simulation Tools

6. Conclusions and New TrendsVídeo Demo Web/Java

28 Sept. 2007

4

Introduction VR - Virtual Reality

[Paul Milgram et al. 95]

From REAL to VIRTUAL3D + Immersion + Interaction

Virtual Reality

28 Sept. 2007

5

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

Virtual RealityVirtual Reality...

* VRML - 3D Worlds (Geometry)

* QTVR - Panorama 3D (Images)

Introduction VR - Virtual Reality

\\VRMLVRML

Virtual Reality

28 Sept. 2007

62424

IRISA / INRIA IRISA / INRIA -- FranceFrancehttp://www.irisa.fr/lagadic/demo/demohttp://www.irisa.fr/lagadic/demo/demo--ar3/demoar3/demo--ar3ar3--eng.htmleng.html

Real World Integrated with Virtual Objects

Augmented Reality

28 Sept. 2007

7

Virtual Reality

Virtual RealityVirtual Reality...

* 3D Virtual Environment

* Interaction => Virtual Reality Devices

* Immersion => Virtual Reality Devices

* Realism => Graphical Realism (photo-realism)

Movements

Interaction Real x Virtual

"Physics Realism"

VISUALIZING 3D & VIRTUAL ENVIRONMENTS

How to do it?

28 Sept. 2007

8

1. Introduction

Sources of Inspiration:

3D Virtual Worlds - Hierarchy of Models

[Funge 1999]

28 Sept. 2007

9

1. Introduction



[Funge 1999]

28 Sept. 2007

10

1. Introduction



[Funge 1999]

28 Sept. 2007

11

1. Introduction



[Funge 1999]

28 Sept. 2007

12

1. Introduction



[Funge 1999]

28 Sept. 2007

13

1. Introduction



[Funge 1999]

PhantomPhantom

HaptionHaption

CyberForceCyberForce

OmegaOmega

28 Sept. 2007

14

1. Introduction



[Funge 1999]

[Ari Chapiro - Dance]

[CromosLab]

Agents

28 Sept. 2007

15

1. Introduction



[Funge 1999]

28 Sept. 2007

16

1. Introduction



[Funge 1999]

28 Sept. 2007

17

1. Introduction



[Funge 1999]

Knowledge

AutonomousBehaviour

28 Sept. 2007

18




[Funge 1999]

Virtual World

xx

Real World

28 Sept. 2007

19

Presentation Topics

Agenda:










5. Autonomous Robots VR Simulation Tools

6. Conclusions and New TrendsVídeo Demo Web/Java

28 Sept. 2007

20



[Funge 1999]

Virtual World

xx

Real World

Increasing Reality in VR Applications: Physical and Behavioral Simulation


28 Sept. 2007

21

1. Introduction

RealisticVR

Increasing Reality in VR Applications: Physical and Behavioral Simulation

Virtual World

xx

Real World

Real WorldReal WorldSimulationSimulation

28 Sept. 2007

22

Simulation Tools:

* ODE - Open Dynamics Engine

* OpenSteer

* PhysX AGEIA

* Deformable Objects and Fluids:- Finite Elements Methods- Spring-Mass Systems- CFD (Computational Fluid Dynamics)- Level Set Methods

VR Simulation: Some important questions...


28 Sept. 2007

23

Simulation Tools:


* OpenSteer

* PhysX AGEIA


VR Simulation: Some important questions...


Physics:

Physical structure: resistance, mass, density, elasticity;Position and orientation in the 3D space; Kinematics and Dynamics;Linear and angular velocities; Motion (w/ forces and torques), trajectories; Acceleration, deceleration;Attraction and repulsion;Gravity, friction, inertia; Kinetic and potential energy; Laws of energy conservation, linear and angular momentum;Collisions and reaction to collisions;Steering models (wheeled cars, aircrafts, projectiles, boats and ships); Articulated Rigid Bodies Simulation (skeleton, robotic arm);Dynamic Simulation of Deformable Objects: elastic objects; Fluid simulation and Particle Systems (fire, smoke, clouds and liquids).

28 Sept. 2007

24

* Deformable Objects and Fluids

- Finite Elements Methods

- Spring-Mass Systems

- CFD (Computational Fluid Dynamics)

- Level Set Methods


28 Sept. 2007

25


Simulation of Articulated Rigid Body Dynamics

Open Source Library (C/C++ API)

Used with OSG, Ogre3D, CrystalSpace, ...


Slider jointSlider jointHinge jointHinge jointBall and socket jointBall and socket joint

Universal jointUniversal joint

28 Sept. 2007

26


Simulation of Articulated Rigid Body Dynamics


Physics Simulation:

- Gravity, friction, acceleration, deceleration; - Generation of motion: applying forces and torques (motors);- Collision avoidance and treatment (reaction, object bounce);- Kinematics models and rigid body dynamics simulation; - Different types of joints with actuators (motors)

Webbots uses ODE [Cyberbotics]

28 Sept. 2007

27



28 Sept. 2007

28



28 Sept. 2007

29



WebotsCyberbotics

28 Sept. 2007

30



Juice[Nate W.]

28 Sept. 2007

31


Simulation Tools:


* OpenSteer

* PhysX AGEIA


VR Simulation: Some important questions...REAL TIME SIMULATIONREAL TIME SIMULATION

28 Sept. 2007

32

Intelligent Agents:

Agents: Perception, ActionAgent Behaviours

Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning


Comportamento

28 Sept. 2007

33

Intelligent Agents:


Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge Integration


Reactive-Deliberative Architecture

Control Architectures: Reactive, Deliberative, Hierarchical, Hybrid

Hybrid Architecture

Architecture BDI (Beliefs-Desires-Intentions)

F. Osório et al.[Virtual Concept 2005]

28 Sept. 2007

34


RobotSensors Actuators

PositioningEstimator

(Monte Carlo)

Environment Representation Maps

Polygonal

Grid

Topologicaland

Semantic

SharedMemory

Control Layers

Path Planning

Behaviors

Sequencer

Control Modules

Robust Hybrid Control COHBRA / HyCAR

SimRob3DUnisinos

COHBRACOHBRA: : Controle Híbrido Controle Híbrido de Robôs Autônomosde Robôs Autônomos

HyCARHyCAR -- Hybrid Control Hybrid Control for Autonomous Robotsfor Autonomous Robots

Published at HIS2002Hybrid Intelligent Systems 2002, Chile

28 Sept. 2007

35

Intelligent Agents:


Control ArchitecturesAutonomous Agents Multi-Agents SystemsKnowledge / Reasoning


Robotic

Teams, Squads, Swarms

Artificial Intelligence Tools

28 Sept. 2007

36

Presentation Topics

Agenda:











6. Conclusions and New Trends

28 Sept. 2007

37

5. Applications: VR Simulation Tools

Applications @ Unisinos

1. Autonomous Robots in VR Environments

SimRob3D - Mobile Robots Simulator

SEVA 3D - Autonomous Vehicle Parking

LEGGEN - Legged (articulated) Robots Simulator

Robombeiros - Multi-Robots Fire Fighting

28 Sept. 2007

38


SimRob3D

- Our Simulation Tools: SimRob2D (Khepera)SimRob3D Seva2D

Autonomous Robots in VR Environments

28 Sept. 2007

39


SimRob3D Simulator

> Sensors: Infrared, Sonar, Bumper

> Actuators / Kinematics: Differential, Ackerman

> Realistic Simulation Model: 3D World + noise / error (imprecise sensors and actuators)


28 Sept. 2007

40


SimRob3D

Simulator


28 Sept. 2007

41


SEVA 3D - "Sistema de Estacionamento de Veículos Autônomos"


- Baja Buggy remotely controlled by a cell phoneC. Kelber - UNISINOS, Brazil

Published at:IEEE WCCIIJCNN 2006


28 Sept. 2007

42


SEVA 3D

28 Sept. 2007

43

SEVA 3D Simulator

> Vehicle Simulation xx Vehicle Control

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization

SimRob3D SEVA3D

Commands

Sensors

Control:SEVA3D-A (FSA)SEVA3D-N (Neural)

ControlSimulation

MotorActions

SensorialInformation

Kinematics:Estimation ofPosition and Orientation

Perception:Sensor Simulation

3DWorld

RobotModel

Visualization


5. Applications: SEVA 3D

28 Sept. 2007

44

SEVA: FSA - Finite State Automaton

Inputs:

- Sonar Sensors: Stochastic ray-casting / 3D cone)

Outputs:

- Steering Wheel Angle- Gas pedal (car speed + direction: fwd, back)

States:



28 Sept. 2007

45

SEVA: NEURAL FSA - Learning the FSA...

Si(t) + Si(t-1) + ...Temporal Window

Adapted Jordan-Net using RProp Learning


28 Sept. 2007

46

SEVA3D - Autonomous Vehicle Parking Simulator

3D

SensorsActuatorsKinematicsFSA CtrlANN Ctrl


28 Sept. 2007

47







Robombeiros - Multi-Robots Fire Fighting

28 Sept. 2007

48


Legged Robots Evolution and Walking Control



[EPFL]

28 Sept. 2007

49


Legged Robots Evolution and Walking Control

Sources of Inspiration: LEGGEN - Published at: IEEE WCCI CEC 2006SBIA 2006

33


28 Sept. 2007

50

28 Sept. 2007

51


LEGGEN - Legged Robots Evolution and Walking ControlAutonomous Robots in VR Environments

Simulation of Robots: 3D Realistic Virtual Environments- Sensors: infrared, sonar, bumpers, gyro (accelerometers),

GPS, compass, light and vision sensors, etc. - Actuators: legs and arms with angular motors (joints) - Physics: collision, kinematics, rigid body dynamics

Simulation of Legged Autonomous Robots:- Robot ControlArchitectures Implementation

Genetic Evolved Control of Articulated Robots (w/legs)

28 Sept. 2007

52



Simulation of 3D Realistic Virtual Legged Robots LEGGEN Simulator - Tools:

1. OSG- Open Scene Graph (OpenGL + Extensions)[ http://www.openscenegraph.org/ ]

2. ODE - Open Dynamics EngineRigid Body Physics Simulation(gravity, inertia, friction, collision, joints, etc)[ http://www.ode.org/ ]

3. GALib - Genetic Algorithms Simulation[ http://www.lancet.mit.edu/ga/ ]

4. Robot Control FSM: Finite State Machine = Sense + Act

28 Sept. 2007

53



Simulation main goals:

- Evaluate different Robot Models(hardware configurations)IEEE WCCI / CEC 2006 - Vancouver, Canadá

- Evaluate different Fitness FunctionsIBERAMIA / SBIA - Ribeirão Preto, SP Robot Models

Boston Dynamics

Evaluate different robot models in order to select a better hardware configuration

28 Sept. 2007

54

LEGGEN SIMULATOR

Simulation Results:

28 Sept. 2007

55

LEGGEN SIMULATOR

Simulation RESULTS:

Tetrapod Video - Distance, Gyro

28 Sept. 2007

56

LEGGEN SIMULATOR

Simulation RESULTS: Tetrapod Video - "bloopers"

28 Sept. 2007

57







Robombeiros - Fire Fighting

28 Sept. 2007

58

Robombeiros - Fire Fighting VR Simulation

Virtual Simulation Environment:* 2D and 3D Simulation* Simulation of fire propagation* Autonomous fire-fighting team* Define: Strategy, Mission, Execution

Fire PropagationSimulation:

- Direction and Speedof wind

- Vegetation type andcoverture density

(speed of propagation)- Terrain

Figure: 2D Simulation using SDL library => http://pessin.googlepages.com/robombeiros

Published at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]

28 Sept. 2007

59


Virtual Simulation Environment:

http://pessin.googlepages.com/robombeirosPublished at SVR 2007 (Symposium on Virtual and Augmented Reality) [G. Pessin, F. Osório, S. Musse, V. Nonnenmacher, S. Ferreira]

3D Visualization:- Vegetation, Fire- Autonomous mobile Robots- Stereo 3D - Tools: OSG, ODE, Demeter

28 Sept. 2007

60


Virtual Simulation Environment:

3D Simulation:- Fire propagation- Physics- Robot Control

http://pessin.googlepages.com/robombeiros

28 Sept. 2007

61

Presentation Topics

Agenda:











6. Conclusions and New Trends

28 Sept. 2007

62

New Trends

Claytronics - Nanotechhttp://www.cs.cmu.edu/~claytronics/[Published at IROS2006]

IEEE Intelligent Robot and Systems Conference

28 Sept. 2007

63

Conclusions and New Trends

Virtual Reality Environments:

Geometric + Kinematic + Physical + Behavioural + Cognitive=

Realistic VR Environments

New Trends: VR + PhysicsArtificial IntelligenceAR - Augmented RealityHaptic Interfaces

Autonomy IVREIntelligent Virtual Reality Environments

Simulation VR PBSimVR Physical and Behavioral Simulation

More Real VR++

28 Sept. 2007

64

CONTACT INFORMATION

UNISINOS University UNISINOS University -- BrazilBrazil

Applied Computing Research PostApplied Computing Research Post --grad Programgrad Program -- PIPCAPIPCAAutonomous Vehicles Research GroupAutonomous Vehicles Research Group -- GPVAGPVA

WebWeb: : GoogleGoogle -- veiculosveiculos autonomosautonomos

GPVA GPVA WebWeb PagePage: : http://www.eletrica.unisinos.br/~autonomhttp://www.eletrica.unisinos.br/~autonom

ContactContact -- WebWeb PagePage: : http://inf.unisinos.br/~osorio/http://inf.unisinos.br/~osorio/

This conference This conference -- Web Page: Web Page: http://inf.unisinos.br/~osorio/palestras/cerma07.ht mlhttp://inf.unisinos.br/~osorio/palestras/cerma07.ht ml

Contact: Prof. Dr. Fernando OsórioE-Mail: [email protected]

desenvolvimento de aplicações de robótica inteligente em

Documents