Long-distance Long-distance Teleoperation systemTeleoperation system

using Internetusing Internet

JUSTSAP-PISCES Symposium 2008JUSTSAP-PISCES Symposium 2008

November 12, 2008November 12, 2008

2

Latency Cancel TechnologyLatency Cancel Technology

You can develop You can develop advancedadvanced remote control remote control system system easilyeasily and at a and at a low costlow cost..

TTeleelePPresence over resence over IPIP TTeleelePPresence over resence over IPIP

Platform for the remote control Platform for the remote control system system which requires the digital which requires the digital video video transmitted by wireless transmitted by wireless

LAN and internet.LAN and internet.

3

Advanced remote control system Operation from everywhere of Internet available places

High operativity using Motion JPEG video

Easily and low cost I/F for using COTS equipments (WirelessLAN, Internet, PC, game

controller, etc.)

Software Library for easy system development by users

All-in-one platform for video-based remote control system

Latency Cancel TechnologyLatency Cancel Technology

4

An Introduction of Products

“TPIP Unit” and “TPIP Library”

5

Image processing boardImage processing boardImage processing boardImage processing board

Control boardControl boardControl boardControl board

The feature of TPIP unitThe feature of TPIP unit

Value remote control platformValue remote control platform

All-in-one platform for video-based remote control systemAll-in-one platform for video-based remote control system((Image processing board, control board, software libraryImage processing board, control board, software library))

Various input/output for remote control systemVarious input/output for remote control system

TPIP UnitTPIP UnitTPIP UnitTPIP Unit

Input image : NTSC

Output image : MotionJPEG (VGA/QVGA 30fps)

Sound input/output : Audio

Wireless Communication : IEEE802.11a/b/g

Digital I/O RC servo I/F

DC motor I/F

Pulse input

Analog input CAN I/F

RS232 I/F

6

TPIP Unit TPIP Unit SpecificationsSpecifications

Vout

V1

V2

V3

pw

RS232

SH250MHz

S1

S2

S3

S4

S5

S6

S7

S8

S9

S10

WirelwssLAN(11a,11g)

Servo

NTSCCAMERA

Voltage

Encoder

V in

RS232

pw

Ethernet

WirelessLANCard

DC

6～

16

V SensorAnalog Input

JPEGchip

Motor SH4240MHz

Image processing boardImage processing boardTarget control boardTarget control board

P1

P2

A1

RS232

CAN I/F

RS232

Other unit

Other unit

5V

A3

A2

A4

A5

A7

A6

A8

I1I3I2I4

O1

O3

O2

O4

O５

O７ O

６O

８

SensorDigital INPUT

Switch

Audio

Mikespeaker

Vout NTSCdisplay

Item Specifications / performance

Communication

LANWireless ： IEEE802.11a,g,b (PC card) × １Cable ： 10baseT / 100baseTX × １

Image Image transmit

VGA/QVGA 30fps, delay ： Less than 100ms（ Remove IP network delay time ）

Camera I/F Max ３（ change signal ）

SoundSound

transmitMonaural recording bidirectional transmit（ Sampling Rate:16/8KHz,44.1KHz)

Board size 105×84×60mm

Item Specifications / performance

I/O RC servo I/F Max 10ch

DC motor I/F 1ch

Pulse input Encoder Input 2ch

Analog input 8ch

Digital I/O Input 4ch / output 8ch (TTL)

CAN I/F 1ch

RS232 I/F 2ch

7

Compatibility with the radio control system Compatibility with the radio control system

Image processingboard

R/C receiver

Servo

Servo Motor

amp

Motor

amp

Radio control system for hobby

Servo

Servo Motor

Motor

amp

Target controlboard

TPIP unitTPIP unit

DC MotorDC MotorOutputOutput

RC servoRC servoOutputOutput

11a

CameraCameraInputInput

SensorSensorInputInput

RC partsRC partsusableusable

RC servoRC servoOutputOutput

replaceablereplaceable

Two-wayTwo-wayDigital control commandDigital control commandVideo & Sensor dataVideo & Sensor data

One-wayOne-wayAnalog control Analog control command onlycommand only

DC MotorDC MotorOutputOutput

8

SoundSoundcontrolcontrol

TPIP libraryTPIP library

OS : LinuxOS : Linux

JPEG compression boardJPEG compression board

Motor control boardMotor control board

JPEG device driverJPEG device driver

Real-Time Image transmitReal-Time Image transmitSoftwareSoftware

UDP / IPUDP / IP

S/W

H/W

Internet

TPIP UnitTPIP UnitTPIP UnitTPIP Unit

ControllerControllerSoftwareSoftware

PC/AT compatible machinePC/AT compatible machine

OS : Windows XPOS : Windows XPUDP / IPUDP / IP

JPEG decodeJPEG decodeSoftwareSoftware

Real-TimeReal-TimeImage transmitImage transmit

SoftwareSoftware

High-speedHigh-speedrenderingrendering

User programUser program

DLL DLL (Dynamic Link Library)(Dynamic Link Library)

TPIP LibraryTPIP LibraryTPIP LibraryTPIP Library

9

TPIP libraryTPIP library

USBUSB

DisplayDisplay

WirelessWirelessLANLAN

RS232RS232OtherOther

controllercontroller

Controller inputController inputfunctionfunction

Image control / transmitImage control / transmitfunctionfunction

SIO CommunicationSIO Communicationfunctionfunction

Graphic controlGraphic controlfunctionfunction

UDP functionUDP function

UDP-SIO communicationUDP-SIO communicationfunctionfunction

Base libraryBase library apply libraryapply library

TPIP libraryTPIP library

TPIP UnitTPIP Unit

10

UniversityUniversityTechnicalTechnicaljunior collegejunior college

ResearchResearchlaboratorieslaboratories

SanritzAutomationSanritzAutomation

TechnicalTechnical informationinformation

TPIP Library TPIP Library source codesource code

CommunityCommunity

ProductProductinformationinformation

Research &Research &developmentdevelopment

resultsresults

offer & technical supportoffer & technical support

archivearchive

Research and development(school) Research and development(school) ：： 26 members26 members

Research and development(Enterprise):Research and development(Enterprise):

11 members11 members

RRC Student member : 23 membersRRC Student member : 23 members

Student member : 9 membersStudent member : 9 members

Activities themeActivities theme・・ Robot ControlRobot Control・・ OpenCV Image data OpenCV Image data processingprocessing　　　　　　　　　　　　　　　　　　　　　　 etc…etc…

TPIP User CommunityTPIP User Community

11

And…And…・・ TPIP Platform customizeTPIP Platform customize・・ TPIP Systems construction supportTPIP Systems construction support

Image processing boardImage processing board（（ compression & transmitcompression & transmit ））

\140,000\140,000

TPIP Platform PACKAGETPIP Platform PACKAGE\200,000\200,000

academic price \100,000academic price \100,000

Price in JapanPrice in Japan

Tele-Operation Rover PACKAGETele-Operation Rover PACKAGE\2,500,000\2,500,000

academic price \1,500,000academic price \1,500,000Remote controlRemote control

TargetTarget(Rover)(Rover)

TPIP UnitTPIP Unit

TPIP libraryTPIP library

TPIP UnitTPIP Unit

TPIP libraryTPIP library

Image processing boardImage processing board

TPIP libraryTPIP library

You can choose TPIP PlatformYou can choose TPIP Platform

12

TPIP ApplicationsTPIP Applications

13

TPIP ApplicationsTPIP Applications

TPIP-model car drive simulatorTPIP-model car drive simulator

Jet engine noise measurement roverJet engine noise measurement rover

Telemeter systemTelemeter system

Rescue robotRescue robot

14

TPIP-Car Drive simulator TPIP-Car Drive simulator （（ kids kids ProjectProject ））

Event of automobile operation Event of automobile operation experience for kids experience for kids sponsored bysponsored by TOYOTA MOTOR CORPORATION.TOYOTA MOTOR CORPORATION.

diorama town

TPIP control carcockpit

15

Jet engine Noise measurement RoverJet engine Noise measurement RoverAkita: Noshiro proving ground(August, 2007)Akita: Noshiro proving ground(August, 2007)

A collaboration with JAXA(2006.04 ～ ) Clean Engine Team

Noise measurement by rover

FunctionFunction・・ Driving of the RoverDriving of the Rover・・ camera control camera control ・・ Sound source data collection (JAXA)Sound source data collection (JAXA)

16

Telemeter systemTelemeter system 　　　

Hokkaido Japan: Shikabe airport(October, 2007)Hokkaido Japan: Shikabe airport(October, 2007)

Presented by Japan Aerospace Exploration Agency(JAXA)

datadata

Downlink dataDownlink data・・ VideoVideo・・ Sensor data Sensor data (battery,GPS, etc.)(battery,GPS, etc.)

17

Rescue Robot ContestRescue Robot Contest

No. 8 Rescue Robot ContestNo. 8 Rescue Robot ContestAugust 9-10, 2008

Player: 12 teamsVisitor: 6,027guests

Presented by Rescue Robot Contest executive Rescue Robot Contest executive committeecommittee

FunctionFunction・・ Driving of robotsDriving of robots・・ Robot hands controlRobot hands control・・ Video camera control Video camera control (3 Camera Change) (3 Camera Change) ・・ Sensor data monitorSensor data monitor

18

Theme

Rescue activities in a catastrophic urban disaster

Purpose

Raise the interest in disaster prevention and rescue activities

Training of next-generation researcher and engineer

Rescue Robot ContestRescue Robot Contest

19

PROPOSALPROPOSAL

““Tele-Robotics Contest for Lunar Tele-Robotics Contest for Lunar explorationexploration””

A idea of the educational program A idea of the educational program using PISCESusing PISCES

20

Tele-Robotics Contest for Lunar explorationTele-Robotics Contest for Lunar exploration

MoonLunar Base

2. Teleoperaton from the Earth

1.Teleoperaton on the MoonIs "long-distance teleoperation" possible?

PISCESLunar surface on the Earth

PISCES

Long-distance teleoperaton from user’s laboratories

Robots at PISCES

Yes.

“Latency cancel technology” and “TPIP products”make it possible at low cost.

21

Tele-Robotics Contest for Lunar explorationTele-Robotics Contest for Lunar exploration

Rescue Robot Contest Tele-Robotics Contest

Theme Rescue activities in a catastrophic urban disaster

Exploration on the Moon

Purpose Raise the interest in disaster prevention and rescue activities

Raise the interest in exploration on the Moon

Training of next-generation researcher and engineer

Field 1/6 scale diorama lunar-like terrain of PISCES

R&D Target

Rescue Robot Tele-robotics applications on the moon

Participants

Students in Japan Students in the world

Sponsor Local government, private enterprises

Major Space Agencies in the world