integration of internet and real time control systems in telerobotics
TRANSCRIPT
Introduction
Many devices are today accessible through Internet:
Web cams
Washing machines, coffee pots and other appliances
Robots
The term “control” is referred to the possibility of sending a list of commands to the remote device
Introduction
Note:
The control loop is not closed across the network
The remote equipment executes the commands under the supervision of its own local controller
Why?
The unpredictable performance does not allow the realization of an Internet-based, real-time control system, in which the control loop is closed across the network
Introduction
Question:
If the characteristics of the Internet connection in use are known, is it possible to implement a real-time, closed-loop control system, integrating that connection?
Answer:
JBIT – Java Based Interface for Telerobotics: a telerobotics equipment for real-time teleoperation over Internet, with visual and force feedback
Summary
Internet modeling
Delays, losses and bandwidth
Control of dynamic systems with variable time-delay
Solutions available in literature
Packet loss handling
The JBIT project
Internet modeling
Internet can be considered as a strongly connected network of computers, communicating with each other using packet switched protocols
A
B
DC
To B
To D
To B
To BTo B
Internet modeling
From source to destination, each packet traverses several nodes
Each node has different throughput, routing policy, and buffering and queues management
Each node handles many other data flows, this resulting in random service time
Internet modeling
When the data flow between end users exceeds the available bandwidth, congestion occurs
The available (or marginal) bandwidth for a single user depends on the data flow generated by other users, sharing the same physical connection
Internet modeling
End-to-end connection modeled by:Average delayAvailable throughputDelay statistics/variationsPacket loss statistics
Simple experiments to get the modelICMP packet injection and Round Trip Time (RTT) measurement
Internet modeling
RTT distribution:Depends on the number of nodes traversed:
10000 km connection30 km connection
1 arianna.dei.unipd.it 2 dei-cisc.dei.unipd.it 3 147.162.6.254 4 pd4000.unipd.it 5 cnaf-gw-2.infn.it 6 cnaf-gw1.infn.it 7 131.154.8.1 8 mix-serial3-4.Washington.mci.net 9 core1-fddi-0.Washington.mci.net 10 border1-fddi-0.Bloomington.mci.net 11 border1-fddi-0.Bloomington.mci.net 12 csunet-losnettos.Bloomington.mci.net 13 ISI-SWRL-GW.LN.NET 14 acg-isi.ln.net 15 jpl-acg.ln.net 16 192.138.85.65 17 b198-fddi.jpl.nasa.gov 18 helios.jpl.nasa.gov
1 arianna.dei.unipd.it 2 dei-cisc.dei.unipd.it 3 147.162.6.254 4 pd4000.unipd.it 5 147.162.5.254 6 berica2.gest.unipd.it
17 hop
s5
hops
Internet modeling
RTT phase plots (RTTn+1 vs. RTTn)
Non-congested Congested
rttn+k+1 = rttn+k + P/ -
Note: P is the packet length, is the channel throughput, is the period of the probing packet
Internet modeling
Available bandwidth can be obtained by injecting a data flow that brings the connection close to saturation.
The closer to saturation, the higher the packet loss rate
Host name
Average RTT [msec]
Std. Dev. [msec]
Packet Loss rate
Berica 10 8.44 6.25 5.97 Helios 10 319.0 16.70 51.13 Berica 100 8.10 5.35 0.08 Helios 100 326.3 27.20 41.36
Internet modeling - Summary
Available bandwidth, average delay, delay deviation and loss rate can be easily obtained with simple injection of probing packets
Saturation can be avoided by setting the upper limit of the application throughput below the available BW
Control over Internet
Local system
Variable delay 2
Variable delay 1
Remotesystem
Internet
Closing the loop across Internet
Control over Internet
Few solutions available in literature for time-varying delay
All algorithms are designed on the basis of maximum delay and its max. derivative
Structural constraints:With Internet, short interruptions may occur Decentralized control is preferredIn case of interruptions, it must be allowed to have reduced perfomance
Control over Internet
Other solution: provide a pair of queues to even out the delay jitter
In this case, all constant-delay techniques can be applied (e.g. Smith predictor)
Control over Internet
Which protocol is best suited for Internet-based control?
TCP: Acknowledge mechanism interrupts control loop
RTP: Oriented to audio-video streaming
UDP: No detection of missing packets
Enhanced UDP:
Packet loss detection and time stamping
Control over Internet
Other problem: random packet losses
Losses may be single or burst (more than one packet lost sequentially)
Burst losses occur if the data rate is over the maximum available BW
Data flow must be adapted to operating conditions
Monitoring of available bandwidth through RTT on-line measurement
Control over Internet
Single losses can be handled with a predictor
The prediction is used instead of the missing data
This works in conjunction with the buffering mechanism and a packet loss detection procedure
Based on E-UDP features
Control over Internet - Summary
Not all applications are suitable for IP-based control
The application must “survive” even in case of long interruptions:
Decentralized control must be allowed
Monitoring of connection characteristics and adaptation of the controller to operating conditions
Data recovery must be provided
Case study: Telerobotics
This application is well suited for IP-based control:
Decentralized structureSeveral control algorithms for constant time delay are available
s
Telerobotics at the Industrial Electronics Lab.
Research started in 1995Biorobotics Lab. – UW – SeattleTelerobotics Group – Jet Propulsion LaboratoryLVR - Università di Orleans-Bourges
Design and realization of an Internet-based telerobotic equipment with force feedback
Enanched-UDPJBITP@dus - OTELO
Web-based telerobotic systems
Several systems are available
None allow real time feedback and control
CGI interface and script-like commands
Connection is open when the query is sent and closed upon reply arrival
no continuous video streaming
e-mail or updated environment picture as feedback
JBIT: Java-Based Interface for Telerobotics
Targets:
to enable any Internet user to access a remote robot, with real time video, VR and (possibly) force feedback
low cost (overall!)
no special tools (SW & HW)
JBIT solutions - 1Cost:
Freeware
Low cost camera (Quickcam)
Commercial force display (Microsoft Side Winder Pro) as master
Direct-drive, low cost robot as slave (cannibalized from HDDs)
Sensorless force feedback
JBIT solutions - 2
Real-time visual and force feedback:
Java servlets for fast server response
compressed video streaming (H.263) for video
VR-based interface to improve the visual feedback performance
coordinating force feedback + queuing + data recovery to cope with IP delays and losses
Server (implemented with Java servlets)
Waiting for client requestsAccess control, timeout Data exchange between client and robot (no direct access form client to robot)Video encoding (H.263) and streamingMonitoring of available bandwidthAdaptation of video and control output rate
JBIT: Client-server structure
Client (implements user interface)Communication with servletsSelection of operating modeRadio button and active joystick interfaceVR interfaceVideo decoding and playing (Java Media Framework - JMF)
JBIT: Client-server structure
JBIT - Force feedbackPerception of the remote environment
Depends on the connection delay
Remote tracking of a square object
Conclusions
The realization of an IP-based closed loop control system is not a trivial task.
Precise knowledge of the connection’s characteristics is needed
Such characteristics must be constantly monitored
Reconstruction of lost data should be ensured
Need for some smart strategies to deal with long term interruptions
Few applications are suitable for IP-based control
Several issues still to be addressed in order to achieve a general solution for IP-based control
Conclusions
Some help may arrive from new network technologies:
Hard real-time connections (e.g. Tenet) New version of Internet (IPv6)RSVP (Resource Reservation Protocol)New video compression (MPEG4)
It will be possible to have connections with prescribed Quality of Service (QoS), in terms of throughput, losses and delays
Reliable control of remote equipment using Internet could be implemented more easily.
Conclusions
On-going project: P@dus
COMMUNICATIONCHANNEL
PATIENT STATION (SLAVE)
EXPERT STATION (MASTER)
HAPTIC MASTEREXPERT PC
ECHO PROBE
PCECHOGRAPHSLAVEPATIENT