applying a predictive display in a mixed reality interface for a ...what is the problem? in a...
TRANSCRIPT
What is the problem? In a traditional tele-robotic system, extended and variable time delay decreases the system performance
What is Our Research About? In our research, we try to minimise the effect of the time delay for tele-robot system by utilising a mixed reality interface with predictive display (Figure 1).
We apply predictive display in a mixed reality environment for tele-robotics to improve the manoeuvrability and situational awareness. A virtual 3D arm robot model position is synchronised with video overlay to show a predictive display from both a video and virtual scene. We believe, this approach can help the operator to improve their performance and minimise the effect of the time delay.
What is Mixed Reality? Mixed Reality Environment is the combination of visual information between the video views and 3D computer generated virtual objects and text.
Video view information:
1. Shows the dynamic information of the tele-robot worksite environment.
2. Increases the operator’s confidence in performing the tasks.
Virtual information:
1. Provides extra information to the operator.
2. Shows the condition of tele-robot, including its position and orientation.
3. Can be used as a predictive display.
4. Produces viewpoint freedom.
What is predictive display? A predictive display shows the modelled response of the system to the operator commands prior to their application rather than waiting on feedback from the remote site.
Applying a Predictive Display in a Mixed Reality Interface for a Tele-Manipulator
How is the predictive display different from the traditional system?
How do we approach the predictive display system in our research? Predictive Display with Inverse Kinematics Inverse Kinematics are applied to generate position and rotation of each 3D link in the model. It works by calculating the required motions (position and rotation of joint angles) to achieve a desire position.
Example: In figure 4, Inverse Kinematics are used to calculate the required joint angles (γ, θ4, α, β and θ5) of each link when a target position is defined.
Predictive Display with Coordinate Transform Approach Coordinate Transformation is applied to transform the three dimensional coordinate system (World coordinates – 3D) to a two dimensional coordinate system (Camera coordinates – 2D).
Figure 1: A Mixed Reality Interface of Tele-robotics system with predictive display as a visual feedback concept
By Ida Bagus Kerthyayana Manuaba*,**, ANU CECS iHcc Group, [email protected]. Supervisor Panel: Dr. Ken Taylor **, Prof. Tom Gedeon *, Assoc. Prof. Henry Gardner *.
Figure 5. Coordinate transformation
Figure 4. Example of different angels and length used for the inverse kinematical problem
Figure 3. The predictive display is used as visual feedback of tele-manipulator actions within a Mixed Reality framework.
Figure 2. Traditional system
* Research School of Computer Science, College of Engineering & Computer Science, Australian National University.
** Commonwealth Science and Industrial Research Organisation (CSIRO) ICT Center, Canberra, Australia.
Predictive Display
Virtual Information
Real Scene Tele-Manipulator
Visual Feedback
Defined target position
Camera Coordinate 2D Position in Image Plane
Position in 3D world
World Coordinate