Employing Augmented Reality Technologies in Digital Work

Instructions

Dr. Eliot Winer

Associate Director, Virtual Reality Applications Center

Associate Professor, Department of Mechanical Engineering

Faculty Affiliate, Human Computer Interaction

Iowa State University

Use of AR in Manufacturing

• Mapping from digital work instructions to the actual parts is time and cost consuming

“.... 45 percent of every technicians' shift is actually spent on finding and reading procedural and related information (Ott, 1995)”Ott, J. (1995): Maintenance executives seek greater efficiency. Aviation Week and Space Technology, 142 (20), pp. 43-44.

Cutaway view Procedural View Exploded View

Virtual Reality Continuum

AR-supported service 3D model

Augmented Reality

• Augmented Reality (AR) technology enhances the real-world seen by a user with computer-generated information (e.g. 3D models, annotation, and texts).

• AR typically presents this information in a context-sensitive way

– Integrated into the environment by geometrical and photometrical registration

– Manipulable in real-time.

A Functioning AR System

Capture video stream of the environment

Augmented Reality Software

Video Image 3D Model

+ video

Rendering

AR Displays • Video See Through (VST) Advantages:

•Easy Augmentation •No accommodation necessary.

Disadvantages: •Limited Field of View •Limited Resolution

AR Displays • Optical See Through (OST) Advantages:

•Real Environment •Weight •„Unlimited“ Field of View

Disadvantages: •Limited Resolution •Focus problems •Transparent virtual objects

Artificial marker

Object Tracking

• Augmented Reality relies on object tracking:

– Marker-based tracking

– Natural feature tracking

– Rigid object tracking

Natural “marker”

Artificial marker

Object Tracking

• Augmented Reality relies on object tracking:

– Marker-based tracking

– Natural feature tracking

– Rigid object tracking

Natural “marker”

Object Tracking

• Typically a photo of the object to track in input – Find and store unambiguous points

(feature)

– Identify these points in a video stream Photo of the object to track

Information Presentation

•Must provide detailed instructions and next part location – Attention vs. explanation

Photorealistic 3D model of the part to assemble

Red object: abstract 3D model indicates the next part

Blue object: 3D models of tools are used to explain the assembly operation.

Abstract 3D models indicate the next assembly location.

Information Presentation

• Method of information presentation depends on:

– Skill level of the operator, complexity of task, and variety of products and tools

Interaction Design Requirements

• Always provide clear view of part/assembly

– To much information overlay can easily overwhelm

• Must present next step in an intuitive manner

– Are there sub-steps?

• How does user/supervisor know where they are?

– Tracking time, efficiency, and mistakes

• Challenge: Account for user experience level

Prototype Design Current task

Red object: 3D model; further information can be shown if selected

Button icons to step through the assembly sequence

Potential Applications - Concept Design • Assess different design solutions and variants before

prototypes have been built

Virtual front section

Foam model

Potential Applications - Detailed Design • Examine specific parts, analyses, and systems

Physical vehicle body

Physical engine

Blue 3D-model: virtual wires

Red 3D-model: virtual tube system

Potential Applications - Assembly Assist

• Existing documentation could be enhanced

Potential Application - Quality Control

Mobile Device

The screen shows the object under evaluation. A list indicates critical points

Don’t forget doors wheels pressure

Text

AR Assembly Training Station

Video camera

View on monitor

Annotations give additional details

A animated 3D model shows the assembly location and how the part becomes assembled.

Prototype

Thank You

Eliot Winer Iowa State University Virtual Reality Applications Center 1620 Howe Hall Ames, Iowa 50011 Phone: 515.989.1750 Email: ewiner@iastate.edu