augmented reality:

Download Augmented  Reality:

Post on 22-Feb-2016




2 download

Embed Size (px)


Augmented Reality: Intuitive User Interfaces in Maritime Navigation. Prof. Dr.-Ing. Reinhard Koch 1 Kristine Haase 1,2 In cooperation with Prof. Dr.-Ing. Uwe von Lukas 2 1 Christian-Albrechts-Universitt zu Kiel 2 Fraunhofer IGD, Rostock - PowerPoint PPT Presentation


Folie 1

Augmented Reality: Intuitive User Interfaces in Maritime NavigationCESAMAN DieselFraunhofer IGDProf. Dr.-Ing. Reinhard Koch1Kristine Haase1,2

In cooperation withProf. Dr.-Ing. Uwe von Lukas2

1Christian-Albrechts-Universitt zu Kiel2Fraunhofer IGD, Rostock

supported by SevenCs GmbH, Hamburg, and by I-SH Innovationsstiftung Schleswig-Holstein, Germany

1Definition of Augmented Reality (AR)

CESASource: Milgram, 1994 AR: Mixing of virtual content into the real environmentReality

Fraunhofer IGDVirtual Reality (VR)

MAN Diesel (AR: Metaio)Augmented Reality2Augmented and Mixed RealityAR mixing of virtual and real objects for film post production: include lighting, shadows, occlusion and physical interaction

Source: CAU-MIP3

AR Design glasses, Project Artesas. Images: Zeiss

AR DevicesAR glasses: see-through projectors, projected overlayTracking of head-position and view directionOptical mixing of real scene and virtual contentVirtual content enhances (augments) reality

Image: Project ArvikaExample: Maintenance support

Film: Metaio.comExample: Maintenance supportUse and examples of AR ApplicationsUse-AR: AR in maritime environment (IGD Rostock)Case study: AR-Binocular in maritime NavigationPotential of AR in maritime NavigationOverview7

Time to market of AR-Technologie (2008)8AutomotiveAerospaceConstructionMaintenanceMedicineMarketingTourism/EntertainmentDefence/SecurityMaritime Industry

Potential of AR for business areas9AutomotiveHead-up DisplaysDriver safety warningsLooking through car bodyAerospaceErgonomic Cockpit designRadar target overlay

Ergonomic design and safety issues10Mobile AR for guidance and intuitive navigationAR Games on mobile phones and I-PadSuperposition of AR content (historical sites, TV and Film production)


Construction and MaintenanceInformation overlayelectronic online manualx-Ray VisionTutoring and support12Example: Functional overlay

Fraunhofer IGDAR Maintenance on I-Pad Visual overlay of functionality and measured flow data13Example: X-ray vision of pipes in ship cabin

Source: LrssenUseAR study: Position of pipes behind walls, extracted with AR14

Planning, adapting of industrial processesIndustrial plants and productions sitesTesting production processes with virtual prototypesRetrofit of new parts to existing sites15Example: Fitting new parts on-site

Source: MAN Diesel SE, MetaioUse-AR Study: Visual superpositioning of novel diesel engine into existing structures. Visual markers register the virtual prototypes to the real world16Fraunhofer AR-Study for the maritime IndustryUse-AR (2009): Study to apply AR in maritime Industry, Fraunhofer IGD in Rostock Content (in German)- Technology- Case studies- Evaluation & RecommendationsLiteratureDownload:

17Augmented Reality System for maritime navigation

Case Study: AR-Binocular

18Navigation problem: Map adjustmentSource: [Porathe, Shepard]: 2D/3D Reprsentation of navigation maps

2D North-Up2D Head-Up 3D Egocentric 19Study (Porathe, Shepard) reveals: Comparison of map adjustment between navigation officer and Layman:Transformation (Rotation) needs time proportional to rotation angle, can not be trained, navigation officers and Layman indistinguishableNavigation officers value the intuitive use of 3D egocentric maps

Concept AR-Binocular: Direct 3D overlay of map in view with AR (3D Egocentric view)Navigation problem: Map adjustment20

Traditional: 2D Top-ViewNautic chart top view

New: 3D Egocentric view

3dSzene bildSeekarte 3DAugment 3D view with nautical chart data

Setup of AR Binocular: transform to 3D Perspective

View from Bridge


Prepare Chart dataAISReadout Sensor dataINSKameraGPSMarkerTrackingObjectTrackingData flow of AR-Binocular

Point AR-Binocular

Augment videoAdjust egocentric view, superimpose chart data to video22Chart data base: ECDIS maps Electronic Navigation: ENC International Hydrographic Organization (IHO), Special Publication, Transfer Standard S-57 ENC/PS Product Specification for ECDIS (Electronic Chart Display and Information System directEnc/SENC (System Electronical Navigational Chart) ECDIS software SevenCs GmbH


Prepare chart data: extract layers and objectsENC charts Knowledge base/Object extraction 3D Scene/Visualization View camera: 7-fold magnification, 640x480 Pixel Angle of view: 7x10 (1/60 Resolution per Pixel)Binocular display for viewing, 2 x 640x480 PixelPosition (3 DoF) GPS ship navigation systemArtificial Horizon (Gravitometer, Gyroscope)Visual heading sensor relative to ship CompassSensor data for 3D view estimation

Mobile system: 6 DoF ~ 6m, pitch ~45 , roll ~ 45 ,heading 360 5. Heading-Camera1. View camera4. Gravitometer, Gyroscope2. Display25

Image processing for heading angle estimationHeading-Sensor: Heading camera (Fish-eye camera) InfraRed-Marker on ceiling (rotation estimation)Object tracking in view camera (image processing) Visual Heading-Sensor

Fisheye heading camerashowing IR markersHeading angle26Example-Scene using AR-Binocular

Example-Scene using AR-Binocular

AR on Tablet PCs

Potential of AR for maritime Navigation+ AR holds potential for use in maritime navigation+ Map superposition in camera view feasible, head-up projection displays on bridge is an option- Systematic content production (authoring) still missing- Sensors for mobile AR expensive and still in progress+ Integration of AR on Smartphones/I-Pads in the future30Thank you very much

In cooperation with Prof. Dr.-Ing. Uwe Freiherr von LukasMaritime GraphicsFraunhofer IGD, Rostock