advanced optical inspection with inline computational

Advanced Optical Inspection WithInline Computational Imaging (ICI)Ernst Bodenstorfer et [email protected]

AIT Austrian Institute of Technology GmbHCenter for Vision, Automation & ControlVienna, Austria

www.ait.ac.at/hpv

Overview - Advanced Optical Inspection with ICI

Who is the AIT?

Motivation

Practical Examples Rail inspection

Metal band inspection

What is Inline Computational Imaging (ICI)?

Technologies powering ICI Fast multi-line-scan sensor xposure

Fast strobed LED illumination

Results

Conclusion

2

© 2019 AIT Austrian Institute of Technology GmbH. All rights reserved.

Nuclear Engineering Seibersdorf

GmbH

SeibersdorfLabor GmbH

AIT Austrian Institute of Technology


Energy Health &Bioresources

Digital Safety & Security

Vision, Automation &

Control


Mobility Systems Low-EmissionTransport

Technology Experience

Innovation Systems & Policy

40% UF

AF 30%

KF 30%

1300+ employees

Budget: 140 Mio €

3


Vision, Automation & Control

High-Performance Vision

3D Vision and Modeling

Complex Dynamical Systems

4

RESEARCH TOPICS High speed imaging Scalable embedded Vision Computational Imaging Deep Learning

In-line based real-time

fast image processingIndustrial applications


Motivation – In our projects we observe …

Inspection from a single viewing point and with single lighting direction is not sufficient

Surface inspection needs to access more information from the scanned surface to perform

Robust material classification Robust failure detection Inspection of glossy material

We need multi-view imaging and multi-directional lighting !

5


Overview

Who is the AIT?

Motivation



What is Inline Computational Imaging (ICI) ?



Results

Conclusion

6


Example #1: Rail Inspection

7


Vienna subway network Rail car with the AIT inspection system

Example #1: Rail Inspection – Defect Types

8


Top view of a rail head (200 µm resolution)

Mic

ro-

crac

ks

Rail crack

Grin

ding

Example #1: Rail Inspection - Multi-directional Lighting

9


Inspection system on rail carWorking principle

R B

Inspected rail

Colorcamera

Lightsources

Lightsources

Photometric stereo with two lighting directions - color coded (RB)

Example #2: Metal Band Inspection

10


750 m² of metal band

2.3 µm resolution

approx. 1 TB/m²

goal inspection speed10 m²/h ≈ 3 GB/s

Example #2: Metal Band Inspection

11


Working principle Inspection system in the factory

G G

R B

Colorcamera

Objectplane

← Transport direction

Lightsources

Lightsources

Multi-directional lighting - color coded with three lights (RGB)

Overview

Who is the AIT?

Motivation



What is Inline Computational Imaging (ICI)?



Results

Conclusion

12



Generic approach for robust, precise and fast 2D & 3D scanning of surfaces based on Multi-view imaging

Multi-directional lighting

Powerful computation of image information

Suitable for industrial inline inspection

High accuracy and robustness due to algorithms exploiting redundant image information

Multi-view with single camera (self–aligned)

Needs fast multi-line scan camera

Needs fast LED strobing

13


Constantillumination Inspected

object

Transportstage

Multi-line scancamera

Standardnon-telecentricoptics

AIT Inline Computational Imaging – Optical Setup

14


Multiple views with 3D and directional reflectance information

15


Standard Scale Configuration

Sensor Multi-line scan sensor

Working Distance 108 mm

Field of View 46 mm

Depth Range 7.2 mm

Lateral Sampling 20 µm/px

Lateral Optical Resolution 42 µm

Typical Number of Views 13

Depth Resolution 19 µm

AIT Inline Computational Imaging - Technical Data

16


Technologies powering ICI

Fast multi-line-scan image sensor technology Partial readout with zero frame overhead time (speed)

Fast ADC’s (speed)

Light shielding and anti-blooming measures (large irradiance range)

Pipelined processing of exposure, AD conversion, and data output (speed)

High bandwidth data output bus (speed)

Fast strobed LED illumination technology High efficiency power LED‘s

High luminance (cd/mm2)

Short switching times ~100ns

17


xposure Sensor - Fast Multi-line-scan Sensor

18


600 kHz line rate

Pixel matrix 60 x 2016

9 x 9 µm pixel size

Large portion of chip area for low-noise analogue circuit

Column-parallel on-chip ADC‘s

Rigorous light shielding

Continuous reset of unused pixel lines

Flexible row access via 6 wire-control interface

Manufacturer: Fraunhofer IMS

xposure Sensor - Block Diagram

19


CDS amplifiers

S/H

ADC #0 ADC #1

Pixel matrix60 x 2016

com

man

d se

quen

cer

Pixel control

amplifier control

amplifier control

Output bufferbuffer control

6 wireinterface

Data output

...

...

...

AIT xposure:camera – Ultra Fast Multi-line Scan Camera

20

Line frequency up to 600 kHz 40 GigE Vision interface (QSFP with 4x10 Gbit/s Ethernet) Altera Arria 10 SOC FPGA with Dual Core

ARM Cortex A9 MPCoreTM and Linux OS


Flexible switching between several readout modes: 600.000 fps single line 300.000 fps RB color 200.000 fps RGB color 100.000 fps 6 color channels 10.000 fps 60 color channels

World’s fastest line-scan camera

AIT xposure:camera - Specifications

21


xposure:camera – Technical Data

Pixels per Line 2016 Plus 32 dark pixels/line

Pixel Size 9 x 9 µm 100% fill factor

Number of Lines 60 Organized in 20 triples

Vertical Pitch 18 µm

Line-rate Mono (max.) 600 kHz Single line read out

Line-rate RGB (max.) 200 kHz Tri-linear read out

Frame-rate (max.) 10 kHz Read out of 60 lines

ADC‘s 600 kSamples/s On-chip, column-parallel

Sensor Output 16 taps á 10 bit, 80 MHz

Total Image Data Bandwidth 12 Gbit/s

AIT xposure:flash - Fast Strobed LED Illumination

Fast LED-strobing

~600 kHz switching rate

Switch-off < 300 ns

High illuminance ~1Mlx

High homogeneity

22


LED-current (yellow), luminious flux (blue), time base 500ns/div

Fast time-multiplex for multiple lighting directions

LED and camera shutter timing

Non-overlapping light pulses!

23


Output red LED (lm)

Camera Shutter

Output green LED (lm)

Output blue LED (lm)

Output NIR LED (lm)

Control red LED

T_shut-off

1666 ns

camera

LED line 1

LED line 2 LED line 3

LED line 4

moving object

Overview

Who is the AIT?

Motivation






Results

Conclusion

24


Pin Grid Array – Precise 3D Model of Complex Surface

25

Precise 3D model using AIT approach:Fusion of Light Field and Photometric Stereo

3D model using state-of-the-artstereo algorithm

Pin grid array


PCB - Computational Bright Field and Gloss Suppression

26

Computationalbright field

Computationalgloss suppression

PCB with glossy components


Directional Reflectance

27


Light source

Object

Reflectance function

Surface slope

Coin inspection - Fine Surface Structures via Photometry

28

Circulation coin

2D color image

2.5D surface structures


Security Print - Intaglio Print and Optical Variable Devices

29

Color image

Fine surface structuresof intaglio print

OVD / hologram detection

Banknote €10


Conclusions

For surfaces with difficult reflection properties acquisition from a single perspective is not sufficient

There is a need for inspection from multiple viewing and/or multiple lighting angles

Inline Computational Imaging is a generic and scalable technology for systematic acquisition of multiple views and multiple lighting directions

Technologies powering ICI are

Fast multi line scan camera xposure:camera

Fast LED strobing xposure:flash

Practical results for fast inline inspection of 3D surface features demonstrate the broad applicability of the presented technology

30


Thank you!Ernst [email protected]

advanced optical inspection with inline computational

Documents