Introduction to Image Processing and

A l iAnalysis

Gilbert Min Ph DGilbert Min, Ph.D.Applications ScientistNanotechnology Measurements DivisionMaterials Science Solutions Unit

Working with SPM Image Files

Raw data files (binary / ASCII formats)Limited tools for display & analysis

Realtime acquisitionRealtime acquisition

Post processing software68

136

Roundness

ISO 25178

0.2 0.3 0.4 0.5 0.6 0

Height ParametersSq 3.19Ssk -0.945Sku 3.85Sp 4.77

Results for presentation/publication

µm

-10

-8

-6

-4

-2

0

2

4

6

8

0 2 4 6 8 10 12 14 16 18 20 22 24 m m

Elem ent: segm ent o f width : 1 m m , Enclosed area : 0 .0653 m m 2

Agilent PicoImageResults for presentation/publication(.jpg, .tiff, .avi, .xls, etc.)

First Step: Image Leveling Most all SPM images require a basic leveling to remove inevitable artifactsMost all SPM images require a basic leveling to remove inevitable artifacts from image acquisition (sample tilt, scanner bow / nonlinearities, z-drift, line skips, etc.).

nm0 2 4 6 8 10 µm0

nm0 2 4 6 8 10 µm

0

350

400

450

500

550

6000

1

2

3

4 120

140

160

1800

1

2

3

4

100

150

200

250

300

3505

6

7

8

9

40

60

80

1005

6

7

8

9

Original raw image After leveling process

0

50

µm

9

100

20

µm

9

10

Common approaches to leveling:- plane flattening- line by line flattening

Leveling Images: Plane FlattenSimplest approach – a linear plane is subtracted from surfaceSimplest approach a linear plane is subtracted from surface

nm

2527.53032.535

0 1 2 3 4 5 µm0

0.5

1

1.5

57.51012.51517.52022.52

2.5

3

3.5

4

4.5

02.5

µm5

nm

1314

0 1 2 3 4 5 µm0

0 5

LS plane fit

56789101112

0.5

1

1.5

2

2.5

3

01234

µm

3.5

4

4.5

Useful when there is very minimal curvature relative to the surface topography

Plane Flattening: 3-Point Method

Pl i i l d fi d b th d fi d f i t th fPlane is simply defined by three user-defined reference points on the surfaceUseful for step height applications, where a user specific leveling reference is required and where the surface can be leveled to an average.

Line Flattening

Each scan line is fit to a polynomial and the polynomial shape is subtracted.

ZX1st order

The height average of each line is set equal to the previous line to remove any offset

ZX2nd order

ZX

X

3rd order X3rd order

scan linesleveled line

Line Flattening: a Cylindrical Hair Follicle0 2 4 6 8 10 µm

0

1

2

3

4

5

6

7

80th order (raw)

µm

9

10

0 2 4 6 8 10 µm0

1

2

3

4

5

6

7

8

9

1st order

µm

10

0 2 4 6 8 10 µm0

1

2

3

4

5

6

7

8

9

2nd order

µm

10

Using Include/Exclude with Line Flattening0 2 4 6 8 10 µm

0

1

2

3

4

5

6

7

8

9

µm10

1st order 1st order excluding raised stamps

0 2 4 6 8 10 µm0

1

2

3

4

0 2 4 6 8 10 µm0

1

2

3

4

Line by line levelled

5

6

7

8

9

10

5

6

7

8

9

10

Artifacts from line flattening can be avoided by identifying structures to include/exclude in the calculated polynomial used in subtraction

Line by line levelledµm µm

p y

2D / 3D Display OptionsColor Pallette

nm

300

350

400

450

500

550

6000 2 4 6 8 10 µm

0

1

2

3

4

5

nm

300

350

400

450

500

550

600

0 2 4 6 8 10 µm0

1

2

3

4

5

0

50

100

150

200

250

300

µm

5

6

7

8

9

100

50

100

150

200

250

300

µm

6

7

8

9

10

Add Visualization Effects

3D continuous mesh 3D copper material

2D photo simulation

Adding Data Overlay onto 3D SurfacesMore info can be extracted when combining multiple data channels - surface topography with functional imaging (phase, KFM, EFM, MFM, etc.)

0 1 2 3 4 5 µm0

0.5

1

V

0.8

0.9

10 1 2 3 4 5 µm

0

0.5

1

1.5

2

2.5

3

3.5

4 0 2

0.3

0.4

0.5

0.6

0.7

0.8

1.5

2

2.5

3

3.5

4

+ =

µm

4

4.5

0

0.1

0.2

µm

4

4.5

5

surface potentialtopography 3D overlay

Organic materialPZT filmSDRAM Organic material phase overlaid on topography

PZT filmSP overlaid on topography

SDRAMSP overlaid on topography

Filtering: Removing Noise from Images Using a filtering algorithm can remove unwantedUsing a filtering algorithm can remove unwanted noise that often appears in acquired images

Matrix / Spatial FilteringMatrix / Spatial Filtering

Spatial filtering is made by moving a transformation matrix over the surface. Input Ipixels are interpolated/modified according to the weighted values of adjacent pixels to produce filtered image of output O pixels

1 2 1 0 0 0Types of Matrix Filters:

-Smoothing/denoising (median, mean, Gaussian)

-Min/Max

1 2 12 4 21 2 1

0 0 00 1 00 0 0

A “Custom” 3x3?3x3 Gaussian-Edge detection (Laplacian, Sobel, Gradient)

-Many more…including custom user-defined!

A “Custom” 3x3?

No effect:every pixel is

multiplied by 1

3x3 GaussianFilter

Applying Matrix / Spatial Filters0 1 2 3 4 5 µm 0 1 2 3 4 5 µm

0.5

1

1.5

2

0.5

1

1.5

2

median denoising7x7

2.5

3

3.5

4

4.5

2.5

3

3.5

4

4.5

µm5

0 1 2 3 4 5 µm

µm5

0 1 2 3 4 5 µm

0.5

1

median denoising27x27Sobel 7x7

edge detection0 1 2 3 4 5 µm

0.5

1

1.5

2

1

1.5

2

2.5

3

2.5

3

3.5

4

4.5µm

3.5

4

4.5

5

µm5

Fourier filtering

Filtering: Removing Noise from ImagesFourier filtering

Calculates a spectral representation of frequency components (FFT) of an image and user identifies bandwidths for inclusion/exclusion into the filtered surface.

Useful for images with periodic patterns, eg. atomic lattices

Raw data (1o line level)

2D FFT spectrum FFT filtered

Analysis Tools: Profile Extraction / Step Heightnm 1 2 3 4 5nm

100

150

200

250

300

Extracted profile

0 1 2 3 4 5 6 7 8 9 10 11 µm0

50

1 2 3 4 5

Maximum height 168 nm 153 nm 153 nm 150 nm 150 nm

Mean height 157 nm 152 nm 151 nm 148 nm 148 nm

Width 0.415 µm 0.415 µm 0.415 µm 0.401 µm 0.386 µm

1 2 3 4 5nm

300

Total height v-p-v 172 nm 157 nm 155 nm 154 nm 154 nm

Total height v-p 170 nm 157 nm 155 nm 153 nm 154 nm

Minimum height 149 nm 150 nm 150 nm 145 nm 147 nm

50

100

150

200

250

Extracted profi le

0 1 2 3 4 5 6 7 8 9 10 11 µm0

1 2 3 4 5

Maximum height 150 nm 152 nm 162 nm 161 nm 112 nm

Mean height 141 nm 149 nm 158 nm 154 nm 111 nm

Width 0.414 µm 0.429 µm 0.443 µm 0.414 µm 0.343 µm

Total height v-p-v 165 nm 167 nm 170 nm 181 nm 135 nm

Total height v-p 142 nm 167 nm 170 nm 154 nm 133 nmTotal height v-p 142 nm 167 nm 170 nm 154 nm 133 nm

Minimum height 133 nm 145 nm 155 nm 145 nm 109 nm

Measuring Surface Roughness Roughness parameters quantify height statistics of a surfaceRoughness parameters quantify height statistics of a surface

Some commonly reported values

Root Mean Square Standard deviation of the height distributionheight distribution

Arithmetic Mean Mean surface height

1st moment of distribution

Skewness 3rd statistical moment, qualifying the symmetry of

distribution

Kurtosis 4th statistical moment describing flatness of

EUR and ISO Standards exist for 2D & 3D describing flatness of

distribution

Maximum peak height Height between the highest peak and the mean plane

parameters to ensure conformity

Maximum pit height Depth between the mean plane and the deepest valley

Maximum height Heightbetween the highest peak and the deepest valley

Surface Roughness Examplesnm

14

15

16

17

18

19

20

21

nm

1.922.12.22.32.42.52.62.72.82.9

4

5

6

7

8

9

10

11

12

13

0 60.70.80.911.11.21.31.41.51.61.71.8

0

1

2

3

ISO 25178Height Parameters

00.10.20.30.40.50.6

ISO 25178Height Parameters “Smooth” film “Pitted” film Height Parameters

Sq 1.78 nmSsk -3.04Sku 18.4Sp 5.97 nmSv 15.3 nm

Height ParametersSq 0.268 nmSsk 0.0306Sku 3.08Sp 1.01 nmSv 1.92 nm

Sz 21.3 nmSa 1.06 nm

Sz 2.93 nmSa 0.214 nm

Surface Roughness: Same Surface, Different Scan Sizesnm

60

30

35

40

45

50

55

60

ISO 25178Height Parameters

Sq 10.5 nmSsk 0.408

5 um scan

0

5

10

15

20

25

30Sku 2.39Sp 35.7 nmSv 25.4 nmSz 61.1 nmSa 8.91 nm

0

nm

90

100

110

120

30

40

50

60

70

80 ISO 25178Height Parameters

Sq 8.38 nmSsk 0.74Sku 5.89Sp 96.8 nmSv 27.5 nm

0

10

20 Sz 124 nmSa 6.22 nm

Important calculations are made over appropriate length scales, as roughness

25 um scan

p pp p g gvalues depend on sample size

Using the Thresholding Toolf f ff / fAllows user to select surface planes of different altitudes/height levels for

manipulation

µm0 20 40 60 80 100 %

µ

0.119

0.139

0.159

0.178

0.1980 20 40 60 80 100 %

Place along curve corresponds to height level

0.0198

0.0397

0.0595

0.0793

0.0991

00 10 20 30 40 50 60 %

Abbott – Firestone Curve(height histogram & bearing ratio)

Using the Thresholding ToolSt b t t ISO 25178Stamp substrate ISO 25178

Height ParametersSa 2.94 nmSq 3.9 nmSp 14.5 nmSv 14.3 nmSz 28.8 nm

Top surface of stamp bits

ISO 25178Height Parameters

Sa 24.2 nmSq 29.3 nmSp 54.2 nmSv 73.4 nmSz 128 nm

ISO 25178Height Parameters

Sa 2.93 nmSq 4.51 nmSp 33.7 nmSv 6.56 nmSz 40.3 nm

Stamp bits including sidewallsidewall

Example Workflow for Pore Analysis nm

18.8

29.5

40.3

51

61.7

0 20 40 60 80 100 %0 0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25 2.5 2.75 3 3.25 3.5 3.75 µm0

0.25

0.5

0.75

1

1.25

1.5

-34.8

-24.1

-13.4

-2.63

8.09

0 2.5 5 7.5 10 12.5 15 17.5 20 %

1.75

2

2.25

2.5

2.75

3

3.25

3.5 Thresholded -

208

Form factor

µm

3.75

0 1 2 3

1. Choose proper flattening method

2. Use height thresholding tool to select pits of interest

1 2 3 4 5 6 0

104

0 1 2 3 µm0

0.5

1

1.5

2

Mean parameters on 545 grains

Number of grains: 545Total area occupied by the grains: 6.94 µm2 (48.5 %)Density of grains: 38.1 grains / µm2.

Area = 0.0127 µm2 +/- 0.166 µm2Perimeter = 749 nm +/ 9319 nm Mean diameter

µm

2

2.5

3

3.5

Perimeter = 749 nm +/- 9319 nmMean diameter = 68.3 nm +/- 24.8 nmMin diameter = 52.2 nm +/- 24.3 nmMax diameter = 97.1 nm +/- 52 nmForm factor = 1.07 +/- 1.4Aspect ratio = 2.88 +/- 4.31Roundness = 87.7 +/- 1996Orientation = 64.2° +/- 51.8° 70.5

141

Mean diameter

3 Binarization defines pores for 4 Display results

0 1 2 3 µm0

0.5

1

1.5

2

2.5

3

20 40 60 80 100 120 140 nm0

3. Binarization defines pores for 4. Display resultsµm

3.5

Always remember…

When working with images, it’s good practice to:

1) Preserve raw data files before applying operators &1) Preserve raw data files before applying operators & filters

2) Keep a consistent workflow among data sets2) Keep a consistent workflow among data sets, especially when comparing statistical results

3) Try to avoid “over-processing” data and introducing3) Try to avoid over processing data and introducing artificial software image artifacts