An automated image prescreening tool for a printer qualification process

by

†Du-Yong Ng and ‡Jan P. Allebach

† Lexmark International Inc.

‡ School of Electrical and Computer Engineering, Purdue University

Synopsis

Anatomy of a formatter-based EP laser printer

Overview of a printer qualification process

Motivation

Examples of artifacts

Image fidelity metrics – prior work

Prescreening tool

Experiment

Results

Conclusion

Anatomy of a formatter-based EP laser printer

A standalone network device

Print Engine

Formatter

Control Panel

Overview of a printer qualification process

Phase I The formatter (hardware + firmware) and the print engine are

developed in parallel.

The hardware portion of the formatter is not ready.

The firmware of formatter is tested using a simulator .

Test image (PDL)

Newly developedfirmware + simulator

Firmware of an earlierproduct + simulator

Master Current

Digital outputsmatch qualitatively?

Overview of a printer qualification process (cont.)

Phase II Preproduction print engines are either scarce or not available yet.

Preproduction formatter hardware is available.

The formatter (firmware + hardware) is tested with a print engine emulator.

Test image (PDL)

Newly developed formatter + print engine emulator

Formatter of an earlierproduct + print engine emulator

Digital outputsmatch qualitatively?

Master Current

Overview of a printer qualification process (cont.)

Phase III Preproduction printers (formatter + print engine) are available.

Test image (PDL)

Newly developed printerEarlier printer model

Hardcopiesmatch qualitatively?

Master Current

Overview of a printer qualification process - summary

Simulator without actual formatter/ emulator with actual formatter

Test image(PDL)

Softcopy current image

Comparison masters

Softcopy

Formatter/firmware

development teams

Debug

Error flagged

Actual formatter Hardcopy current imagePrint engine

HardcopyDebug

Motivations Image screening (comparison of master-current image pairs)

is mostly performed by trained observers.

is needed for thousands of softcopy and hardcopy image pairs throughout the printer development process.

is very labor intensive.

is often performed only on a fraction of test suites before the product is rolled out due to

» a relatively short development time.» a large number of test images in the test suites.

Our goal is to develop a automated tool to reduce the workload of trained

observers and increase the volume of tests the by screening out softcopy image pairs with

» highly objectionable errors (failed).» visually insignificant errors (passed).

Trained observers only need to focus on image pairs which

could not be screened out by the tool (further evaluation)

Examples of master-current image pair

Master Current

Different halftone algorithms

Missing pixels

Master Current

Examples of master-current image pair (cont.)

Different in character size

Master

Current

Sporadic difference

CurrentMaster

Image fidelity metrics – prior work First category

Examples» Peak-signal-to-noise ratio (PSNR), root mean square error, and ∆Ea*b*

Pros» They are fast and easy to compute, and produce a single number.

Cons» Averaging effect destroys local information and spatial interaction of

pixels is ignored.

Second category Examples

» Wu’s color image fidelity assessor (HVS model) and structural similarity image metric (first and second order statistics the luminance channel of a local window)

Pros» Spatial processing model is included.

Cons» The algorithm is computational expensive and does not produce a

single number (Wu’s color fidelity assessor) » SSIM produces a single number but suffers from the averaging effect.

Prescreening tool - requirements

needs to work reasonably fast.

classifies the master-current pairs into one of the categories:

‘passed’, ‘failed’, and ‘further evaluation required’.

must adapt automatically as the spatial resolution of the images

changes in dpi.

has to be capable of processing any image content.

needs to handle both halftone and continuous-tone images.

will process full color, indexed color, grayscale, and bilevel

images consistently.

will detect and ignore small spatial shifts in content and

differences in orientation

Master image Current image

PASSEDFAILED

The prescreening tool

FURTHER EVALUATION

13

Preprocessing

Compute 2D error map

Cluster error map

Compute error metric

Thresholding

Prescreening tool - preprocessing

Determine shift in content

Master Current

• Determine image type

• Perform quick diff

• Determine resolution

• Correct orientation

• Correct spatial shift

Other processing

Con

ten

t si

ze

Imag

e si

ze

Check dimensionRotated CWRotated CCW

Master Current Current

Check orientation

14

Prescreening tool – compute and cluster 2D error map (illustration)

Master M i

j

i

j

Current C

i

j

Binary Error Map

j

iClustered Error Map

Clu

stering

kth cluster

(k+1)th cluster

(k+2)th cluster

Prescreening tool – compute error metric (contrast sensitivity function CSF component)

Master i

j

i

j

Currentcorrespond to the lth pixel of the kth cluster

Mkl

Ckl

Average filtered pixel value for the kth cluster

Master :

Current:

Error for the kth cluster:

CSF error component for the image pair:

MM

kllk

k N 1

CC

kllk

k N 1

To CIE L*a*b*Mk

Ck

CMkk

CSF

kbaE

**

k

CSF

kbaktot

CSFba EN

NE ****

1

Prescreening tool – compute error metric (contrast sensitivity function CSF component)

(cont.)

Prescreening tool – compute error metric (visual acuity filter VAF component)

Master i

j

i

j

Currentcorrespond to the lth pixel of the kth cluster

Mkl

Ckl

Average filtered pixel value for the kth cluster

Master :

Current:

Error for the kth cluster:

VAF error component for the image pair:

MM

kllk

k N 1

CC

kllk

k N 1

To CIE L*a*b*Mk

Ck

CMkk

VAF

kbaE

**

k

VAF

kbaktot

VAFba EN

NE ****

1

Prescreening tool – compute error metric (visual acuity filter VAF component)

(cont.)

Prescreening tool – compute error metric (overall error)

Combined error

Error metric value for the image pair

maperror thein pixels of # total1 where

**

totf

NVAFCSFba

NN

Ef

Ea*b*{ }CSF+VAF

= ΔEa*b*{ }CSF⎡

⎣⎤⎦

N p

+ ΔEa*b*{ }VAF⎡

⎣⎤⎦

N p

{ }

1

N p

where N p = 1+ 2 ⋅tanh(max( ΔEa*b*{ }CSF

, ΔEa*b*{ }VAF

))

Experiment There are 147 image pairs.

Test image pairs are of 300, 600 and 1200 dpi. binary, grayscale, color bitmap and full color.

Six expert observers classify the image pairs into 3 categories Acceptable (passed). Need further evaluation. Objectionable (failed).

These image pairs are processed with our prescreening tool.

The peak signal to noise ratio (PSNR) metric and structural similarity image metric (SSIM) are also computed for each preprocessed (to ensure fair comparison) image pair.

21

Results – error metric 22

A zero error metric value indicates a perfect match.

Decision thresholds exist to pass and fail image pairs.

The prescreening tool is able to screen out as many as 35% of the image pairs tested.

Results - PSNR A larger PSNR value (PSNR = ∞ for a perfect match) indicates a closer

match.

Only decision thresholds to pass image pairs exist.

The PSNR metric is able to screen out only as many as 4% of the image pairs.

Results - SSIM 24

A larger SSIM value (SSIM = 1 for a perfect match) indicates a closer match.

Only decision thresholds to fail image pairs exist.

The SSIM metric is able to screen out only as many as 3.4% of the image pairs.

Conclusion

We have successfully developed an automated prescreening tool

along with an image fidelity metric for a printer qualification

process.

This tool works for a wide range of image types, content, and

image resolutions.

It requires no training and it is able to reduce the workload of

expert observers by a substantial amount.

25