Partially Coherent Ambiguity Functions for Depth-Variant Point Spread Function Design

Roarke Horstmeyer1

Se Baek Oh2

Otkrist Gupta1

Ramesh Raskar1

PIERS Marrakesh, 3/20/11

1MIT Media Lab2MIT Department of Mechanical Engineering

Conventional camera PSF measurement:

Conventional PSF Blur

defocus

Circular Aperture

I(z0) I(z1) I(z2)

z0 z1 z2

f

Pt.

So

urce

Problem Statement

3

Design apertures for specific imaging tasks

Determine mask

Aperture mask:amplitude/phase

defocus

Desired set of PSFsI(z0), I(z1), I(z2)

z0 z1 z2

Problem Statement

ExamplesDe

focu

s

PSFs: Depth-Invariant Rotating Arbitrary

Cubic phase Gauss-Laguerre modes Iterative Design

?

PSF Design Similar to Phase Recovery

d

z0 z1 z2

Diff

racti

ve

Elem

ent

f

z0 z1 z2

Aper

ture

El

emen

t

3D PSF Design Measurement for phase recovery

I(z0), I(z1), I(z2) I(z0), I(z1), I(z2)

General Goal: Find (A,ϕ) from multiple intensities

(A,ϕ) (A,ϕ)

Similar to

Models: Fresnel propagation, k-space, light fields, phase space

I(x1,z1)I(x0,z0)U(A,ϕ)

(a) Phase Retrieval

iterative

iterative

I(xn,zn)I(x0,z0) …

(c) Phase Space Tomography

U(A,ϕ)

simultaneous

I(x1,z1)I(x0,z0)

(d) Mode Selective (proposed)

U(A,ϕ)

simultaneous

iterative

Overview of 3D Design Techniques

I(x1,z1)I(x0,z0)

(b) Transport of Intensity

U(A,ϕ)

simultaneous

I(x1,z1)I(x0,z0)U(A,ϕ)

(a) Phase Retrieval

iterative

iterative

I(xn,zn)I(x0,z0) …

(c) Phase Space Tomography

U(A,ϕ)

simultaneous

I(x1,z1)I(x0,z0)U(A,ϕ)

simultaneous

iterative

I(x1,z1)I(x0,z0)U(A,ϕ)

simultaneous

Phase space extends nicely to partially coherent design

(b) Transport of Intensity

Overview of 3D Design Techniques

(d) Mode Selective (proposed)

Phase Space Functions

Wigner Distribution (WDF) Ambiguity Function (AF)

AF “easier” than WDFTu, Tamura, Phys. Rev. E 55, 1997

OTF(z1)

OTF(z0)

OTF(-z1)

F-slice

PSF(z0)

PSF(-z1 ) PSF

(z 1)

WDF Projections: PSFs

x

u

u

x'AF Slices: OTFs

z0 z1

fx

U(x)

Phase Space Camera Model r

Δz

z0I

xʹ

I

xʹ

z1

OTFs

Aperture mask

z0 z1

fx

U(x)

tan(θ0)=0

u

xʹ

Phase Space Camera Model

Why AF is useful:

z0I

xʹ

I

xʹ

z1tan(θ1)=(W20k/π)

OTFs

r

Δz1. Polar display of the OTF

z0

z1

Aperture mask

Aperture mask z0 z1

fx

U(x)

tan(θ0)=0

u

xʹ

Phase Space Camera Model

Why AF is useful:

z0I

xʹ

I

xʹ

z1tan(θ1)=(W20k/π)

OTFs

r

Δz

2. Convert AF to Mutual Intensity: inverse FT, 45° rotation, scale by 2

3. Recovery of U(x) from AF (up to constant Δϕ)

z0

z1

1. Polar display of the OTF

OTF(z1)

OTF(z0)

PSF OTF Inputs

Output: Desired Aperture Mask, 1D

(1) AF Population

(2) One-time AF Interpolation

OTF(z2)

θn

(5) Optimized AF

xʹ

xʹ

u 1

0

xʹ

u

xʹ

u

x2

x1

x1

x2

(3) Mutual Intensity J

Rank Constraint

(4) Optimized J

Error Check

1

0

1

0

Iterate &

Rank-constraint on Mutual Intensity, J

= λ1 + λ2 +…+ λ3

x1

x2

x1

x2

Represent J with coherent mode decomposition1

Coherent, orthogonal modes from singular value decomposition

J = UΛVT = Σ λi Ui(x1)Ui*(x2)

Imperfect J guess: Many coherent modes

Assume: J symmetric, nxn λi = Singluar ValuesUi orthogonal to Uj for all i≠ji=1

n

J

1 E. Wolf, JOSA 72 (3), 1982

= λ1 + λ2 +…+ λ3

1st Mode: Coherent

x1

x2

x1

x2

PSF = response to a point source: restricted to 1 mode

Rank-constraint on Mutual Intensity, J

Jest = λ1 U1(x1)U1*(x2)

J

Represent J with coherent mode decomposition1

Coherent, orthogonal modes from singular value decomposition

1 E. Wolf, JOSA 72 (3), 1982

Ground Truth AF Reconstructed AF Computed Phase Mask

Ground Truth and Reconstructed OTFs

W20=0 W20=λ/2 W20=λ

-π

+π

xʹ

u

xʹ xʹ xʹ

xʹ

u

Reconstruction Example: Cubic Phase Mask

Example aperture mask function: exp(jαx3), α=40, 20 iterations

Simulation

Experiment

Amplitude

One (fixed) mask

Simple Example: Arbitrary InputInput

I1(x) I2(x) I3(x) z1=50mm z2=50.1mm

Rank-1 constraint

25μ – resolution, 1cm2 binary mask in 50mm f/1.8 Nikkor, 200μ pinhole @ z=4m

z3=50.2mm

50μ

-π

+π

No Constraints on (A,ϕ)

Ph

ase

(rad

.)

x (cm)

Am

plitud

e (AU

)

Amplitude-onlyconstraint

Phase-only constraint

Constrained Decompositions

In Experiment: Amplitude-only or Phase-only required

MSE vs. # iterations

MS

E

# of iterations

Aperture mask constraints:

-Varied performance-Algorithm still converges

Keeping More than One Mode

= λ1 + λ2 +…+ λ3

Several Modes: Partially Coherent

x1

x2

x1

x2

-More accurate estimate found with n > 1 modes

J

n = 3: (J - Σ λiUi(x1)Ui(x2))2 = global minimum

Eckert-Young Thm.: 1st n-modes of SVD(J) = optimal rank-n estimate

SVD = Optimal estimate (L2 norm, no prior knowledge)

i=1

3

Simulating Partial Coherence

= λ1 + λ2 +…+ λ3

x1

x2

x1

x2

-More accurate estimate found with n > 1 modes

Multiple modes can be multiplexed over time1,2

JSeveral Modes: Partially Coherent

1 P. De Santis, JOSA 3 (8), 1986, 2 Z. Zhang, private communication, 2011

Spatial Light Modulator:Vary over time

CPU

Simulation

Example: Benefit of Several ModesInput

MSE improvement ~100x(modes contain both A and ϕ)

1 2

3

Display 3 Optimal masks

I1(x) I2(x) I3(x)z1=50mm z2=50.1mm z3=50.2mm

50μ

1cm2 masks

Rank-3 constraint

Experimentally: A,ϕ over time = hard

“Weights”:1 - 12 - .613 - .38

Adding a Constraint to Several Modes

SVD & constrain in separate operations: No convergence

= μ1 + μ2 +μ3

x1

x2

On(J)

x1

x2

General solution: convex optimization

e.g.: amplitude-only, phase-only, spatial1 and coherence constraints

min || J – Σ μiWi Wi* || 2 subject to constraints on W, given n

i=1

n

1 Flewett et al., Optics Letters 34 (14) 2010

W1=? W2=? W3=?

Operation On(J) = find closest n rank-1 outer-products, constrained

Example Constraint: Amplitude-only

Problem: n optimal coherent modes that are real, positive

min || J – WWT || 2 W ≥ 0, real (J = kxk, W=kxn)

Example Constraint: Amplitude-only

Problem: n optimal coherent modes that are real, positive

min || J – WWT || 2 W ≥ 0, real (J = kxk, W=kxn)

Solution: Non-negative matrix factorization1

-e.g. Netflix challenge: low-rank rep. of 0-5 star movie scores

Symmetric NMF: add to update rules (solve for W &H, W≈H)

Note: Optimal “Coherent modes” are no longer orthogonal

Update Rules: Add line 1δ=tiny value, error ~2-5%

1. H=WT

2. W=W.*(HTJ)./((HHT)H+δ)3. H=H.*(WTJ)T./H(WW)+δ)

1Lee and Seung, Nature 401, 2001

A Simple Example: Multiple Amplitude ModesI1(x) I2(x) I3(x)

1 2

3

1cm2 masks

Amplitude-only, 3 masks

Amp-only masks: Sym. NMF

“Weights”:1 - 12 - .783 - .08

Simulation

Input

MSE improvement ~7x (vs. 1 Amp. mode)

Buildup of a baseline bias…

z1=50mm z2=50.1mm z3=50.2mm

50μ

Conclusion & Future Work

-Phase space functions = intuitive window into 3D PSF design

-Multiple modes (partially coherent) = increased flexibility

-Constrained searches can be achieved w/ convex methods- Amplitude-only: Symmetric NMF- Other constraints: Phase-only (another convex

implementation), coherence length (weighted SVD)- Subtracting modes: J=U1U1

* ± U2U2*±… (take 2 images)

-Current: Initial Experimental tests using an SLM

-Next Step: Find a nice application

Thanks! Questions?

Jpc(x1,x2)

AFpc(x',u)

x1(cm)

x2

u

.5

3 Coherent Modes

1cmMask

x'(cm-1) 5e4-5e4

-.5 .5-.5x(cm)

u

x2

Partially Coherent Reconstruction: 3 Modes

x1(cm) .5-.5

x'(cm-1) 5e4-5e4

Ground Truth Reconstructed

Constraining Several Modes

Apply Constraint: Amplitude-only, Phase-only, prior knowledge, etc.

= λ1 + λ2 +…+ λ3

x1

x2

Sum=No longer optimal (localized constraints will not converge)

individualconstraint

x1

x2

x1

x2

+ λ2 + λ3λ1

SVD(J)

amplitude-only amplitude-only amplitude-only

Example: Amplitude-only maskindividualconstraint

individualconstraint

A Simple Example: Prior Knowledge

A B

C

3 modes hitting unknown structure

(A=.4, B=.5, C=.7)+

SVD(J) = 2 orthogonal modesx

SVD(J)

=x2

x1

Negative values = phase

U1U1* ε (0,.9) U2U2

* ε (-.1,.4)

A Simple Example: Prior Knowledge

A B

C

3 modes hitting unknown structure

(A=.4, B=.5, C=.7)

+

SNMF(J)

SVD(J) = 2 orthogonal modes

=

x

x2

x1

Symmetric NMF: Assume no phase change- 3 modes>0, more info about structure

+ +

U1U1* ε (0,.47) U2U2

* ε (0,.6) U3U3* ε (0,.6)

Negative

Keeping More than One Mode

= λ1 + λ2 +…+ λ3

Several Modes: Partially Coherent

x1

x2

x1

x2

-More accurate estimate with n>1 mutual intensity modes-Jest= Σ Ji <-> AFest = Σ AFi = Σ L(Ji)1,2 (L=linear transformation)-If Jest more accurate, then AFest more accurate

J

Multiple Modes can be: a. Multiplexed over time (Desantis, Zheng)b. Could also multiplex over space and/or angle

1M. Bastiaans, JOSA 3(8) 1986, 2Lohmann and Rhodes, Appl. Opt. 17, 1978