Some Problems (and Solutions) in Bio-Optical Inverse Problems and Image Processing Eric Miller, Prof and Chair, Electrical and Computer Engineering Tufts University eric.miller@tufts.edu

2

Tufts School of Engineering

Founded in 1898 77 full-time faculty Six departments (14 majors):

– Biomedical, ChemBio, Civil and Environmental, Computer Science, Electrical and Computer, Mechanical

– Seven ABET-CAC accredited degree programs

750 undergrad students 200 full-time/200 part-time graduate

students

3

Tuft’s Electrical and Computer Engineering Department

16 Core Faculty – Plus adjunct faculty and faculty with joint appointments

with Biomedical Engineering and Computer Science ~60 Graduate Students

– 75% Ph.D. students – 25% M.S. students – 10 Teaching Assistants and ~28 Research Assistants

Approximately 120 ECE undergraduates – Average SAT score of admitted:

1465 (Math + Verbal)

4

The Faculty and Their Research

Research Honors and Awards – 7 NSF CAREER awardees – ONR, AFOSR, Intelligence Community Young Investigators – Four IEEE Fellows – Presidential Science, Math, Engineering Mentor Award

Research program stats – $2.5 M expenditures last year – Healthy mix of NSF, NIH, DoD, DoE, DHS, private support

Areas of expertise – Compressive sensing, distributed detection and estimation, cyber-physical energy systems,

cognitive radio and multi-user communications, network information theory, physics-based image formation and fusion, image processing for security and medical applications

– Microwave circuits, millimeter and far IR spectroscopy, microplasmas for sensing and processing

– Analog/Mixed signal VLSI, Silicon Electronic-Photonic ICs, CMOS Nanoelectronics, Biomedical; circuits ands systems, thermophotovoltaics, photovoltaics, infrared detectors

5

Bio-optical Imaging

Basic idea: Use of near infrared light (650-900nm) to probe tissue noninvasively

Minimal absorption, but highly scattering Physics ~ diffusion (transport) rather than

propagation Low spatial resolution, but good temporal Capable of functional imaging

6

Diffuse Optical Tomography

Input light at one wavelength Observe at same Image optical absorption and

scattering Multiple wavelengths then

image chromophore concentrations

Useful for functional brain imaging and breast cancer detection

7

Fluorescence Molecular Tomography

Input light at one wavelength Light excites fluorochrome in the

issue – Naturally occurring – Exogenous problem – Genetic manipulation

Data collected at fluorescing wavelength

Image fluorochrome distribution Molecular imaging Unlike DOT, optical properties of

the tissue are a nuisance, not the objective

8

My DOT Foci

Combining DOT with information from structural modalities e.g. tomosynthesis

Chromophore inversion in space and time – Hyperspectral data sets

Inverting for shape rather than pixels

9

Background Medium Object of interest

1. Lasers probe medium (multiple locations and wavelengths)

2. Light diffuses into medium

3. Photon density wave scatters from optical inhomogeneities

4. Photon density wave measured at surface

Physics of DOT

10

Background Medium

Object of interest, 𝜇𝜇𝑎𝑎,𝐷𝐷

1. Lasers probe medium, q

3. Photon density wave, Φ

The DOT Problem

Diffusion equation −𝛻𝛻 ⋅ 𝐷𝐷 𝑟𝑟; 𝜆𝜆 𝛻𝛻Φ 𝑟𝑟; 𝜆𝜆

+ 𝜇𝜇𝑎𝑎 𝑟𝑟; 𝜆𝜆 Φ 𝑟𝑟; 𝜆𝜆 = 𝑞𝑞 𝑟𝑟𝑠𝑠, 𝜆𝜆

• Vary 𝑟𝑟𝑠𝑠 and 𝜆𝜆 • For each source/wavelength

pair measure Φ at 𝑟𝑟 =𝑟𝑟1, 𝑟𝑟2, … , 𝑟𝑟𝑁𝑁

• Use these data to determine 𝜇𝜇𝑎𝑎 and (maybe) 𝐷𝐷.

11

From Optical Properties to Chromophores

Optical absorption is nice, but practitioners more interested in distribution of chromophores – Oxygenated hemoglobin, deoxygenated hemoglobin, lipid

concentration, water, etc

These quantities more directly related to the physical state of the system; e.g., tumor consume more O2

Model

εk = extinction coefficient for k-th chromophore at wavelength λ

ck = concentration of k-th chromophore at location r

12

The Inverse Problem

Regardless of the specific application, the inverse problem takes the form

Let f be the property of interest (absorption, chromophores, etc.)

Think of f as being parameterized in some manner; f = f(r ; p)

Regularization functional Physical

model

Observed data

(photon counts)

Weight matrix

13

Pixel-based methods

p = collection of pixel values

Power =1: small penalty for large gradient allows edges to form

Power = 2: larger penalty leads to smoothed images

14

Shape-based Methods

S

The model

Typically • p = boundary • fin and fout contrast coefficients

(could be generalized for texture modeld)

• Regularize by seeking short boundary

𝑓𝑓 𝑟𝑟 = 𝑓𝑓in𝑆𝑆 𝑟𝑟 + 𝑓𝑓out 1 − 𝑆𝑆 𝑟𝑟

𝑆𝑆 𝑟𝑟 = �1 𝑟𝑟 ∈ Object0 else

15

Our Approach to Shape

A very specific model

so that 𝑝𝑝 = 𝑎𝑎𝑖𝑖 ,𝛽𝛽𝑖𝑖 , 𝑟𝑟𝑖𝑖 Intuition:

– Where 𝜙𝜙 𝑟𝑟 > 0 we have the object – With 𝑏𝑏”bumps” we are looking to adjust the

various parameters to get a nice shape A variant of so-called “level set” methods

quite popular in the computer vision community

𝑆𝑆 𝑟𝑟 = 𝐻𝐻 𝜙𝜙 𝑟𝑟;𝑝𝑝 with 𝐻𝐻 𝑥𝑥 = �1 𝑥𝑥 ≥ 00 else

𝜙𝜙 𝑟𝑟;𝑝𝑝 = �𝑎𝑎𝑖𝑖𝑏𝑏 𝛽𝛽𝑖𝑖 𝑟𝑟 − 𝑟𝑟𝑖𝑖

𝑁𝑁

𝑖𝑖=1

16

In pictures

• Merging of positively weighted bumps makes something like the union of the regions where they exceed zero.

• Difference of bumps all us to build objects with holes and corners • Bumps with equal weights give straight edges

17

The Fantini System

DOT standalone Scanning system

– Illuminate from the bottom – Five detectors on the top – 25 points per cm2

Hyperspectral capabilities – Well over 100 frequency bins

between 650 and 1000 nm Only CW data will be used What can we do in terms

of tomographic inversion?

18

PaLS for DOT

Recently have begun developing these ideas for the Hyperspectral DOT problem

Use Born (linearized) forward model

Anomaly geometry shared across chromophores, contrasts differ

Recover shape parameters and ai Simple Gaussians on a fixed grid for the PaLS basis set

𝑐𝑐𝑖𝑖 𝑟𝑟 = 𝑎𝑎𝑖𝑖𝐻𝐻 𝜙𝜙 𝑟𝑟, 𝜇𝜇 𝑖𝑖 = 1,2, … ,𝑁𝑁 + 1

19

Experimental Setup

Tank of 2% milk – Properties carefully

measured India Ink and Blue Food

Dye mixed with milk and water to achieve μa=0.029/cm at 600 nm

Embedded in clear tubes 8 source positions and 3

receivers per source – Not a lot of data

20

Experimental Results

6 wavelengths 126 wavelengths

Ink

Dye

21

Experimental: PaLS

6 wavelengths 126 wavelengths

Ink

(70%

) D

ye

(30%

)

22

Blood Vessel Reconstruction

Recover support of blood vessels

Very limited data One PaLS function per

slice Cross-slice

regularization to enforce slice-to-slice coherence

Only one wavelength for this work

23

Simulation results

Truth

Recovered 10 detectors per source location

Note: pixel based methods fail miserably

24

Real data: Setup

• Silicone phantoms embedded in tissue mimicking slabs

• Realistic contrasts • Three detectors per source

25

Real data: Results

26

Conclusions

Overview of bio-optical image formation Physics-based tomographic techniques Concentration on geometric methods Pixel-based (and mixed) are easily done Many issues not covered including time lapse

and multimodal Please contact me for more information!