zeev zalevsky for knowledge stream

1

Photonic Remote and Continuous

Biomedical Diagnostics

Zeev Zalevsky1,2

1Faculty of Engineering, Bar-Ilan University, Israel2SAOT, Friedrich-Alexander Universität Erlangen-Nürnberg, Germany

2

Main collaborators:

Yevgeny Beiderman1

Javier Garcia2

Vicente Mico2

Israel Margelith1

Asaf Shahmoon1

Alexander Douplik3

Dan Cojoc4

1Faculty of Engineering, Bar-Ilan University, Israel2Departamento de Óptica, Universitat de València, Spain3Ryerson University, Toronto, Canada4Trieste, Italy

3

Outline

•“Hearing” with light – Introduction

•Biomedical monitoring:

•Introduction

•Measuring of breathing

•Heart beats monitoring

•Glucose level monitoring

•Blood pulse pressure monitoring

•IOP monitoring

•Malaria detection

•Alcohol detection

•Oximetry and coagulation of blood

•Micro endoscope

•Conclusions

4

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

5

Any visible distance

Imaging module

Invisible Laser projection

Camera

Sensor

Laser


Imaging module


Camera

Sensor

Laser


Imaging module


Camera

Sensor

Laser

Opto-Phone: Hearing with Light

6

Hearing with Light: Features

•The ultimate voice recognition system compatible to “hear” human speech from any point of view (even from

behind).

•There is no restriction on the position of the system in regards to the position of the sound source.

•Capable of hearing heart beats and knowing physical conditions without physical contact for measuring.


7

Features- cont.

•Works clearly in noisy surroundings and even through vacuum.

•Allows separation between plurality of speakers and sounds sources.

•Works through glass window.

•Simple and robust system (does not include interferometer in the detection phase).


8

Let’s listen…from 80m

Heart beat pulse

taken from a throat

Cell phone

Counting…1,2,3,4,5,6

Face (profile)

Counting…5,6

Back part of neck

Counting…5,6,7

All recordings were done in a very noisy constriction site at distance of more

than 80m.

120 140 160 180 200 220 240 260 280 300 320

-1

-0.5

0

0.5

1

X - movement

9

Results: Detection of occluded objects I

(a). Camouflaged object. (b). Camouflage without the object. (c). The object (upper left part) and the low

resolution camouflaged scenery.

(a). (b). (c).

(d). The spectrogram of the camouflaged object with its engine turned on. (e). The spectrogram

of the object with its engine turned on and without the camouflage. (f). The spectrogram of the

camouflaged object without turning on its engine.

Spectrogram

Fre

quency [H

z]

Time [sec]

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

0

50

100

150

200

250

300

350

400

450

500

200

400

600

800

1000

1200

1400

1600

1800

2000

Spectrogram

Fre

quency [H

z]

Time [sec]

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

0

50

100

150

200

250

300

350

400

450

500

100

200

300

400

500

600

700

800

900

Spectrogram

Fre

quency [H

z]

Time [sec]

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

0

50

100

150

200

250

300

350

400

450

500

2

4

6

8

10

12

(d). (e). (f).

10

Results: Detection of occluded objects II

0 1000 2000 3000 4000 5000 6000

-20

-15

-10

-5

0

5

10Y - pos

Sample

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000-15

-10

-5

0

5

10Y - pos

Sample

0 1 2 3 4 5 6 7 [sec]

(a). The scenario of the experiment. (b). Experimental results: upper recording is of the

camouflaged subject. Lower recording is the same subject without the camouflage.

(a). (b).

11

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

12

Technological Description

•Unique technological platform allows remote and continuous wearable monitoring of many biomedical parameters simultaneously.

•It is based upon inspection of secondary speckle pattern back reflected from skin near main blood artery, after properly adjusting the imaging optics.

•The biomedical monitoring capabilities include: heart beats, breathing, blood pulse pressure, glucose concentration, alcohol level, IOP, blood coagulation (INR), oximetry, ICP etc.

•Unique patented IP and know how.

•Part of the applications have already been commercialized.

13

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

14

-15 -10 -5 0 5 10 150

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

X: -1.872

Y: 1.808

Frequency [Hz]

Am

plitu

de

(Abs) Spectrum

Detected rat’s breathing beating at

frequency around 1.87Hz.

Reflected speckle pattern.

Noise level

Measuring of breathing from rat’s cornea reflections

15

Detected heart beating of humans at frequency of around 1.5Hz.

Subject #1 (measurement taken while

subject was holding his breath)

Subject #2 (measurement taken while

subject was holding his breath)

Reference noise level (detected

reflection from a wall)

-20 -15 -10 -5 0 5 10 15 200

5

10

15

20

25

30

35

X: 1.502

Y: 33.65

Frequency [Hz]

Am

plit

ud

e

(Abs) Spectrum

-20 -15 -10 -5 0 5 10 15 200

5

10

15

20

25

X: 1.522

Y: 19.96

Frequency [Hz]

Am

plit

ud

e

(Abs) Spectrum

-20 -15 -10 -5 0 5 10 15 200

2

4

6

8

10

12

X: 3.078

Y: 11.68

Frequency [Hz]

Am

plit

ud

e

(Abs) Spectrum

Noise level

Noise level

10 20 30 40 50 60

10

20

30

40

50

60

Reflected speckles pattern.

Measuring of heart beating from human’s cornea reflections

16

Measuring breathing of pigs

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9

Breath Breath measured

Brea

ths p

er m

inut

e

Experiment

Statistical breathing experiment

The non-visible

laser system

The swine's

location

40 m

Laser beam

17

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

18

Remote heart beats monitoring

Hand

CameraLaser

50 cm

0 1 2 3 4-1

-0.5

0

0.5

1

1.5

2

2.5

3

Time [sec]

Am

plit

ud

e

[pix

]

0 1 2 3 4

-10

-8

-6

-4

-2

0

2

Time [sec]

Am

plit

ud

e

[pix

]

Temporal plot of the outcome from the

system used in the clinical trials for two

different participants.

The implemented optical configuration for

remote measuring of heart beats and blood

pulse pressure from subject’s hand

19

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

20

Temporal plot of the outcome from the system used in the clinical tests with the graphical description of the observed

parameters.

Glucose level monitoring

21

0

5

10

15

20

25

30

35

40

0 5 10 15 20 25 30

Am

pli

tud

e [s

am

p];

G

luc

os

e[m

l/d

l/1

0]

Time [min]

Glucose /10

Param.6

Stability of the system: constant glucose level in blood (denoted

by blue line with triangles) and the estimated parameter 6

(denoted by magenta line with rectangles). Glucose level is

given in units of 0.1[ml/dl] (representing a constant level of 100

[ml/dl), while the estimated optical values are given in pixels.

50

70

90

110

130

150

170

190

210

0 5 10 15 20 25 30

time [minutes]

Glu

co

se

[m

g/d

l]

,

Data of subject #1: Glucose level in blood and amplitude

of positive peak (parameter #1). Glucose level is denoted

by blue line with triangles and the optically measured

parameter is denoted by magenta line with rectangles.


22

50

70

90

110

130

150

170

190

210

0 5 10 15 20 25 30

time [minutes]

Glu

co

se

[m

g/d

l]

,

Data of subject #1: Glucose level in blood and

amplitude of positive peak (parameter #1). Glucose

level is denoted by blue line with triangles and the

optically measured parameter is denoted by magenta

line with rectangles.

22

50

70

90

110

130

150

170

190

210

0 10 20 30 40

time [minutes]

Glu

co

se

[m

g/d

L]

Data of subject #3: Glucose level in blood and amplitude of positive peak

(parameter #1). Glucose level is denoted by blue line with triangles and the

optically measured parameter is denoted by magenta line with rectangles.

50

70

90

110

130

150

170

190

0 5 10 15 20 25 30

time [minutes]

Glu

co

se

[m

g/d

l]

Data of subject #4: Glucose level in blood and amplitude of

positive peak (parameter #1). Glucose level is denoted by blue

line with triangles and the optically measured parameter is

denoted by magenta line with rectangles.


23

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

24

Blood pulse pressure measurement

-50 0 50 100 150 200 250 300 350 40020

40

60

80

100

120

140

Time [sec]

Am

plit

ud

e

[mm

Hg]

M , Corr(M , ) = 0.99507

Systolic

Diastolic

= S-D

An example of the obtained remote blood pulse pressure measurement using the proposed device for one subject

participating in the clinical test group. The reference pulse pressure is shown by the green curve (denoted as ) was

obtained using manual sleeve based reference measurement device. The blue curve (denoted as M) is the measurement

obtained using the proposed optical technique. The time duration of the measurement was 350sec. The sampling of the

camera was performed at 300Hz. One may see that strong correlation exists between the green (reference) curve and the

blue curve obtained by the developed approach.

25

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

26

The proposed experimental configuration for

remote continuous monitoring of the IOP.

Remote IOP monitoring

Changing IOP via modifying the height of an

infusion bug.

Changing IOP via applying mechanical pressure

on the sclera.

27

0 1000 2000 3000 4000 5000 6000-50

-40

-30

-20

-10

0

10

20

30

40

Sample

Am

plit

ud

e [p

ix]

Data amp: 8.017 10.7025 11.2649 11.6035 9.3484 7.2619 7.2177 6.4588

0 500 1000 1500-6

-4

-2

0

2

4

6

Sample

Am

plit

ud

e [p

ix]

Data amp: 7.3465

Data amplitude:

8.02 10.7 11.27 11.6 9.35 7.26 7.22 6.46

Experimentally extracted readout

obtained when changing the height of

the infusion bag every 500 samples.

Remote IOP monitoring

6.872

3.54042.9932

2.1451.76402

0

1

2

3

4

5

6

7

8

9

0 20 40 60 80

Am

pli

tud

e

Pressure (mm/Hg)

amp/pressure (mmHg)

1501

6.872

3.54042.9932

2.1451.76402

0

1

2

3

4

5

6

7

8

9

0 20 40 60 80

Am

plit

ude

Pressure (mm/Hg)

amp/pressure (mmHg)

1501

With infusion bag

With applied pressure

Experimentally extracted readout compared to

absolute reference IOP measurement obtained

with Goldmann tonometer.

28

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

29

Detection of malaria

An example of one out of the 20 relevant inspected parameters. Left: Healthy RBC. Right: Infected RBC.

30

Detection of malaria

Separation between infected and

healthy cells. Plotting the length of

the vectors versus cells’ index.

31

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

32

Remote alcohol level monitoring

Green laser

Camera (a). (b).

(a). The side (left) and top (right) view of the experimental setup. (b). Typical temporal beating signals extracted

using the proposed remote optical sensing device, before (left) and after (right) the effect of alcohol obtained

over the same subject.

33


Definition of the Ratio wid (through the ratio between the main

and the secondary negative peak’s temporal positions), Main

sec peak ratio (through the ratio between the main negative

peak amplitude and the secondary positive peak’s amplitude),

and Standard deviation of background noise (STD) parameters.

STD of background noises: long duration test with

error bars representing the std values of the

measured data.

34


Time [min]

Pu

lse

size

[m

sec]

Time [min]

Pos

itiv

e pu

lse

size

[m

sec]

(a). (b).

Time [min]

Pea

kd

is [

mse

c]

Time [min]

Rat

io w

id

(c). (d).

Time [min]

Mai

n s

ec p

eak

rat

io

Time [min]

Std

[m

sec]

(f).(e).

Summary:

(a). Pulse size, (b). Positive pulse

size, (c). Peakdis, (d). Ratio_wid,

(e). Main_sec_peak ratio, (f). Std

of background noises.

35

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

36

Oximetry and coagulation of blood

0 2 4 6 8 10 120.5

1

1.5

2

2.5

3

3.5

Oxygen

Am

plit

ud

e [p

ix]

Test # 0 2 4 6 8 10 12 14 16 18

1

1.2

1.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3

INR

No

rmili

ze

d IN

R

Exp #

Oximetry experiment Blood coagulation experiment

37

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

38

The edge of the fabricated micro

probe having approximately 5,000

cores while each one of them is

being used as light transmitting

channel (each core is a single

pixel in the formed image). In this

image each core transmits red

channel of light at wavelength of

632nm.

Multi-Functional Probe

50m

Object ImageMulti core probeInput plane Output plane

U1 U2 V

F

1 2

1 1 1

U U V F

Laser (632nm) Beam Expander Mirror

Beam Splitter

Objective Lens

Probe Location

CameraLaser Controller

Sample

Location

39

Multi-Functional Probe- experiments

5m

2m

Experimental results of images transmitted backwards by the proposed micro probe. The scanned objects are as follows;

From left to right: black vertical lines, black rectangles, horizontal black lines, black lines and black rectangle appearing

in the left side of the backwards transmitted image.

Experimental results of images with Fe beads having diameter of 1m imaged

through an agar solution.

40

(a) Fabricated phantom. (b) shows a 3D view sketch of the phantom having two drilled channels with diameter of

400µm each. One longitudinal channel (along the x axis) while another angled channel was made making both

channel crossed inside the phantom. The openings indicated as “in” and “out” enable the connection of microfluidic

system. (c) shows a cross-sectional schematic view of the fabricated phantom.

(a) (b) (c)

Phantom fabrication

41

Experimental results of Fe micro

particles imaged inside a drilled

phantom

Imaging of a manipulated micro wire (indicated by

the solid arrows) inside an hemoglobin mixture.

Experimental results with phantom

Imaging of fluorescence protein. HEK 293 cells transfected

with pEGFP-N3. Left: Top view microscope image. Right:

Imaging using the microendoscope device. Scales bar of

left and right image are 50 and 20 µm, respectively.

(a) Top view microscope image of the resolution target. (b) imaging

of the resolution target using the microendoscope device. Inset.

Zoom image of the encompass area. Scales bar of (a) and (b) are 10

µm and 20 µm, respectively.

42

Monitoring different hemoglobin concentrations inside the phantom.

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

0,8

0,9

1

Reference Saline 16.25 [gr/L] 32.5 [gr/L] 65 [gr/L] 130 [gr/L]

No

rma

lize

d m

ean

va

lue

Type of solution

Monitoring Hemoglobin Concentration

Experimental results with phantom

43

Imaging along a blood vein

of a chicken wing. The solid

arrows indicate the blood

vein, while the dashed arrows

as well as the labeling letter

indicate the cascading point

between the images for

constructing an image with a

larger field of view

In-vivo experimental results

44

Imaging of blood vessel inside the rat’s brain using the micro endoscope


45


46

Outline



•Introduction





•IOP monitoring




•Micro endoscope

•Conclusions

47

Conclusions:• A new technology for accurate remote and continuous sensing of

movements was developed.

• The technique is based upon processing of back reflected secondary

speckles statistics.

• We demonstrated remote estimation of breathing, heart beating, blood

pulse pressure, alcohol and glucose concentration in the blood stream,

intra-ocular pressure measurement, oximetry, coagulation of blood etc.

• To extract precise absolute value for the measured biomedical

parameters periodic personalized calibration is needed every 2-3 years.

• Ultra thin and multi-functional micro endoscope for minimally invasive

medical treatment and diagnostics was presented

zeev zalevsky for knowledge stream

Education