0ô à8Ÿ 0Łdt 3 äÉ àt ł › uploads › 2 › 5 › 6 › 7 › 25679421 › 22... ·...

$: 0ô à8Ÿ 0ŁDt 3 äÉ àt ł › uploads › 2 › 5 › 6 › 7 › 25679421 › 22... · 2019-12-07 · ˛‹ Y|8 Master’s Thesis 0ô à8Ÿ‹ 0ŁDt'\3(—äÉ àt`˜ ł‰ Array$
석사 학위논문

Master’s Thesis

배열신호처리기법을이용한 3차원다색

초고해상도형광현미경

Array Signal Processing Based 3D Multi-color

Super Resolution Fluorescence Microscopy

민 준 홍 (閔準泓 Min, Jun Hong)

바이오및뇌공학과

Department of Bio and Brain Engineering

KAIST

2012



Advisor : Professor Ye, Jong Chul

by

Min, Jun Hong

Department of Bio and Brain Engineering

KAIST

A thesis submitted to the faculty of KAIST in partial fulfillment of

the requirements for the degree of Master of Science in the Department of

Bio and Brain Engineering . The study was conducted in accordance with

Code of Research Ethics1.

2011. 12. 22.

Approved by

Professor Ye, Jong Chul

[Advisor]

1Declaration of Ethical Conduct in Research: I, as a graduate student of KAIST, hereby declare that

I have not committed any acts that may damage the credibility of my research. These include, but are

not limited to: falsification, thesis written by someone else, distortion of research findings or plagiarism.

I affirm that my thesis contains honest conclusions based on my own careful research under the guidance

of my thesis advisor.



민 준 홍

위 논문은 한국과학기술원 석사학위논문으로

학위논문심사위원회에서 심사 통과하였음.

2011년 12월 22일

심사위원장 예 종 철 (인)

심사위원 정 기 훈 (인)

심사위원 박 지 호 (인)

MBIS

20103234

민준홍. Min, Jun Hong. Array Signal Processing Based 3D Multi-color Super Resolution

Fluorescence Microscopy. 배열신호처리기법을이용한 3차원다색 초고해상도형광

현미경. Department of Bio and Brain Engineering . 2012. 51p. Advisor Prof. Ye, Jong

Chul. Text in English.

ABSTRACT

Thanks to the extensive researches on fluorescence probe including the invention of immunofluores-

cence and discovery of green fluoresence protein, fluorescence microscopy has been a popular instrument

tool to understand properties organic and inorganic specimens. However, due to the diffraction limit,

conventional fluorescence microscopy cannot discern details of specimen that are closer than half of

wavelength of emission light. Recently, far-field fluorescence microscopy to overcome the diffraction limit

has been extensively studied. For example, stimulated emission depletion microscopy (STED) exploits

stimulated emission phenomenon of fluorescence material to design sharp effective point spread function

(PSF), by suppressing the rim of excitation spot. Saturated structure illumination microscopy (SSIM)

can unlock the diffraction limit by high power parallel line pattern illumination which drops high order

frequency components onto range of optical transfer function (OTF) as the similar way of structured

illumination microscopy (SIM). In single molecule based approach such as STORM and PALM/FPALM,

photoswitchable fluorescence molecule is used for super resolution localization. However, all these super

resolution techniques have their own limit such as requirement of high power laser and massive hard-

ware setup or specific photoswitchable dye. Here, we propose a novel 3D multi-color super resolution

microscopy, called nanometer resolution imaging method using speckle illumination and multiple signal

classification (Nano-MUSIC). The novelty of Nano-MUSIC is that it can achieve super resolution image

by using array signal processing technique and speckle illumination, by converting super resolution image

to multiple source localization problem. Using Nano-MUSIC with focus stabilization, we demonstrate

the multi-color 3D imaging with biological sample such as microtubule, mitochondria, F-actin to discover

the sub-diffraction-scale details of cellular structures.

Keywords: Super Resolution, 3D Imaging, Multi-color Imaging, Speckle, Array Signal Processing, Focus Stabi-

lization

i

Contents

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i

Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Chapter 1. Introduction 1

Chapter 2. Far-field Nanoscopy 3

2.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.2 The Diffraction Limit . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.3 Advanced Microscopy with Diffraction-Limited Resolution . . . 5

2.3.1 Confocal and two photon microscopy . . . . . . . . . . . 5

2.3.2 4pi Microscopy and I5M . . . . . . . . . . . . . . . . . . . 6

2.3.3 Near-field imaging . . . . . . . . . . . . . . . . . . . . . . . 6

2.4 Far-field Nanoscopy . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.4.1 Patterned Excitation Based Approach . . . . . . . . . . . 8

2.4.2 Single Molecule Imaging Based Approach . . . . . . . . . 12

2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Chapter 3. 3D Multi-color Nano-MUSIC 15

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2 Speckle Illumination . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.3 Theory of Nano-MUSIC . . . . . . . . . . . . . . . . . . . . . . . . 17

3.4 Extension of 3D and Multi-color Imaging . . . . . . . . . . . . . 20

3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Chapter 4. Focus Stabilization Method 23

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.2 Implementation and Experimental Results . . . . . . . . . . . . 24

4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Chapter 5. Experimental Results of 3D Multi-color Nano-MUSIC 29

5.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5.2 Cell Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5.2.1 Reagents and Contructs . . . . . . . . . . . . . . . . . . . 31

ii

5.2.2 Immunofluorescences . . . . . . . . . . . . . . . . . . . . . 31

5.3 Two-color Imaging Results . . . . . . . . . . . . . . . . . . . . . . 31

5.3.1 Microtubule and F-actin in Hela cell . . . . . . . . . . . . 31

5.3.2 Mitochondria and F-actin in Hela cell . . . . . . . . . . . 32

5.4 3D Imaging Results . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Chapter 6. Conclusion 43

6.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

6.2 Future Research directions . . . . . . . . . . . . . . . . . . . . . . 44

References 46

Summary (in Korean) 52

– iii –

List of Tables

6.1 Super resolution Fluorescence Microscopy . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

iv

List of Figures

2.1 (a) Point Spread Function (PSF). (b) Optical Ttransfer Function (OTF). . . . . . . . . . 5

2.2 Confocal and 4pi microscope (a) Confocal: using the spatial pinhole increase resolution

by√

2. (b) 4pi: two opposite objectives substantially improve axial resolution. . . . . . . 6

2.3 Near-field Scanning Optical Microscopy (NSOM). (a) NSOM with scanning tip of illim-

ination and detection. (b) NSOM with scanning tip and objective lens for illumination

and detection repectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.4 The process of stimulated emission. A fluorophore is pumped to excitation state from

ground state. When excited fluorophore encounters photon having matched energy be-

tween ground and excitation state, stimulation emission is generated otherwise sponta-

neous emission happens. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.5 The implementation of STED microscopy and principle of nanoscale effective PSF. The

doughnut-shaped STED beam (red) deexcites the flurorecent molecules at the rim of exci-

tation PSF (green). The excited fluorescence molecules emit the fluorecent light (orange)

within nanoscale. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.6 Saturated Structured Illumination Microscopy (SSIM). (a) The generation of illumination

pattern. A diffractive grating in the excitation path splits the light into two beams.

Their interference after emerging from the objective and reaching the sample creates a

sinusoidal illumination pattern. (b) The observable region of a spatial frequency domain

by conventional microscopy. (c) The illumination pattern for spatial modulation of (b). (d)

the extended spatial frequency domain of (b) by spatial modulation. (e) The observable

regions after total imaging procedure is finished. . . . . . . . . . . . . . . . . . . . . . . . 12

2.7 PhotoActivation Localization Microscopy (PALM) and STochastically Optical Recon-

struction Microscopy (STORM). (a) PALM/STORM reconstruct a super resolution image

by marking different fluorescent probes by centroid calculation of each probe, when probes

are activated at different time points. (b) For the given target sample, STORM achieved

a super resolution image by localizing activated sparse subset of probes for total imaging

cycles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

v

3.1 Comparison between STORM/PALM and Nano-MUSIC for Fluorophores activation. In

single molecule approach such as STORM/PALM, fluorophores within diffraction limit

sized zone are excited one by one. In Nano-MUSIC, all fluorophores within diffraction

limit sized zone are randomly excited. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.2 Speckle pattern from coherent laser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.3 Extension to 3D imaging scheme in Nano-MUSIC. PSF of each fluorophore on image plane

is changed, which is depends on the level of defocus in axial direction. . . . . . . . . . . . 21

4.1 Implementation of focus stabilization method. (a) Scheme of the implementation. A

cylindrical lens is inserted within reference beam path. (b) Reference beam is focus on the

bottom surface of cover glass while illumination light is focused on sample. The spot of

reference beam is imaged on the reference detector, changing the shape of spot correspond-

ing to the displacement of image focal plane. The PZT of objective get feedback control

based on change of PSF. (c) The shape of reference spot changes along axial defocus. (d)

The intensity sum of spot image in one of quadrant is approximately proportional to the

displacement of defocus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.2 Experimental results of focus stabilization for ten minutes. (a) Without external noise

case. (b) With external noise such as rotatoring diffuser vibration or electrical inference

of system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

5.1 The implementation of Nano-MUSIC based on efi-fluorescence microscopy. The light from

two lasers is combined and scattered though the rotatable diffuser. Then speckle pattern

from the diffuser is reflected by dichroic mirror and then illuminates the sample through

100x, 1.30NA, oil immersion typed objective lens (UIS2 Olympus). The emission light from

the sample is imaged on EMCCD (Luca-S, Andor). The focal plane of the objective lens

is stabilized by the proposed method implemented by PZT, objective stage and reference

laser (HeNe or IR laser) with customized program. . . . . . . . . . . . . . . . . . . . . . 30

5.2 The two-color image of microtubule and F-actin. Alexa488(green) and TRITC(red) were

used for microtubule and F-actin, respectively. Raw images are acquired by 100x mag-

nification. (a) Conventional fluorescence image, created from temporal mean of 1000

snapshots for each color. (b) Nano-MUSIC image. (c,d) Closed-up images from line boxes

in (a,b). (e) Line profile of line in (c). (f) Line profile of dotted line in (d). Scale-bars are

5µm in (a,b) and 500nm in (c,d), respectively. . . . . . . . . . . . . . . . . . . . . . . . . 33

– vi –



nification.(a) Overlapped image from conventional and Nano-MUSIC (b) Conventional

fluorescence image, created from temporal mean of 1000 snapshots for each color. (c)

Nano-MUSIC image. (d,e) Closed-up images from dotted line boxes in (b,c). Scale bars

are 5µm in (b,c) and 500nm in (d,e) respectively. . . . . . . . . . . . . . . . . . . . . . . 34




snapshots for each color. (b) Nano-MUSIC image. (c,e) Closed-up images from line boxes

in (a). (d,f) Closed-up images from line boxes in (b). Scale-bars are 5µm in (a,b) and

1µm in (c,d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35





in (a,b),respectively. Scale-bars are 5µm in (a,b) and 500nm in (c,d) . . . . . . . . . . . . 35

5.6 The two-color image of mitochondria and F-actin. Alexa488(green) and TRITC(red)

were used for mitochondria and F-actin, respectively. Raw images are acquired by 100x

magnification. (a) Conventional fluorescence image, created from temporal mean of 1000

snapshots for each color. (b) Nano-MUSIC image. (c,e) Closed-up images from line boxes


500nm in (c-f) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36





in (a,b). (e,f) Closed-up images from dotted line boxes in (a,b). (g) line profile from yellow

lines in (e,f). Scale-bars are 5µm in (a,b) and 500nm in (c-f) respectively. . . . . . . . . . 37

– vii –




snapshots for each color. (b) Nano-MUSIC image.(c,e) Closed-up images from line boxes


500nm in (c,d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.9 Calibration of PSF 60nm fluorescence bead in axial direction. (a) In the graph, measured

width of PSF in x and y direction is changed corresponding to the axial position. (b) 2d

PSF shape of single particle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39


used for microtubule and F-actin, respectively. Raw images are acquired by 160x magnifi-

cation. (a) Temporal mean image from 1000 snapshots for each color. (b-d) Nano-MUSIC

images from +250nm, 0nm, -250nm image planes,respectively. (e,h) Closed-up images

from line boxes in (b). (f,i) Closed-up images from line boxes in (c). (g,j) Closed-up

images from line boxes in (d). (k) 3D rendered image by AMIRA. Scale-bars are 2µm in

(a-d) and 500nm in (e-j) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41



magnification. (a) Temporal mean image from 1000 snapshots for each color. (b) Nano-

MUSIC images from +250nm, 0nm, -250nm image planes,respectively. (e-g) Closed-up

images from line boxes in (b-d) respectively. (h) 3D rendered image by AMIRA. Scale-bars

are 1µm in (a-d) and 500nm in (e-g). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

– viii –

Chapter 1. Introduction

Thanks to the extensive researches on fluorescence probes including the invention of immunofluo-

rescence [1] and discovery of green fluorescence protein[2], fluorescence microscopy has been a popular

instrument tool to understand properties of organic and inorganic specimens. For biologisist, labeling

fluorescence protein or molecule to specific targets of interest help them to achieve meaningful infor-

mation such as functional inter-cellular interaction as well as its morphology. In order to comprehend

biological phenomena in more detail, biologists are interested in imaging nano-scale resolution. However,

according to the Abbe’s criterion[3], conventional fluorescence microscopy cannot discern the details of

specimen which are closer together than half of wavelength of emission light. For example, the resolvable

distance in optical system is about 200-300nm in lateral and 500-700nm in axial direction that is quite

larger than subcellular structures. In order to overcome the limit of spatial resolution in fluorescence

microscope, several methods had been developed with improved spatial resolution in small degree. For

example, confocal microscopy[4] can accomplish increased resolution in factor of√

2 using spatial pinhole

which is role of squaring of point spread function.The 4pi[5] and I5M[6] microscopy have achieved sub-

stantial axial resolution improvement by using two opposite objective lens for illumination and detection

in order to increase the numerical aperture. In addition, other research direction is to exploit the near-

field evanescence wave to reduce the wavelength. For example, near-field scanning optical microscopy

(NSOM) can achieve 20-50nm resolution using sharp scanning tip apex across the sample[7]. However,

the application of near-field imaging is constrained on near surface imaging of sample. For the general

purpose, many researchers hope to develop a far-field microscopy with super resolution capability.

Recently, some far-field fluorescence microscopy to achieve sub diffraction limited resolution have been

demonstrated such as STED[8], RESOLFT[9], SSIM[10] and single molecule imaging based approach such

as PALM/FPALM[11, 12] and STORM[13]. All these methods exploit nonlinear optical phenomena. For

example, stimulated emission depletion (STED) microscopy exploits stimulated emission phenomenon of

fluorescence material in order to design sharp effective PSF, by suppressing the rim of excitation spot.

However, it needs high power STED laser aligned with illumination laser in order to quench the periph-

ery of excitation spot, outperforming spontaneous emission. STED has been demonstrated to achieve

20-30nm in later[14] and 30-60nm in axial resolution, by combining with 4pi[15]. Moreover, multi-color

imaging[16] and live imaging in a small region[17] have been demonstrated. Since high power STED laser

which cause cell damage, limited choice of fluorescence dyes are available and massive hardware setup

often prevents STED from being widely used for general purpose. Saturated structured illumination

– 1 –

microscopy (SSIM)[10] is another super-resolution method based on saturable process of fluorescence

molecule. It is extended version of Structured illumination microscopy (SIM)[18] whose theoretical reso-

lution is limited by factor of 2. SSIM can unlock the diffraction limit by high power parallel line pattern

illumination which drops high order frequency components onto range of OTF as the similar way of SIM.

In single molecule based approach[11, 12], super resolution can be achieved by using photoswitchable

fluorescence molecule. Here, only one fluorophore should be excited at a time within diffraction limit

sized area. These methods have demonstrated two color 3D imaging with 20nm in lateral and 50nm in

axial resolution using cyanine dyes and FPs [13, 11, 19, 20]. Furthermore, live cell imaging has been

demonstrated for small fraction of whole sample[21]. However, these methods are relatively slow for wide

field imaging and have limited choice of fluorophores. Thus, new type of super resolution technique is

needed to overcome the limitation of previous super resolution techniques.

In this thesis, we proposed a novel 3D multi-color super resolution microscopy, called nanometer reso-

lution imaging method using speckle illumination and multiple signal classification (Nano-MUSIC). The

novelty of Nano-MUSIC is that it can achieve super resolution image by using array signal processing

technique that has been widely developed in radar, sonar, wireless communications, by considering super

resolution image as multiple source localization problem. Nano-MUSIC can accomplish super resolution

without requiring photo-switchable fluorophores, and massive complicated hardware implementation un-

like the other super resolution methods. However, when it comes to multi-color imaging and the other

applications, Nano-MUSIC should be supported by focus stabilization techniques in order to maintain

the focal plane for few minutes which is generally required for imaging in Nano-MUSIC. Thus, we propose

a new focus stabilization method by simply adding additional laser, common commercial CCD detector

and piezoelectric transducer objective stage. By using it, Nano-MUSIC is guaranteed system to keep up

the focal plane within 30nm, which has been demonstrated experimentally.

In the following, we first review previous super resolution techniques in detail. Then, we discuss the

details of Nano-MUSIC where we explain the characteristic of speckle illumination and MUSIC algo-

rithm relating to its theoretical performance of resolution and practical limiting factors. Furthermore,

we discuss how to extend Nano-MUSIC to multi-color and 3D imaging. After that, new type of focus

stabilization method is proposed and its accuracy is demonstrated by experiment. Next, we illustrate

Nano-MUSIC for imaging biological samples such as microtubule, mitochondria, F-actin using two-color

and 3D imaging. Finally, summary and discussion in current limitation of the proposed Nano-MUSIC

will be discussed.

– 2 –

Chapter 2. Far-field Nanoscopy

2.1 Background

Fluorescence microscope has been an essential tool to study properties of organic or inorganic spec-

imens [22]. It is based on the phenomenon, instead of reflection or absorption, that fluorescent material

emits light after it absorbs specific wavelength light, which has shorter wavelength than that of emission

light. By labeling specific targets with fluorescent probes, we can acquire target-specific image which

gives meaningful information such as cellular interaction or morphology. However, according to the

Abbe’s criterion[3], conventional fluorescence microscopy cannot discern the details of sample that are

closer together than half of wavelength of emission light. In other words, the resolution of optical instru-

ment is limited by diffraction limit. This diffraction limit is about 200-300nm in lateral and 500-700nm

in axial direction that is quite larger than subcellular structures. Thus, conventional fluorescence micro-

scope is not able to observe the details of the subcellular structure. During the last decades, there have

been extensive efforts to break the diffraction limit for nano-scale resolution. Even if some techniques

had been developed to improve the resolution, such as confocal and 4pi microscopy [4, 5], they are still

not sufficient to observe sub-diffraction structures.

Currently, there are few nanoscopy technique to achieve nano-scale resolution, by using patterned exci-

tation techniques [8, 9, 18, 10] and single molecule imaging techniques[13, 11, 12] with photo-switchable

fluorescent probes. Basically, all these methods take advantage of optical nonlinearity. For exam-

ple, stimulated emission depletion(STED) microscopy[8, 23] uses the nonlinearity to sharpen the point

spread function (PSF) and it can be generalized with special dyes such as reversible saturable optically

linear fluorescence transitions (RESOLFTs). Structured illumination microscopy (SIM)[18, 24], and

saturated structured illumination microscopy (SSIM)[10] exploit the synthetic aperture in radar princi-

ple. In the single molecule imaging techniques, there are stochastic optical reconstruction microscopy

(STORM)[13], photoactivated localization microscopy (PALM)[11], and fluorescence photoactivated lo-

calization microscopy (FPALM)[12]. All the above mentioned methods can observe unresolved details

of sub-cellular structures although each method has different pros and cons relating to resolution (lat-

eral and axial), multi-color, 3D imaging, and live imaging. In this chapter, first of all, we will explain

diffraction limit and introduce advanced microscopy with diffraction limited resolution such as confocal

microscopy and 4pi. Then, current far-field super resolution fluorescence microscopy will reviewed in two

categories: patterned excitation based methods and single molecule imaging based methods.

– 3 –

2.2 The Diffraction Limit

An optical microscope has the limita in resolution to distinguish the two adjacent objects. In other

words, lights from a single point cannot converge into a perfect point in image domain but a blurred

spot surrounded by a series of diffraction rings, referred to as Airy disk because of the diffraction of the

light (Fig 2.1(a)).

The bright central point corresponds to as zeroth-order diffraction spot while the surrounded rings

are referred to as the first, second, etc., order diffraction rings. The size of the Airy disk depends on

the wavelength of light and the numerical aperture of lens. The numerical aperture (NA = n sin(θ)) is

multiplication refractive index of imaging medium (n; air(1), water(1.33), immersion oil(1.515)) and sine

of half aperture angle of objective lens. Then, Airy disk in axial dimension forms an elongated geometry

because of non-symmetrical wave front from objective lens. The three dimensional light intensity distri-

bution of the image from a point source such as single fluorophore is referred to as point spread function

(PSF). When it comes to the resolution, the radius of Airy disk in the lateral and axial dimension is

defined by German physicist Ernst Abbe in 1873 [3].

Rxy,Abbe =λ

2NA, (2.1)

Rz,Abbe =2λ

NA2, (2.2)

For example, when imaging visible light (λ = 550nm) and 100x magnification objective lens with NA =

0.9, the image spot size in image domain is about 30 µm. While, if imaging with general oil immersion

objective lens with NA = 1.4, the image spot size in image domain is approximately 20 µm which is 50%

smaller spot size. Thus, conventional method to minimize the size of PSF, is to decrease wavelength

of imaging light or increase numerical aperture ((NA) with imaging medium having high refractive

index. However, even in ideal conditions, lateral resolution is still limited to about 200nm due to

physical constraints of wavelength below 400nm and numerical aperture of objective lens. Moreover,

axial resolution is even 2 or 3 times worse than lateral resolution on the order of 500nm. Therefore,

when we image thick biological sample, the image is severely affected by poor axial resolution due to

out of focus light interference. Practically,spatial resolution is often defined as the smallest separation

distance between two small objects. A most commonly used resolution criterion (or full width half

maximum FWHM) is defined by Rayleigh[25]

Rz,Rayleigh =0.61λ

NA, (2.3)

Rz,Rayleigh =2nλ

NA2, (2.4)

– 4 –

If the distance between the two point like objects is greater than the FWHM, they can be regarded

as being resolved. The image domain resolution can be interpreted in frequency domain. In optical

system, image in detector can be described by convolution between sample and lens PSF (I d = I s *

PSF) In reciprocal space, it is expressed as multiplication between Fourier transform of each sample and

OTF (I d = I s×OTF) where optical transfer function(OTF) is the Fourier transform of point spread

function(PSF). As the OTF is band limited (Fig 2.1 (b)), image in detector loses the high frequency

spectral components of sample by OTF. Since the loss of high frequency information cannot be recovered,

extending bandwidth of OTF is one of the main interest for microscopy.

frequency

Abbe diffraction

limit frequency

a b

Figure 2.1: (a) Point Spread Function (PSF). (b) Optical Ttransfer Function (OTF).

2.3 Advanced Microscopy with Diffraction-Limited Resolution

2.3.1 Confocal and two photon microscopy

Confocal microscopy is one of the widely used optical instrument for life sciences, semiconductor

inspection and materials science[4]. The principle of the confocal microscopy is to use point illumination

and a spatial pin-hole to eliminate the out of focus light in specimens that is primary reason for degrading

image. By using the point illumination and pin-hole, its effective point spread function is square of PSF

in conventional wide-field microscopy. The squaring of PSF can theoretically guarantee the factor of√

2

improvement in spatial resolution. Two-photon microscopy[26] is another such microscopy that utilize

the squaring effect of PSF. Here, it needs light having 2 times longer wavelength to excite fluorophore

from ground state to fluorescent state. The confocal and two-photon microscopy is generally used for

the purpose for out of focus fluorescent signal reduction, allowing 3D image of thick biological sample.

– 5 –

objective

objective

objective

Detector

a b

pin-hole

Figure 2.2: Confocal and 4pi microscope (a) Confocal: using the spatial pinhole increase resolution by√

2. (b) 4pi: two opposite objectives substantially improve axial resolution.

2.3.2 4pi Microscopy and I5M

Conventional microscope (Fig. 2.2 (a)) objective cannot utilize to spherical wave front but elliptical

wave front because it is used in only one side of the sample. This elliptical wave front leads to elongated

point spread function in axial direction. If the spherical wave front is measured by optical system, the

axial spatial resolution should be the same as lateral resolution. In order to measure spherical wave

front of objective lens, 4pi (Fig. 2.2 (b)) and I5M use two opposite objectives for both illumination and

detection both [5, 6]. They demonstrated 3-7 times axial resolution improvement and modest enhanced

lateral resolution, by increasing numerical aperture of objective.

2.3.3 Near-field imaging

Since loss of high frequency information results from light propagation through lens system (far

field) over larger distance than wavelength of light, near field microscopy is the one of the approaches to

break the diffraction limit to observe nanometer scale object. By detecting the signal from fluorophores

within shorter distance than excitation light, high resolution information can be preserved. Near-field

microscopy is based on the facts that evanescence waves decay exponentially from the intermediate sur-

face. Near-field scanning optical microscopy (NSOM figure 2.3) can achieve 20-50nm resolution using

sharp scanning tip apex across the sample[7]. Furthermore, superlens based wide-filed imaging has also

been demonstrated recently[27]. However, nature of near-field imaging constrain application of near-field

imaging to near surface imaging of sample and cautious tip control should be treated. These limitations

– 6 –

demand to develop far-field microscopy with sub-diffraction limited resolution.

NSOM

(illumination)

b

h<<λ

Scanning tip

Glass

NSOM

(illumination & collection)

a

h<<λ

Scanning tip

Glass

a<<λ

a<<λ

Objective Lens

Figure 2.3: Near-field Scanning Optical Microscopy (NSOM). (a) NSOM with scanning tip of illimination

and detection. (b) NSOM with scanning tip and objective lens for illumination and detection repectively.

2.4 Far-field Nanoscopy

Recently, few far-field fluorescence microscopy techniques that have nano-scale spatial resolution

have been demonstrated by exploiting different optical nonlinear phenomenon. They can be categorized

to patterned excitation based approach such as STED[8], RESOLFT[9], SSIM[10] and single molecule

imaging based approach such as PALM/FPALM[11, 12] and STORM[13]. Since fluorescence microscopy

has been the most popular tool to understand not only small structure in sample but also its functional

mechanism by labeling fluorescent molecules to specific target in the sample, far-field super resolution

microscopy method should be design to cover 3D imaging, multi-color imaging, live cell imaging, and

its biological application. The patterned illumination based approach exploits to improve the spatial

resolution by engineering effective PSF while single molecule imaging approach improves the resolution

by determining a position of single fluorescent molecule.

The super resolution techniques require 3D imaging because most cellular structures are in 3D. In

addition, to investigate complex cellular morphology or biological process such as functional interaction

between different structures, multi-color imaging is necessary. Moreover, live cell imaging is required as

important field of optical microscopy application. The realization of live cell imaging in super-resolution

microscopy needs high time resolution to figure out cell dynamics. For each super resolution microscopy

– 7 –

technique, we will explain its concept and discuss about above mentioned aspects such as 3D imaging,

multi-color imaging, and so on.

2.4.1 Patterned Excitation Based Approach

The patterned excitation based super resolution techniques such as STED, RESOLFT, can achieve

sub diffraction spatial resolution, exploiting nonlinear effect of fluorophores.

STED/RESOLFT

Stimulated emission depletion (STED) microscopy is first introduced super resolution techniques in

1994[8] as advanced scanning fluorescence microscope. In conventional scanning fluorescence microscopy,

after fluorophores are excited by point illumination sequentially, then detector collects the emission light

from the excited fluorophores, which results in diffraction limited spatial resolution. The concept of

STED microscopy is that it subsequently uses another laser (STED laser) to deplete fluorescence emis-

sion in peripheral area of excited fluorophores. It makes only small fluorophores in center of excitation

emit the light by shaping the PSF using STED laser. This suppression process can be understood by

spontaneous and stimulated fluorescent emission (Fig. 2.4). When electron of fluorescence molecule

absorbs the energy from the illumination light, it moves in excited state from ground state. After very

short time constant, electron naturally goes back to ground state, emitting light corresponding energy

gap between excited state and ground state, called spontaneous emission. When the excited electron

encounters the photon with the same energy gap, electron transition from excited state to ground state

occur immediately, emitting photon with same characteristic of encountered photon, referred to as stim-

ulated emission. In order to realize the concept, STED laser beam has doughnut-shaped pattern having

zero intensity at the center beam.The doughnut shaped pattern of STED beam is generally generated

by specific phase mask in illumination path of microscope (Fig. 2.5). Although STED laser beam also

has diffraction limited pattern, STED can achieve the super resolution image by benefit of nonlinear

effect of depletion. If the intensity of STED beam is higher than a certain level, the spontaneous emis-

sion is depleted at the periphery of excitation. Here, the certain level of intensity can be described as

Isat ≡ (στAB)−1 where σ denotes the optical cross-section of transition and τAB is spontaneous life time

of the state A and B. The high power STED laser expands depletion region except spontaneous emission

at the center point of excitation is not affected. Accordingly, the spontaneous emission outbreaks at

center area of excitation which makes the effective point spread function sharper.

The RESOLFT[9] which stands for reversible saturable optically linear fluoresce transition is the gener-

alized version of STED microscopy. It makes use of reversibly photoswitchable fluorescence probe having

on-state and off-state. The off-state is interpreted as dark state in STED or the triplet state in ground

– 8 –

Excit

ati

on

Sti

mu

late

d

Sp

on

tan

eo

us

Figure 2.4: The process of stimulated emission. A fluorophore is pumped to excitation state from

ground state. When excited fluorophore encounters photon having matched energy between ground and

excitation state, stimulation emission is generated otherwise spontaneous emission happens.

state depletion microscopy (GSD)[28]. Th resolution can be expressed as follow:

λ

2NA√

1 + Imax/Isat, (2.5)

where Imax is the maximum intensity of de-excitation beam and above defined Isat is saturation intensity

for fluorophore. STED microscopy needs high power STED laser,bringing about damage on biological

sample, because of doughnut shaped depletion pattern such as 10MW/cm2 with σ = 10−16cm2 and

τAB = 10−9s while RESOLFT allows much lower depletion laser power [9]. For example, RESOLT has

been demonstrated 100nm resolution with a depletion laser intensity 600W/cm2 using photoswitchable

fluorescent protein as FP593.

STED has been demonstrated to achieve 20-30nm lateral resolution [14] and 100nm axial resolution[15]

with single objective lens and axial phase masking. If STED microscopy is combined with 4pi, its

axial resolution can accomplish 33-60nm resolution[29], providing super resolution 3D imaging. For

example, hollow shape of the outer membrane of mitochondria can be observed [30]. Moreover, multi-

color imaging[16] can be possible in a way that two pairs of excitation and STED lasers can be used for

matched fluorophores such as Atto532 whose excitation at 488nm and STED at 603nm and Atto647N

whose excitation at 635nm and STED at 750-780nm. The available number of color practically is limited

2 because each color imaging needs quite broad band spectrum for excitation and STED in visible light

range. When it comes to live cell imaging, it has been demonstrated within limited few µm field of view

and decreased scaning speed becuase STED needs more scanning time, having more sharper PSF than

confocal microscopy. STED demonstrated live imaging of hippocampal neurons and snaptic vesicles at

28 fps with 60-80nm resolution[17].

– 9 –

100X

1.40 Oil

Excitation STED Eff. PSF

~20nm

y

x

z

x

Δt

Excitation 558nm

STED 766nm

λ/2 Phaseplate

Avalanche

Photodiode

Fluorescence

645-715nm

Scanning

Stage

Figure 2.5: The implementation of STED microscopy and principle of nanoscale effective PSF. The

doughnut-shaped STED beam (red) deexcites the flurorecent molecules at the rim of excitation PSF

(green). The excited fluorescence molecules emit the fluorecent light (orange) within nanoscale.

SIM/SSIM

Structured illumination microscopy (SIM)[18] is one of the advanced microscopy for general purposes

such as 3D imaging, multi-color imaging, live cell imaging. SIM uses periodic line pattern illumination

and frequency mixing with multiple images taken from different illumination patterns in order to in-

crease the resolution. The periodic pattern is generated by interferometry of illumination in lateral and

axial both[31, 18, 24]. When structure line pattern illumination is applied, high frequency components

of the sample are shifted to lower frequency band because 2D Fourier transform of line pattern is two

opposing delta functions, placed abbe diffraction limit frequency, corresponding angle and period of line.

As Fourier transform of image on detector can be described as convolution between illumination and

sample in frequency domain, the delta function acts like function to shift frequency components of sam-

ple to band limited lower frequency band. With frequency mixing of multiple images from patterned

illumination with different angle of line, high frequency components can be restored. In other words,

– 10 –

reconstructed image has broadened OTF, leading spatial resolution improvement. SIM has demonstrated

the spatial resolution improvement lateral and axial both[18, 24]. Moreover, recently 3D live cell imaging

covering wide field of view within a minute was reported[32]. Besides, multi-color 3D imaging is available

without requirements of fluorescence dye[33]. However, its resolution is theoretically limited to double

enhancement in axial and lateral both, about 100nm in lateral, and 300nm in axial. In addition, SIM

relatively needs high intensity power (10MW/cm2) of illumination to form clear patterned illumination

which might cause damage to cell.

In order to break the double resolution improvement of SIM, nonlinear phenomenon of fluorescence

molecule with respect to saturable process is applied to SIM. Saturated structured illumination mi-

croscopy (SSIM) has super resolution based on saturation of fluorescent emission[10]. As increasing

power of excitation laser toward fluorophores, emission signal from fluorophores is no longer increased

proportionally to illumination power after certain level. Under the situation, fluorophores immediately

jump to excitation state right after coming back to ground state by spontaneous emission. Thus, under

high intensity of excitation laser, fluorescent intensity of fluorophores is saturated and amount of the

intensity is determined by fluorescence lifetime. In SSIM, sinusoidal line patterned illumination is gen-

erated by common path interferometry with diffraction grating or SLM (Fig. 2.6). As increasing the

sinusoidal patterned illumination power, excitation signal power is getting to be flat as if clipping the

illumination pattern above saturation level while fluorescent emission is still almost zero nearby valley of

sinusoidal pattern. It means that high order frequency components as form of delta function are applied

to the sample, bringing higher frequency components of the sample to optical OTF band while only first

order delta functions are applied in SIM. SSIM has demonstrated 50nm resolution in lateral dimension

because practical resolution is affected by ratio between intensity of valley and that of peak, which is

limiting factor of resolution like as in STED.

– 11 –

Excitation pattern

Saturated excitation pattern

ObjectiveDiffraction

grating

Excitation

kx

ky

Spatial frequency

kx

ky

Spatial frequency

kx

ky

Spatial frequency

cb

a

d e

Figure 2.6: Saturated Structured Illumination Microscopy (SSIM). (a) The generation of illumination

pattern. A diffractive grating in the excitation path splits the light into two beams. Their interference

after emerging from the objective and reaching the sample creates a sinusoidal illumination pattern. (b)

The observable region of a spatial frequency domain by conventional microscopy. (c) The illumination

pattern for spatial modulation of (b). (d) the extended spatial frequency domain of (b) by spatial

modulation. (e) The observable regions after total imaging procedure is finished.

2.4.2 Single Molecule Imaging Based Approach

In the fluorescence microscopy, we observe image composed of the fluorescent signal from fluo-

rophores labeled to specific molecule. If we consider the fluorophore as a point source, super resolution

fluorescent image can be obtained by spatial coordinates of the fluorophores. In single molecule imaging,

single fluorescent molecule can be localized with below few nanometer resolution, multiple photons emit-

ted from the fluorophore can be fitted to the centroid[34, 35]. For single molecule tracking, localization

precision is given approximately by

∆localization =∆√N

(2.6)

where ∆localization is localization accuracy, ∆ is the size of PSF and N is number of emitted photons.

Thus, when it comes to single molecule imaging, spatial resolution is not limited by the diffraction of

light. However, it is difficult to directly apply this concept general fluorescence imaging because multiple

fluorophores emits light at the same time.

– 12 –

PALM/STORM

Recently, three groups have developed the super resolution microscopy methods which is referred

to as STORM[13], PALM[11], FPALM[12] based on single molecule imaging technique about the same

time. In order to make use of single molecule imaging method toward general biological sample, they

used special fluorescent probes which have switchable characteristic between fluorescence-on state and off

(or dark) state. Until now, many photoswitchable probes are invented from organic dyes to fluorescence

proteins (FPs). In this approach, fluorescence molecules in diffraction limited zone are localized one

by one, activating these molecules in different time. The process of fluorescence molecule localization is

repeated within diffraction limited zones which should not be overlapped in parallel. In order to guarantee

that only one fluorophore is activated, they use two lasers which have different wavelength from imaging

wavelength range for activation and de-activation of fluorophores. First, all fluorophores are tuned off

by using high power de-activation laser and then, spare fluorescence molecules are activated randomly

by weak activation laser on TIRF microscopy (Fig 2.7 (a)). After the parallel localization process for

each imaging cycle is repeated until fluorophores are photo-bleached. Finally, all ensembles’ coordinates

are marked on the image which has sub diffraction limit resolution (Fig 2.7 (b)). The resolution of the

approach is related to number of emitted photons (Eq. 2.6) which varies from hundreds for FPs to

few thousands for Cy5 [13, 11, 19]. Experimentally, its resolution has been demonstrated about 20nm in

lateral [13, 11] and axial resolution is 50nm[20] using cylindrical lens which changes the PSF into elliptical

shape corresponding axial position of fluorophore. 3D imaging techniques has been demonstrated for

few micrometer thick sample such as microtubule and mitochondria[20, 36]. Moreover, with combination

of photoswitchable probes such as cyanine dyes[19, 36] and FPs[37, 38], two color imaging has been

demonstrate. However, spatial resolution for thick sample is affected by refractive index change causing

spherical aberration and generally multi-color imaging is not easy for single molecule imaging based

method because of its limited combination of photoswitchable fluorescent dyes. Thus, number of available

color on the approach is practically constrained for two color imaging. Furthermore, live cell imaging has

been demonstrated for small field of view of the whole sample[21, 39] because these method is relatively

slow, usually taking from few minutes for whole imaging to few hours.

2.5 Discussion

The optical microscopy has fundamentally limited spatial resolution because of the diffraction of

light. Some advanced microscopies such as confocal and 4pi microscopy can achieve mild resolution

improvement in lateral and significant resolution enhancement in axial dimension. However, it is not

enough to understand sub cellular structures and its dynamics. Recently, super resolution techniques

– 13 –

PALM/STORM

Centroid

Stochastical reconstruction

>λ/2n

a b

Target structure

Localizing activated subset of probes

Super-resolution image

N

Figure 2.7: PhotoActivation Localization Microscopy (PALM) and STochastically Optical Reconstruc-

tion Microscopy (STORM). (a) PALM/STORM reconstruct a super resolution image by marking dif-

ferent fluorescent probes by centroid calculation of each probe, when probes are activated at different

time points. (b) For the given target sample, STORM achieved a super resolution image by localizing

activated sparse subset of probes for total imaging cycles.

such as STED, SSIM, STORM and PALM/FPALM have been demonstrated to achieve sub diffraction

super resolution. Moreover, application to multi-color imaging, 3D imaging, live cell imaging have been

demonstrated. However, each super resolution methods has own limitation in order to fully understand

biological mechanism as well as structures. Specifically, all the methods are practically allowed only

for two-color imaging, 3D live imaging is possible at the expense of decreasing resolution as well as cell

damage by high power illumination. Therefore, we need a new type of super-resolution method that

overcomes the limitation of the previous super resolution microscopy methods.

– 14 –

Chapter 3. 3D Multi-color Nano-MUSIC

3.1 Introduction

In this chapter, we propose a novel new type of 3D multi-color super resolution microscopy called

nanometer resolution imaging method using speckle illumination and multiple signal classification (Nano-

MUSIC). Nano-MUSIC can achieve super resolution by exploiting speckle illumination and array signal

processing [40].Under the assumption that adjacent fluorophores are uncorrelated or weakly correlated,

they can be resolved by MUltiple Signal Classification (MUSIC) [41] from array signal processing al-

gorithm. In order to excite fluorophores randomly, we use speckle illumination using a rotatable diffuser.

In single molecule imaging based approach such as STORM[13]or PALM/FPALM[11, 12], super res-

olution image can be achieved by mapping coordinates of photoswitchable fluorophores whose center

location is calculated by Gaussian fitting. Thus, this approach is relatively slow method compared to

other approach such as STED[8] or SSIM [10] because it needs to localize all activated fluorophores one by

one as shown in Fig. 3.1. In addition, super-resolution optical fluctuation (SOFI) has been demonstrated

for sub diffraction spatial resolution directly using second-order characteristic of blinking quantum dots,

which is not applicable to other fluorophores. However, Nano-MUSIC can accomplish super resolution

imaging based on randomly activated without any photoswitchable dyes. One of the novelty of Nano-

MUSIC is to interpret the super resolution imaging as multiple source localization problem. Another

advantage of Nano-MUSIC is that there is no additional requirement of type of fluorophores. Thus,

multi-color imaging can be achieved by repeating single color imaging process for each color. Besides,

when it comes to 3D imaging, Nano-MUSIC can image few hundred nanometer thick 3D imaging with

multiple snapshots which are focused at single layer. In the rest of the chapter, first, we will briefly

discuss about the statistical properties of speckle illumination. Then, theory of Nano-MUSIC will be

explained and its extended Nano-MUSIC 3D imaging as well. Lastly, we will analyzed theoretical per-

formance with respect to Cramer-Rao lower bound.

3.2 Speckle Illumination

A speckle pattern is a random intensity pattern produced by the mutual interference of a set of

wavefronts[42]. In general, the speckle pattern is generated when coherent light is reflected of transmitted

from rough surface or turbid media, yielding construction and destruction of light randomly[43] as

– 15 –

t

t

:NanoMUSIC

:PALM/STORM

Be

low

Dif

fra

cti

on

Lim

it

Figure 3.1: Comparison between STORM/PALM and Nano-MUSIC for Fluorophores activation. In

single molecule approach such as STORM/PALM, fluorophores within diffraction limit sized zone are

excited one by one. In Nano-MUSIC, all fluorophores within diffraction limit sized zone are randomly

excited.

shown in Fig. 3.2. The speckle pattern of light illumination had been considered as subject to be

reduced [44, 45] in order for decreasing noise on image. However, recently speckle illumination has

been widely used in optical application. For example, time-varying speckle pattern illumination was

exploited for image sectioning of thick sample [46, 47, 48] because its statistical correlation difference

between in-focus and out-of-focus. Furthermore, it can be applied to cerebral blood flow imaging, not only

improving the spatial resolution based on second order statistic of speckle pattern[49], but also effectively

measuring velocity of blood flow by Doppler-effect[50]. Besides, the dynamic speckle illumination has

been demonstrated for denoising and improved sectioning within digital holography[51]. In Nano-MUSIC,

we make use of dynamic speckle pattern illumination in order to excited fluorescent dyes independently

and then super resolution image can be reconstructed by MUSIC algorithm with covariance matrix of

multiple snapshots. Thus, we should know statistics of speckle pattern.

Here, speckle field on sample plane is given by [43]

φs(θ; t) =

∫PSFill(x− α, y − β)e(α, β, t)dαdβ (3.1)

where θ = (x, y) is coordinate on sample plane and e(α, β; t) is speckle field distribution on rough surface

from which speckle patterned light is generated. Then, cross-correlation of speckle pattern is described

– 16 –

as[43]

Re(α, β, α‘, β‘) = e(α, β; t)e∗(α‘, β‘; t− τ)

= |eo|2exp[−σ2o(1− e−(r/ro)

2

)]

=

|eo|2exp(−σ2

o) r/ro →∞

|eo|2δ(∆α,∆β) r/ro → 0(3.2)

where r =√

∆x2 + ∆y2, ro which is radius of normalized surface correlation falls in to 1/e, and σ2o is

phase variance which is proportional to the height variance which is related to surface height (σ2o = 4π

λ σ2h).

If the speckle pattern is generated from the sufficiently rough surface, speckle pattern has negative

exponential characteristic. Then, concerning fully developed speckle which is generated from numerous

scatters, we have Re = (α, β;α′, β

′) = |eo|δ(α − α

′, β − β′

) [43].Therefore, cross-correlation of the fully

developed speckle is given by

Rφ(θn, θm) = < φs(θn; t), φ∗s(θm; t) >

=

∫ ∫PSFill(θn − θ)Re(θn, θ

′

m)PSF ∗ill(θm = θ′)dθdθ

′(3.3)

Therefore, the correlation matrix of signal (fluorophore emission light) is not a diagonal matrix because

emission light intensity of fluorophores is proportional to the speckle illumination whose correlation

matrix is also non-diagonal matrix.

Figure 3.2: Speckle pattern from coherent laser

3.3 Theory of Nano-MUSIC

Nano-MUSIC is based on MUSIC(Multiple Signal Classification)[41] which is one of the most

successful algorithm in sensor array processing. It has been widely used in not only DOA(Direction of

– 17 –

Arrival) but also source localization problem in many imaging system such as EEG[52]. It basically makes

full use of second order statistics of multiple source which are in random process. In Nano-MUSIC,

localization problem of fluorophores is exactly the same with previous interest of MUSIC algorithm. In

more detail, general fluorescence intensity measured at position θd on the image plane can be model by

Id(~θd) =

∫PSFd(~θd − θ)C(~θ)φs(θ)dθ (3.4)

where φs(θ) is speckle illumination intensity in sample and C(~θ) is fluorophore concentration. Denoting

S(~θ) = C(~θ, z)φs(θ) as activated fluorophore intensity, it can be more simplified to

Id(~θd) =

∫PSFd(~θd − θ)S(~θ)dθ (3.5)

For time-varying fluorescence intensity, multiple-time measurements on the CCD detector y(t) can be

described as discrete version of Eq (3.5) given by

y(t) = A(Θ)s(t) (3.6)

y(t) = [y1(t),y2(t), . . . ,yM t] ∈ RM×r

A(Θ) = [a(θ1),a(θ2), . . . ,a(θk)] ∈ RM×k

s(t) = [s1(t), s2(t), . . . , sk(t)]T ∈ Rk×r

where yi(t) is intensity on i-th pixel of CCD at time t , a(θi) is vectorized point spread function of

single fluorescence molecule at θi = (xi,yi, zi), and si(t) denotes time-varying fluorescence emission

corresponding to the θi. For the ideal case where there are noiseless r snapshots and k sources are not

correlated at all, we can rewrite the Eq(3.6) as matrix form:

Y = A(Θ)S (3.7)

where Y = [y(t1),y(t2), . . . ,y(tr)], and S = [s(t1), s(t2), . . . , s(tr)]. Assuming the number of snapshots

is greater and equal to the number of sources (r ≥ k),and A(Θ) and S are full column and row rank

respectively. In this case, rank(S) is k and range of Y and A(Θ) is the same (R(Y) = R(A(Θ)).

Based on the property, we will localize the each source position (θ). First, do the SVD(singular valuew

decomposition) to Y:

Y = [Us|Un]

σ1

. . .

σk

0

. . .

0

VH

– 18 –

where left singular vectors consist of signal subspace(Us) and noise subspace(Un) corresponding to non-

zero singular values and zero singular values respectively. Since noise subspace Un is orthocomplement

of the signal subspace Us ,it is obvious that R(Us) = R(A(Θ)) = R(Un)⊥

. Using the orthogonality

between Us and Un, we can find column of A(θ) corresponding to location of sources. As we plot

f(θ) =1

‖UnHa(θ)‖22

, (3.8)

, then if θ ∈ Θ , f(θ) goes to infinite(i.e ‖UnHa(θ)‖ = 0). In other words, we can get k significant peaks

corresponding sourece locations. In order to understand general noisy case, Eq (3.7) should be modified

as

Y = A(Θ)S + W (3.9)

where W = [w(1),w(2), . . . ,w(r)] is Gaussian noise. Then, we define the sample correaltion matrix:

R =1

MYYH =

1

M

r∑m=1

y(m)yH(m) (3.10)

Substituting Eq (3.9) for Y, we can get following form:

R = A(Θ)

(1

M

r∑m=1

s(m)sH(m)

)AH(Θ) + A(Θ)

1

M

r∑m=1

s(m)wH(m)

+1

M

r∑m=1

w(m)sH(m)AH(Θ) +1

M

r∑m=1

w(m)wH (3.11)

where s(m) and w(m) are second-order ergodic random process. If r→∞, sample correlation matrix R

converges to

R→∞ = R = A(Θ)RsAH(Θ) + σ2I (3.12)

Then we take EVD(eigen value decomposition) or SVD of R,

R = [Us|Un]

σ21 + σ2

. . .

σ2k + σ2

σ2

. . .

σ2

VH

Then, we can divide the signal subspace and noise subspace as long as eigenvalues of signal subspace

is quite lager than those of noise subspace. Then, we can plot the MUSIC spectrum by the same Eq

(3.8). As long as SNR is high and signal covariance matrix is diagonal, MUSIC estimator can achieve

the maximum likelihood estimator [53] which can approach ultimate performance limit of single target

estimation[54]. In more detail, variance of MUSIC estimation error for uncorrelated signal is derived as

– 19 –

[53, 40]

V ARMUSIC(θi) =1

2N · SNRi[1 +

[(AHA)−1]iiSNRi

]/h(θi) (3.13)

V ARCRB(θi) = (1

2N · SNRi)/h(θi) (3.14)

where P is diagonal matrix, SNRi = Pii/σ2 and h(θ) = dH(θ)[I −A(AHA)−1AH ]d(θ), d(θ) = da(θ)/dθ.

From the previous two equations, the efficiency of MUSIC estimator comparing CRB is described as

follow

V ARMUSIC

V ARCRB= 1 +

[(AhA)−1]iiSNRi

(3.15)

Thus, under the condition that diagonal matrix P has large value elements, the performance of MUSIC

algorithm can achieves the CRB. However, in Nano-MUSIC, signal covariace matrix is not the exact

diagonal matrix becuase speckle pattern illumination is correlated in the range of diffraction limit. This

correlation of illumination prevents Nano-MUSIC from achieving performance of CRB. Thus, in order to

increase the resolution, l2-norm spectrum (Eq (3.8)) can be modified following l∞-norm which generally

resovles the detail of sub cellular structures.

f(θ) =1

‖UnHa(θ)‖∞

(3.16)

As using l∞-norm instead of l2-norm, we can surpress the side lobe for non-target loaction because l∞-

norm satisfies inequality ‖UnHa(θ)‖∞ > ‖Un

Ha(θ)‖2 if θ is not the source position. Therefore, it brings

about enhanced spatial resolution and image contrast.

3.4 Extension of 3D and Multi-color Imaging

The extension of Nano-MUSIC into 3D multi-color image is straightforward. For two-color imaging,

two lasers with different wavelength are prepared to share same illumination path including rotatable

diffuser, and dichroic mirrors and emission and excitation filters matched to lasers and fluorescence dyes.

Then, we alternatively excite the fluorescence probes by switching each laser. However, multi-color Nano-

MUSIC should maintain focal plane during acquisition because acquisition time for multiple snapshots

is proportional to the number of colors. This is because thermal drift of imaging medium such as oil or

water generally changes focal plane about several hundred nanometers, which would change the shape of

PSF corresponding to fluorophores. Such drift can degrade achievable spatial resolution of Nano-MUSIC.

Thus, new type of focus stabilization technique should be implemented in order to compensate for the

focal plane drift, which will be presented in next chapter.

One of the possible extensions of 3D imaging in Nano-MUSIC can be realized by using three dimensional

particle tracking method which is essential task in nanofabrication and super resolution imaging. Main

– 20 –

idea of the method is to break the symmetry of PSF along axial direction. Specifically, the three

dimensional localization has been demonstrated by astigmatism imaging[55, 20] with cylindrical lens,

simultaneous multiplane imaging[56, 57] and double Helix microscopy[58, 59].Moreover, the performance

of 3D localization method has been analyzed with respect to Cramer-Rao lower bound [60]. For super

resolution imaging, 3D STROM[20] has been demonstrated that 50nm resolution in axial direction can

be achieved by astigmatism. The astigmatism imaging method is well suited for Nano-MUSIC in order to

extend 3D imaging because it can be realized simply inserting cylindrical lens right after CCD detector.

As shown in Fig. 3.3, we can figure out asymmetrical change of PSF corresponding to defocus. For

example, PSF of fluorophore which placed on focal plane has spherical shape which is the same shape of

PSF in conventional except little bit broaden width of PSF. However, PSF of fluorophore which placed

above or below focal plane looks elliptical shape or inclined Gaussian distribution whose tilting direction

is orthogonal to each other. Thus, we can estimate coordinates of fluorophores by modifying Eq. 3.9

where column of A(Θ) is the vectorized PSF corresponding to three dimensional position, not only being

confined on the focal plane.

z

xy

Source Position Optical System Cylindrical Lens Diffrent PSF in Detector

Figure 3.3: Extension to 3D imaging scheme in Nano-MUSIC. PSF of each fluorophore on image plane

is changed, which is depends on the level of defocus in axial direction.

3.5 Discussion

We examine the new type of super resolution microscopy, called Nano-MUSIC. The advantage of

Nano-MUSIC is that it doesn’t not only impose additional restriction on fluorescence dye, but also light

source compared to conventional microscopy. Moreover, MUSIC algorithm has been demonstrated to

achieve the same performance of single target localization for uncorrelated sources. However, correlation

of speckle illumination prevents Nano-MUSIC from achieving the performance of CRB. This spatial

– 21 –

correlation is more critical factor in 3D imaging because that speckle illumination is more correlated

in axial direction. Thus, we might use l∞-norm instead of l2-norm in order to increase the spatial

resolution. Beside, Nano-MUSIC can be implemented in total internal reflection fluorescence (TIRF)

microscopy as single molecule imaging approach methods did. It will increase performance of Nano-

MUSIC because TIRF microscopy uses evanescence wave as illumination source whose short wavelength

alleviates correlation problem of speckle illumination.

– 22 –

Chapter 4. Focus Stabilization Method

4.1 Introduction

In general, focus drift is one of the biggest issues for implementation of optical microscopy. In

regard to low magnification imaging by using low NA objective, it might not be critical because its axial

resolution reaches about few µm, which is relatively large in comparison with amount of focus drift. In

this case, the main interest for the stabilization problem is mechanical drift. It can be compensated

by means of stable mechanical system within few hundred nanometers, which is quite enough for the

case. However, in high magnification imaging case by using high NA objective lens, the problem of focus

stabilization is problematic or sometime quite critical for nano-scale imaging such as single molecule

tracking or super-resolution microscopy as well as dynamic imaging for a long time. In this case, as

the high NA objective lens usually requires an imaging medium such as water or oil, focus drift is

not only influenced by mechanical stage drift or vibration, but also thermal drift and fluctuation from

imaging medium. In addition, focal drift is more severe right after stage is adjusted in order to find

appropriate sample region because imaging medium must be perturbed by mechanical stage movement.

Furthermore, the drift in axial direction is worse than in lateral direction. In Nano-MUSIC, we should

take multiple snapshots for each color under the assumption that coordinates of fluorophores in image

plane are unchanged. Therefore, implementation of Nano-MUSIC must involve additional system to

maintain the focal plane during imaging.

Some methods have been demonstrated to maintain high precision focus stabilization from different

approach. For example, modified single molecule tracking technique can control the focus drift below

one nanometers, which is implemented by using nano-particle marker and quadrant photo-diode[61,

62]. Another method from STORM[20] is to use reference beam where focus drifted is compensated

within 40nm by intensity profile of reflected reference beam from glass-water interface. In this case,

reference beam is illuminated near critical angle in TIRF microscopy which changes detected intensity

from quadrant photo-diode corresponding to defocus. In addition, others overcome the problem based

on interference of reflected light from object in the range of several nanometers[63]. However, all these

methods are not appropriate for conventional wide field fluorescence imaging. The methods based single

molecule tracking[61, 62] is only appropriate for single particle imaging. The stabilization method in

STORM not only needs TIRF microscopy, but also focus drift error is quite larger than other methods.

Besides, interferometry based technique [63] also is not matched to fluorescence microscopy. Here, we

– 23 –

propose a new simple focus stabilization method.

4.2 Implementation and Experimental Results

The main idea of the proposed stabilization technique is to make use of asymmetric PSF deformation

along axial direction, which is quite similar with astigmatism imaging technique for 3D single molecule

localization. In this focus stabilization, only things needed are a reference laser whose wavelength is

not overlapped with imaging wavelength, piezoelectric transducer, cylindrical lens and CCD camera as

shown in Fig. 4.1 (a). A process of the focus stabilization method is as follows. First, a reference beam

passes through the same objective for imaging and then is focused at bottom surface as interface between

cover-glass and imaging medium (Fig. 4.1 (b)). The spot on the surface is about diffraction spot size

because we use high NA objective lens. Then, reflected light from the spot is imaged on the reference

detector through reference beam path. Since the axial resolution is at least over 500nm, the size of the

reference spot on the surface is almost the same within few hundred nanometer focal plane displacement.

However, the image of reference beam spot is asymmetrically deformed corresponding to the amount of

defocus because of inserted cylindrical lens whose focal length is 200nm in reference path (Fig. 4.1 (c)).

If the focal plane is maintained, the image of the spot on the detector looks spherical shape. Otherwise,

the image of the spot is changed into elongated shape whose direction depends on the defocus direction.

The interesting characteristic of this deformed PSF of the spot is the fact that the sum of intensity within

one of quadrant is approximately proportional to the amount of defocus in the range of 1µm as shown

in Fig. 4.1 (d). In proposed method, focus drift is controlled by this characteristic. Within the range

of 1µm defocus, graph of sum of the intensity is modeled as a line equation. The line equation can be

calculated by using least square fitting method, continuously increasing position of objective lens and

taking reference image simultaneously. In the proposed stabilization method, for the sake of simplicity,

we estimated the line equation with three time-averaged intensity values from measured in the three

layers which are image plane and above/below 100nm layers. The each time-averaged intensity value

can be created by integrating intensity of multiple snapshots in one of quadrant as shown in Fig. 4.1

(b). Next, we keep calculating the focal drift error by line equation and intensity sum from reference

image. For de-noising, we can use low-pass filtered reference image in time domain. After calculating

current focal plane displacement, we give feedback on piezoelectric transducer (PZT) of objective by PID

method which stands for proportional-integral-derivative.

The PID feedback control [64] is one of the widely used control systems. In PID, error can be calculated

as the difference between a measured variable and a desired set-point. PID generates the control input

in order to minimize the error. In the proposed method, measured variable, set-point are considered as

– 24 –

the current sum of intensity and the previous measured sum of intensity from image plane respectively.

In addition, the control input is write-position value of PZT of objective. The PID consists of three

separate terms: the proportional, the integral and the derivative terms, denoted P, I, D. These terms can

be considered as present error, on the accumulation of past errors, and prediction of future errors based

change of current error for P, I, D respectively. The PID can be mathematically described as follows:

U(t) = Kpe(t) +Ki

∫e(τ)dτ +Kd

d

dte(t) (4.1)

. The control input U(t) is determined by weighted sum of the three terms and each weighting coefficients

are adjusted experimentally. In Eq. 4.1, last derivative terms can contribute to prevent radical change

of control input values. However, the last term can boost high frequency spectral components noise of

measurement. Thus, PID method sometimes can be modified such as PI controller where only present

error and past error terms are used for feedback, omitting derivative term.

The proposed focus stabilization method was implemented by using additional two objectives, reference

laser, cylindrical lens, PZT and its controller. In order to focus the reference beam at the bottom surface,

distance between two objectives is adjusted appropriately, which makes reference beam slightly diverg-

ing after passing through two objectives. The PID feedback control was implemented as customized

LABVIEW program and commercial PZT and its controller. In experiment, derivative coefficient for

derivative term is relatively small value, comparing the other coefficients. In the implementation, refer-

ence image can be acquired at 50Hz and we averaged ten images as a measurement for the de-noising.

The current error is calculated by using line equation and the sum of intensity from one of quadrants

of the measurement. We used measurement that is created by integrating intensity in one of quadrant

from multiple images which acquired within 1/4 sec for denoising. The PZT is controlled by customized

LABVIEW program at 300Hz. Under mild condition, proposed method can stabilize the focus within

10nm displacement for 10 minutes as shown in Fig. 4.2(a) while write-value for PZT is changed about

150nm. In order words, without the focus stabilization, focal plane is drifted at least 150nm. Under noisy

circumstance, performance of the proposed technique will be decreased because of mechanical vibration

and electrical inference from other components in the system. In Nano-MUSIC, the proposed method

can maintain the axial focus within 30nm displacement as shown in Fig. 4.2(b) while the write-value

for PZT is also change over 150nm. This error can be considered as quite allowable degree of error for

super-resolution imaging in Nano-MUSIC. Thus, proposed method can guarantee that Nano-MUSIC can

take multiple images for few minutes at fixed image plane.

– 25 –

4.3 Discussion

In conclusion, we presented the new focus stabilization method. It is can be simply implemented

on epi-microscope and easily combined with other applications by using high NA objective. The novelty

of the proposed method is the fact that focal plane is maintained in nano-scale without additional

constraints for imaging method. Furthermore, we expect that focus plane can be controlled within

several µm by modifying the proposed system. In detail, whole range to be stabilized is segmented and

then, the two objectives are adjusted to focus reference beam at the bottom surface for each segmented

range. The distance between of two objectives is controlled by using motorized stage and also recorded

for each segmented range. Next, line equations are acquired corresponding to range segmented range.

Thus, the focal plane is controlled within several µm range based on the line equations. However, the

proposed method cannot compensate lateral drift. Generally, image plane also drifts in lateral direction

about several 10nm when objective lens is moved, perturbing imaging oil. Thus, another method to

correct lateral drift should be considered in order to extend Nano-MUSIC for thick sample imaging.

– 26 –

PS

F

Image Focal plane

Reference Focal plane

Defo

cus

objective

b

c

d

17000

21000

-400nm

400nm

0 nm

Inte

nsity S

um

0 12

(sec)

0 150 -150 -300 -450 300 450

(nm)

objective

lens

Cam

era

Dichroic

Mirror

PZT

PZT Controller

Cylindrical

lens

Speckle

illumination

a

R1objective

R2objective

Figure 4.1: Implementation of focus stabilization method. (a) Scheme of the implementation. A

cylindrical lens is inserted within reference beam path. (b) Reference beam is focus on the bottom

surface of cover glass while illumination light is focused on sample. The spot of reference beam is imaged

on the reference detector, changing the shape of spot corresponding to the displacement of image focal

plane. The PZT of objective get feedback control based on change of PSF. (c) The shape of reference spot

changes along axial defocus. (d) The intensity sum of spot image in one of quadrant is approximately

proportional to the displacement of defocus.

– 27 –

0 100 200 300 400 500 600

−250

−200

−150

−100

−50

0

50

0 100 200 300 400 500 600

−200

−150

−100

−50

0

50

error

offset

error

offset

low noise

high noise

a

b

(nm)

(sec)

(sec)

(nm)

Figure 4.2: Experimental results of focus stabilization for ten minutes. (a) Without external noise case.

(b) With external noise such as rotatoring diffuser vibration or electrical inference of system.

– 28 –

Chapter 5. Experimental Results of 3D Multi-color

Nano-MUSIC

5.1 Implementation

In the implementation of 3D multi-color Nano MUSIC is as shown in Fig. 5.1. The implemen-

tation of Nano-MUSIC is based on epi-fluorescence microscope (Olympus IX71). Illumination was im-

plemented by a 488nm Argon laser (Stellar-Pro-L 488/100, Modu-Laser) and 532nm continuous wave

diode pumped solid state laser (SDL-532-500, Shanghai Dream Lasers Technology). The light from the

two lasers combined by dichroic mirror (LM01-532, Semrock) is expanded by lens (f=-25mm) and then

illuminate the rotatable diffuser. The scattered light from the diffuser is collimated by lens (f=100mm)

and then reflected by dichroic mirror (FF405/496/560/651, Semrock) and multi-band dichroic mirror

(FF405/496/560/651, Semrock). Then, the collimated speckle pattern light is focused on the back focal

plane of objective (100x oil immersion lens with 1.30 NA UIS2 Olympus). The fluorescence emission

light is imaged on the electron-multiplying CCD (Luca-S, Andor) whose pixel size is 10µm through the

emission filter (FF01-422/503/572-25, Semrock ) and multiband dichroic mirror. For 488nm Argon laser,

additional band pass filter (FF01-512, Semrock) is inserted after emission filter. The EMCCD and the

rotatable diffuser are synchronized by function generator (AFG310, Tektronix). The frame rate of EM-

CCD is 10 Hz and 1000 images per each color are acquired, which takes 100 sec (200sec total). Between

tube lens and the EMCCD, additional magnification lens (1.6x magnification) can be inserted. For 160x

magnification, imaging pitch per pixel is 62.5 µm.

For the proposed focus stabilization, feedback control system is implemented by additional reference laser

(HeNe Laser(633nm) or near infrared laser(785nm)) and piezoelectric transducer objective stage (Nano-

F200S, Mad City Labs). The reference light is focused on cover-glass-immersion oil interface through

the same objective lens. The focused reference beam spot is then reflected and imaged on CCD camera

(IEEE 1394 camera) through reference beam path. In front of reference CCD camera, cylindrical lens (f

= 200nm) is inserted to deform the PSF of reference beam. The PZT stage is controlled in real-time by

customized program which coded by LABVIEW (National Instruments).

– 29 –

fO1

(objective)

Camera

fM

(1/1.6X Mag. Lens)

fT (Tube Lens)

Multi-band Dichroic Mirror

Multi-band Emission Filterf4

(200mm)

f2

(100mm)

Sample

Speckle Pattern

532nm

Laser Diode

Rotating

Diffuser

(1000 ppr)

f3

(100mm)

ND Filter

fI

488nm

Ar Laser

Cam

era

Dichroic

Mirror

f5

(200mm)

f6

(200mm)

f7

(1000mm)

PZT

Dichroic

Mirror

fO2

(objective)

fO3

(objective)

Beam

Splitter

f1

(100mm)

633 / 785nm

NIR Laser

Figure 5.1: The implementation of Nano-MUSIC based on efi-fluorescence microscopy. The light from two

lasers is combined and scattered though the rotatable diffuser. Then speckle pattern from the diffuser is

reflected by dichroic mirror and then illuminates the sample through 100x, 1.30NA, oil immersion typed

objective lens (UIS2 Olympus). The emission light from the sample is imaged on EMCCD (Luca-S,

Andor). The focal plane of the objective lens is stabilized by the proposed method implemented by

PZT, objective stage and reference laser (HeNe or IR laser) with customized program.

– 30 –

5.2 Cell Preparation

5.2.1 Reagents and Contructs

For mitochondria, antibody against Tom20 was purchased from BD Biosceince. For F-actin, Phalloidin-

TRITC, Fluor 594-, 488-conjugated goat anti-mouse and goat anti-rabbit IgG were purchased from

Molecular Probes. For microtubule, AE-8 as marker was purchased from Santa Cruz Biotechnology.

5.2.2 Immunofluorescences

HeLa cells were grown in Dulbeccos complete medium (JBI) supplemented with 10% heat-treated

fetal calf serum (Invitrogen), 100U/ml penicillin, and 100 g/mL streptomycin. For immunostaining, cells

grown in the culture dish with cover glasses were fixed with 4% paraformaldehyde for 15min at room

temperature, permeabilized with 0.15% Triton X-100 in PBS for 15 min at room temperature, and then

blocked with 3% bovine serum albumin in PBS for 45 min at room temperature. The cover glasses with

cells were incubated for 2 hr with indicated primary antibody with fluorescent material. After washing

with PBS, the glasses were mounted with fluorescent mounting medium (Dako).

5.3 Two-color Imaging Results

5.3.1 Microtubule and F-actin in Hela cell

Here, we demonstrate multi-color imaging of Nano-MUSIC for sub-cellular structure of HeLa cells

such as microtubule and F-actin. Microtubule and F-actin are components of cytoskeleton of eukaryotic

cells such as HeLa cells. Microtubule which is rope-like polymers of tubulin can grow as long as 25 µm with

diameter of 25nm from centrosome which serves as the main microtubule organizing center. Microtubule

is very important for maintain cell structure and inter cellular interaction as transport platforms[65].

F-actin as a polymerization of actin protein consists of microfilaments of the cytoskeleton. Two parallel

F-actin strands create double helix structure of the microfilaments, rotating on top of each other. The

microfilaments are 7nm in diameter and the helix structure is repeated in 37nm[66]. Microtubule and

F-actin are stained by Alexa488 and TRITC respectively. As shown in Fig. 5.2, microtubule and

F-actin widely spread out in the cytoplasm of a cell. Here, blurred structures of microtubule and F-

actin in conventional image in Fig. 5.2 (a) are clearly resolved in Nano-MUSIC image in Fig. 5.2 (b).

Furthermore, we can observe that microtubule grew outward from centrosome (A) to periphery of the

cell. In addition, Out of focus lights from F-actin, surrounded by nucleus (B) are removed clearly. In

closed-up images from line boxes(in Fig. 5.2 (a,b) ), Nano-MUSIC can resolve up to 65nm as shown in Fig

5.2 (e,f). In another sample as shown in Fig. 5.3, the conventional fluorescence image is diffused because

of complex microtubule. In order to clarify the resolution improvement, we visualized overlapped image

– 31 –

between conventional image and Nano-MUSIC image. In Nano-MUSIC, twig structures of microtubule

are clearly resolved as shown in Fig. 5.3 (d,e). Next, similar result can be observed in Nano-MUSIC

images in Fig. 5.3 and Fig. 5.4 that are images from different parts of the same HeLa cell. Dense F-actin

line structures are sharply reconstructed by Nano-MUSIC and complex microtubule structures are also

resolved.

5.3.2 Mitochondria and F-actin in Hela cell

Here, we demonstrate multi-color imaging of Nano-MUSIC for sub-cellular structure of HeLa cells

such as mitochondria and F-actin. Mitochondrion (plural mitochondria) is a membrane-enclosed or-

ganelle found in eukaryotic cells. Mitochondrion has 300nm to 1µm diameter. Mitochondria are the role

of supply of adenosine triphosphate (ATP), which is source of chemical energy [67]. Moreover, mitochon-

dria are concerned to other processes, such as signaling and cell death, as well as cell growth. As F-actin

consists of cytoskeleton of cells, it usually surrounds mitochondria. We stained mitochondria outer mem-

brane (TOM) complex and F-actin by Alexa 48 and TRITC. As shown in Fig. 5.6(b), Nano-MUSIC

image can reveals outer membrane structure of mitochondria and morphology of F-actin in comparison

with conventional fluorescence image (Fig. 5.6(a)). In closed up images in Fig. 5.6(d,f), complex outer

membrane structures of mitochondria are resolved, showing hollows inside the mitochondria unlike con-

ventional blurred image (Fig. 5.6(c,e)). It means that Nano-MUSIC not only improve lateral resolution,

but also axial resolution. In another mitochondria and F-actin stained sample (Fig. 5.7), we can observe

mitochondria and F-actin are widely distributed in whole HeLa cells. Nano-MUSIC also shows unseen

structures such as hollows of mitochondria and strands of F-actin (Fig. 5.7(b)). In closed-up images

of dotted line boxes in Fig. 5.7(a,b), the boundaries of mitochondria and F-actin are well resolved and

hollows of mitochondria are also observed. In closed-up images of line boxes in Fig. 5.7(a,b), adjacent

two F-actin strands placed below 100nm can be resolved and dim mitochondria morphology are clearly

reconstructed. Next, as shown in Fig5.8, densely distributed mitochondria and F-actin are resolved ac-

curately and their complex structures can be observed well. In closed-up images of line boxes in Fig.

5.8(a,b), the boundaries of mitochondria and F-actin are clearly separated in Nano-MUSIC image(Fig.

5.8(d)) unlike blurred conventional image (Fig. 5.8(c)).

– 32 –

~65nm

100 300 500 700

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

(nm)00 100 300 500 700

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

(nm)

~97nm

a c

b d

e f

A

B

Figure 5.2: The two-color image of microtubule and F-actin. Alexa488(green) and TRITC(red) were

used for microtubule and F-actin, respectively. Raw images are acquired by 100x magnification. (a)

Conventional fluorescence image, created from temporal mean of 1000 snapshots for each color. (b)

Nano-MUSIC image. (c,d) Closed-up images from line boxes in (a,b). (e) Line profile of line in (c). (f)

Line profile of dotted line in (d). Scale-bars are 5µm in (a,b) and 500nm in (c,d), respectively.

– 33 –

a

cb

d e


used for microtubule and F-actin, respectively. Raw images are acquired by 160x magnification.(a)

Overlapped image from conventional and Nano-MUSIC (b) Conventional fluorescence image, created

from temporal mean of 1000 snapshots for each color. (c) Nano-MUSIC image. (d,e) Closed-up images

from dotted line boxes in (b,c). Scale bars are 5µm in (b,c) and 500nm in (d,e) respectively.

– 34 –

a

c

d

b

e f




Nano-MUSIC image. (c,e) Closed-up images from line boxes in (a). (d,f) Closed-up images from line

boxes in (b). Scale-bars are 5µm in (a,b) and 1µm in (c,d)

a

b

c

d




Nano-MUSIC image. (c,d) Closed-up images from line boxes in (a,b),respectively. Scale-bars are 5µm

in (a,b) and 500nm in (c,d)

– 35 –

a

c e

b

d f

Figure 5.6: The two-color image of mitochondria and F-actin. Alexa488(green) and TRITC(red) were

used for mitochondria and F-actin, respectively. Raw images are acquired by 100x magnification. (a)


Nano-MUSIC image. (c,e) Closed-up images from line boxes in (a). (d,f) Closed-up images from line

boxes in (b). Scale-bars are 3µm in (a,b) and 500nm in (c-f)

– 36 –

a

b

c d

e f

0 100 200 300 400

0.2

0.4

0.6

0.8

1

100nm

g




Nano-MUSIC image. (c,d) Closed-up images from line boxes in (a,b). (e,f) Closed-up images from

dotted line boxes in (a,b). (g) line profile from yellow lines in (e,f). Scale-bars are 5µm in (a,b) and

500nm in (c-f) respectively.

– 37 –

a

b

c d




Nano-MUSIC image.(c,e) Closed-up images from line boxes in (a). (d,f) Closed-up images from line

boxes in (b). Scale-bars are 5µm in (a,b) and 500nm in (c,d)

– 38 –

5.4 3D Imaging Results

Experimentally, we demonstrate the transformation of PSF along axial direction with 532 diode laser

and 60nm fluorescence particles ( Bangs Laboratories) in several hundred nanometer range by inserting

convex cylindrical lens whose focal length is 1000mm as shown in Fig. 5.9. In Fig. 5.9(a,b) shows change

of measured FWHM of PSF in x and y direction corresponding to the axial position and transformed

PSF s of nano particle respectively. Here, we can observe that PSF changes asymmetrically along the

axial deformation. Thus, we can extend Nano-MUSCI to 3D imaging by using different PSF shape along

axial direction.

−400 −300 −200 −100 0 100 200

300

350

400

450

500

550

600

650

wx

wy

Wid

th (

nm

)

Z (nm)

200nm

0nm

-400nm

-200nm

(a) (b)

Figure 5.9: Calibration of PSF 60nm fluorescence bead in axial direction. (a) In the graph, measured

width of PSF in x and y direction is changed corresponding to the axial position. (b) 2d PSF shape of

single particle.

For 3D two-color imaging, we insert cylindrical lens in front of the EMCCD and then take 1000 snapshots

for each color at fixed focal plane, which takes totally 200 sec. As shown in Fig. reffig:tubule3D1 (a),

we can visualize temporal mean image for two-color. The temporal mean image looks more blurred

comparing conventional wide field image because inserted cylindrical lens makes conventional spherical

PSF broaden, which increases out of focus components interference. In order to localize fluorophores

in 3 dimensions, we modify A matrix in Eq. 3.9 by using calibrated PSF as shown in Fig. 5.9. With

multiple snapshots from a single focal plane, we reconstruct super resolution 3D image in 500nm axial

direction. We applied 3D Nano-MUSIC to biological sample such as HeLa cells. Sub structures of HeLa

cells such as mitochondria, microtubule, F-actin are prepared by the same way in previous section. In

– 39 –

Fig. 5.10, Nano-MUSIC can achieve 3D super resolution images from single image layer and three images

from bottom, middle and top plane are visualized (in Fig. 5.10 (b,c,d)). The distance of each adjacent

two planes is about 250nm which calculated from calibrated PSF. In the closed-up images from line

boxes in Fig. 5.10 (b,c,d), we can observe different morphology of F-actin in comparison with top and

bottom plane. Moreover, microtubule is usually located near cover glass surface that is estimated as the

bottom plane. In addition, we make the 3D rendered image by commercial software (AMIRA, Visage

Imaging GmbH) as shown in Fig. 5.10(k) and 3d structures of F-actin and microtubule can be observed.

For mitochondria and F-actin stained HeLa cells, Nano-MUSIC can reconstruct super resolution 3D

images. As shown in Fig. 5.11, three image from bottom, middle, top plane are visualized as the same

way in previous microtubule and F-actin stained sample. We can separate different F-actin strands

stretching different directions by compared to top and bottom images (in Fig. 5.10(b,d)). In addition,

mitochondria have different morphology along the axial direction. Then, we also present 3D rendered

image by commercial software (AMIRA, Visage Imaging GmbH) as shown in Fig. 5.10(h) where 3D

structures of F-actin and mitochondria are well resolved.

– 40 –

a

b e h

i

j

f

g

c

d

k



Temporal mean image from 1000 snapshots for each color. (b-d) Nano-MUSIC images from +250nm,

0nm, -250nm image planes,respectively. (e,h) Closed-up images from line boxes in (b). (f,i) Closed-up

images from line boxes in (c). (g,j) Closed-up images from line boxes in (d). (k) 3D rendered image by

AMIRA. Scale-bars are 2µm in (a-d) and 500nm in (e-j)

– 41 –

a

c

d

b e

f

g

h



Temporal mean image from 1000 snapshots for each color. (b) Nano-MUSIC images from +250nm, 0nm,

-250nm image planes,respectively. (e-g) Closed-up images from line boxes in (b-d) respectively. (h) 3D

rendered image by AMIRA. Scale-bars are 1µm in (a-d) and 500nm in (e-g).

– 42 –

Chapter 6. Conclusion

6.1 Summary

In the thesis, we presented new novel 3D multi-color super resolution fluorescence microscopy using

dynamic speckle illumination and array signal processing techniques. Nano-MUSIC can achieve super

resolution without any additional constraints for the optical non-linearity because the spatial resolution

improvement in Nano-MUSIC is based on speckle illumination for uncorrelated fluctuation of individ-

ual fluorophore not photoswitchable dye and great performance of array signal processing technique

for resolving the locations of fluorophores. However, previous super resolution methods require spe-

cial hardware instruments such as STED laser, well calibrated interferometry and TIRF microscope or

photoswichable fluorescence dye compared to conventional fluorescence microscopy. More specifically,

STED needs high power (10MW-1GW/cm2) STED laser in order to doughnut shaped STED beam pat-

tern to quench the periphery area of excitation beam which cause cell damage and massive hardware

setup in order to synchronize STED laser and excitation laser to make stimulated emission outperform

spontaneous emission.Moreover, STED is limited in choice of fluorescence dyes such as atto532 and

atto647N to exploit stimulated emission, which makes multi-color imaging of STED practically limited

as two-color for overlapped spectrum of fluorescence dye in visible range. Thus, these limitations prevent

STED from being widely used for general purpose. SSIM also requires high power excitation pattern

(10MW-50MW/cm2) to raise saturable process of fluorescence dyes. Furthermore, SSIM has been only

demonstrated for lateral spatial resolution improvement while other methods have been demonstrated

with respect to two-color and 3D imaging. When it comes to single molecule based approach such as

STORM and PALM/FPALM, they also have disadvantages compared to other methods. First, imaging

speed is relatively slow which usually takes several minute to few hours depending on image size because

they repeatedly localize the coordinate of single photoswitchable marker. Besides, slow imaging speed

also restricts live imaging because of low time resolution. Furthermore, the systems of this approach

are based on TIRF microscope to excite single molecule within diffraction limit size area. Thus, their

possible axial imaging range is fundamentally limited below few µm even though 3D imaging within 3µm

has been demonstrated by inserting cylindrical lens. In addition, multi-color imaging also generally lim-

ited to two-color imaging because of limited possible combination of photoswitchable dyes. With respect

to the disadvantages of previous super resolution methods, proposed method has advantages as shown

in Table. 6.1. Moreover, we explained the performance of Nano-MUSIC which can approach to CRB

– 43 –

Table 6.1: Super resolution Fluorescence Microscopy

STED SSIM PALM/STORM Nano-MUSIC

Lateral

Resolution20-30nm 50nm 20-30nm 65 nm

Axial Resolution100nm,

30nm(with 4pi)not described 30nm not described

Multi-color 2 1 2the same with

conventional

Laser PowerUltra high

(10MW-1GW)

Very high

(10MW-50MW)

High

( 30W)

Low

(<100mW)

Fluorescence Dye

Special

characteristic

required

Special

characteristic

required

Special

characteristic

required

Any

Photo-bleaching Very High Very High High Weak

Postprocessing No Yes Yes Yes

for uncorrelated excitation of sources with high SNR. Then, we demonstrated Nano-MUSIC to achieve

two-color super resolution imaging without any additional requirements. For biological samples such as

microtubule, mitochondria, F-actin, Nano-MUSIC can discover unresolved details of cellular structures.

Furthermore, 3D imaging is also possible in Nano-MUSIC by inserting a cylindrical lens in front of EM-

CCD. Moreover, we also present new focus stabilization method in order to prevent focal plane drift for

imaging in Nano-MUSIC. Experimentally, it was demonstrated to maintain focal plane in 10nm error

range. However, as speckle illumination is correlated by illumination optics, non-diagonal components

of source covariance matrix prevent Nano-MUSIC from achieving CRB performance. Moreover, since

speckle pattern is highly correlated in axial direction, axial resolution for 3D imaging is more degraded.

we expect that the problem of correlation problem from speckle illumination can be solved by using

TIRF microscope, which can push performance of Nano-MUSIC to CRB.

6.2 Future Research directions

In super resolution fluorescence microscopy, 3D imaging is required because not only most cellular

structures are in three dimensions, but also it helps to investigate complex cellular interactions. In this

thesis, we extended Nano-MUSIC to 3D imaging by inserting cylindrical lens. However, in Nano-MUSIC,

axial resolution is accurately demonstrated and 3D imaging is only possible in 500nm because 3D imag-

ing is achieved by using measurements from single image layer. Furthermore, since speckle illumination

pattern is highly correlated in axial direction, it is one of factors to decrease achievable axial resolution.

– 44 –

Therefore, we should modify the system of Nano-MUSIC for not only 3D imaging in large range by using

multi-layer measurements, but also uncorrelated speckle illumination in axial direction.

Live imaging as an important application of fluorescence microscopy is required in order to understand

biological mechanism. For live imaging, high enough time resolution is required to record dynamics in

cells. However, we have to develop Nano-MUSIC to have few seconds’ time resolution. If we use current

high frame camera such as 100Hz, imaging time of Nano-MUSIC is simply reduced from 200sec to 20sec

for two-color imaging. Furthermore, imaging time can be reduced by modifying Nano-MUSIC algorithm.

In addition, we should implement additional system for live imaging such as cell culture.

Reconstruction time is another issue to be considered in Nano-MUSIC. Since Nano-MUSIC has rela-

tively slow reconstruction speed for super resolution imaging compared to imaging speed. Thus, current

reconstruction time should be decreased up to few minutes to be widely used for researchers. We have

to not only optimize the current MUSIC based algorithm, but also consider different type of array signal

processing algorithm in order to improve time and spatial resolution.

– 45 –

References

[1] A.H. Coons, H.J. Creech, and R.N. Jones, “Immunological properties of an antibody containing a

fluorescent group.,” in Proceedings of the Society for Experimental Biology and Medicine. Society

for Experimental Biology and Medicine (New York, NY). Royal Society of Medicine, 1941, vol. 47,

pp. 200–202.

[2] O. Shimomura, F.H. Johnson, and Y. Saiga, “Extraction, purification and properties of aequorin,

a bioluminescent protein from the luminous hydromedusan, aequorea,” Journal of cellular and

comparative physiology, vol. 59, no. 3, pp. 223–239, 1962.

[3] E. Abbe, “Beitrage zur theorie des mikroskops und der mikroskopischen wahrnehmung,” Archiv fur

mikroskopische Anatomie, vol. 9, no. 1, pp. 413–418, 1873.

[4] J.B. Pawley and B.R. Masters, “Handbook of biological confocal microscopy,” Journal of Biomedical

Optics, vol. 13, pp. 029902, 2008.

[5] S. Hell and E.H.K. Stelzer, “Fundamental improvement of resolution with a 4pi-confocal fluorescence

microscope using two-photon excitation,” Optics communications, vol. 93, no. 5-6, pp. 277–282,

1992.

[6] MGL Gustafsson, DA Agard, JW Sedat, et al., “I5m: 3d widefield light microscopy with better

than 100nm axial resolution,” Journal of microscopy, vol. 195, no. 1, pp. 10–16, 1999.

[7] E. Betzig, JK Trautman, TD Harris, JS Weiner, and RL Kostelak, “Breaking the diffraction barrier:

optical microscopy on a nanometric scale,” Science, vol. 251, no. 5000, pp. 1468, 1991.

[8] S.W. Hell and J. Wichmann, “Breaking the diffraction resolution limit by stimulated emission:

stimulated-emission-depletion fluorescence microscopy,” Optics letters, vol. 19, no. 11, pp. 780–782,

1994.

[9] S.W. Hell, “Far-field optical nanoscopy,” Single Molecule Spectroscopy in Chemistry, Physics and

Biology, pp. 365–398, 2010.

[10] M.G.L. Gustafsson, “Nonlinear structured-illumination microscopy: wide-field fluorescence imaging

with theoretically unlimited resolution,” Proceedings of the National Academy of Sciences of the

United States of America, vol. 102, no. 37, pp. 13081, 2005.

– 46 –

[11] E. Betzig, G.H. Patterson, R. Sougrat, O.W. Lindwasser, S. Olenych, J.S. Bonifacino, M.W. David-

son, J. Lippincott-Schwartz, and H.F. Hess, “Imaging intracellular fluorescent proteins at nanometer

resolution,” Science, vol. 313, no. 5793, pp. 1642, 2006.

[12] S.T. Hess, T.P.K. Girirajan, and M.D. Mason, “Ultra-high resolution imaging by fluorescence

photoactivation localization microscopy,” Biophysical journal, vol. 91, no. 11, pp. 4258–4272, 2006.

[13] M.J. Rust, M. Bates, and X. Zhuang, “Sub-diffraction-limit imaging by stochastic optical recon-

struction microscopy (storm),” Nature methods, vol. 3, no. 10, pp. 793–796, 2006.

[14] V. Westphal and S.W. Hell, “Nanoscale resolution in the focal plane of an optical microscope,”

Physical review letters, vol. 94, no. 14, pp. 143903, 2005.

[15] T.A. Klar, S. Jakobs, M. Dyba, A. Egner, and S.W. Hell, “Fluorescence microscopy with diffraction

resolution barrier broken by stimulated emission,” Proceedings of the National Academy of Sciences,

vol. 97, no. 15, pp. 8206, 2000.

[16] G. Donnert, J. Keller, C.A. Wurm, S.O. Rizzoli, V. Westphal, A. Schonle, R. Jahn, S. Jakobs,

C. Eggeling, and S.W. Hell, “Two-color far-field fluorescence nanoscopy,” Biophysical journal, vol.

92, no. 8, pp. L67–L69, 2007.

[17] V. Westphal, S.O. Rizzoli, M.A. Lauterbach, D. Kamin, R. Jahn, and S.W. Hell, “Video-rate far-

field optical nanoscopy dissects synaptic vesicle movement,” Science, vol. 320, no. 5873, pp. 246,

2008.

[18] M.G.L. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured

illumination microscopy,” Journal of Microscopy, vol. 198, no. 2, pp. 82–87, 2000.

[19] M. Bates, B. Huang, G.T. Dempsey, and X. Zhuang, “Multicolor super-resolution imaging with

photo-switchable fluorescent probes,” Science, vol. 317, no. 5845, pp. 1749, 2007.

[20] B. Huang, W. Wang, M. Bates, and X. Zhuang, “Three-dimensional super-resolution imaging by

stochastic optical reconstruction microscopy,” Science, vol. 319, no. 5864, pp. 810, 2008.

[21] H. Shroff, C.G. Galbraith, J.A. Galbraith, and E. Betzig, “Live-cell photoactivated localization

microscopy of nanoscale adhesion dynamics,” Nature methods, vol. 5, no. 5, pp. 417–423, 2008.

[22] J.R. Lakowicz and B.R. Masters, Principles of fluorescence spectroscopy, vol. 13, 2008.

[23] T.A. Klar and S.W. Hell, “Subdiffraction resolution in far-field fluorescence microscopy,” Optics

letters, vol. 24, no. 14, pp. 954–956, 1999.

– 47 –

[24] M.G.L. Gustafsson, L. Shao, P.M. Carlton, CJ Wang, I.N. Golubovskaya, W.Z. Cande, D.A. Agard,

and J.W. Sedat, “Three-dimensional resolution doubling in wide-field fluorescence microscopy by

structured illumination,” Biophysical journal, vol. 94, no. 12, pp. 4957–4970, 2008.

[25] B.J.W.S. Rayleigh, The theory of sound, vol. 2, Macmillan, 1896.

[26] W. Denk, J.H. Strickler, and W.W. Webb, “Two-photon laser scanning fluorescence microscopy,”

Science, vol. 248, no. 4951, pp. 73, 1990.

[27] I.I. Smolyaninov, Y.J. Hung, and C.C. Davis, “Magnifying superlens in the visible frequency range,”

Science, vol. 315, no. 5819, pp. 1699, 2007.

[28] SW Hell and M. Kroug, “Ground-state-depletion fluorscence microscopy: A concept for breaking

the diffraction resolution limit,” Applied Physics B: Lasers and Optics, vol. 60, no. 5, pp. 495–497,

1995.

[29] M. Dyba and S.W. Hell, “Focal spots of size λ/23 open up far-field florescence microscopy at 33 nm

axial resolution,” Physical review letters, vol. 88, no. 16, pp. 163901, 2002.

[30] R. Schmidt, C.A. Wurm, S. Jakobs, J. Engelhardt, A. Egner, and S.W. Hell, “Spherical nanosized

focal spot unravels the interior of cells,” Nature methods, vol. 5, no. 6, pp. 539–544, 2008.

[31] B. Bailey, D.L. Farkas, D.L. Taylor, and F. Lanni, “Enhancement of axial resolution in fluorescence

microscopy by standing-wave excitation,” 1993.

[32] L. Shao, P. Kner, E.H. Rego, and M.G.L. Gustafsson, “Super-resolution 3d microscopy of live whole

cells using structured illumination,” Nature Methods, 2011.

[33] L. Schermelleh, P.M. Carlton, S. Haase, L. Shao, L. Winoto, P. Kner, B. Burke, M.C. Cardoso, D.A.

Agard, M.G.L. Gustafsson, et al., “Subdiffraction multicolor imaging of the nuclear periphery with

3d structured illumination microscopy,” Science, vol. 320, no. 5881, pp. 1332, 2008.

[34] J. Gelles, B.J. Schnapp, and M.P. Sheetz, “Tracking kinesin-driven movements with nanometre-scale

precision,” Nature, vol. 331, no. 6155, pp. 450–453, 1988.

[35] R.N. Ghosh and W.W. Webb, “Automated detection and tracking of individual and clustered

cell surface low density lipoprotein receptor molecules,” Biophysical journal, vol. 66, no. 5, pp.

1301–1318, 1994.

[36] B. Huang, S.A. Jones, B. Brandenburg, and X. Zhuang, “Whole-cell 3d storm reveals interactions

between cellular structures with nanometer-scale resolution,” Nature methods, vol. 5, no. 12, pp.

1047–1052, 2008.

– 48 –

[37] H. Shroff, C.G. Galbraith, J.A. Galbraith, H. White, J. Gillette, S. Olenych, M.W. Davidson, and

E. Betzig, “Dual-color superresolution imaging of genetically expressed probes within individual

adhesion complexes,” Proceedings of the National Academy of Sciences, vol. 104, no. 51, pp. 20308,

2007.

[38] M. Andresen, A.C. Stiel, J. Folling, D. Wenzel, A. Schonle, A. Egner, C. Eggeling, S.W. Hell, and

S. Jakobs, “Photoswitchable fluorescent proteins enable monochromatic multilabel imaging and

dual color fluorescence nanoscopy,” Nature biotechnology, vol. 26, no. 9, pp. 1035–1040, 2008.

[39] S.A. Jones, S.H. Shim, J. He, and X. Zhuang, “Fast, three-dimensional super-resolution imaging of

live cells,” Nature Methods, vol. 8, no. 6, pp. 499–505, 2011.

[40] Jang jaeduck, “Nanometer resolution fluorescence microsopy using speckle illumination and array

signal processing,” 2011.

[41] R. Schmidt, “Multiple emitter location and signal parameter estimation,” Antennas and Propaga-

tion, IEEE Transactions on, vol. 34, no. 3, pp. 276–280, 1986.

[42] L. Speckle, “Related phenomena,” Topics in Applied Physics, vol. 9, 1984.

[43] J.W. Goodman, Speckle phenomena in optics: theory and applications, Roberts & Co, 2007.

[44] X. Zong, A.F. Laine, and E.A. Geiser, “Speckle reduction and contrast enhancement of echocardio-

grams via multiscale nonlinear processing,” Medical Imaging, IEEE Transactions on, vol. 17, no. 4,

pp. 532–540, 1998.

[45] K.Z. Abd-Elmoniem, A. Youssef, and Y.M. Kadah, “Real-time speckle reduction and coherence

enhancement in ultrasound imaging via nonlinear anisotropic diffusion,” Biomedical Engineering,

IEEE Transactions on, vol. 49, no. 9, pp. 997–1014, 2002.

[46] C. Ventalon and J. Mertz, “Quasi-confocal fluorescence sectioning with dynamic speckle illumina-

tion,” Optics letters, vol. 30, no. 24, pp. 3350–3352, 2005.

[47] C. Ventalon and J. Mertz, “Dynamic speckle illumination microscopy with translated versus ran-

domized speckle patterns,” Opt. Express, vol. 14, pp. 7198–7209, 2006.

[48] C. Ventalon, R. Heintzmann, and J. Mertz, “Dynamic speckle illumination microscopy with wavelet

prefiltering,” Optics letters, vol. 32, no. 11, pp. 1417–1419, 2007.

[49] A.K. Dunn, H. Bolay, M.A. Moskowitz, and D.A. Boas, “Dynamic imaging of cerebral blood flow

using laser speckle,” Journal of Cerebral Blood Flow & Metabolism, vol. 21, no. 3, pp. 195–201,

2001.

– 49 –

[50] J.D. Briers, “Laser doppler, speckle and related techniques for blood perfusion mapping and imag-

ing,” Physiological Measurement, vol. 22, pp. R35, 2001.

[51] Y.K. Park, W. Choi, Z. Yaqoob, R. Dasari, K. Badizadegan, and M.S. Feld, “Speckle-field digital

holographic microscopy,” Optics express, vol. 17, no. 15, pp. 12285–12292, 2009.

[52] J.C. Mosher, S. Baillet, and R.M. Leahy, “Eeg source localization and imaging using multiple signal

classification approaches,” Journal of clinical neurophysiology, vol. 16, no. 3, pp. 225, 1999.

[53] P. Stoica and N. Arye, “Music, maximum likelihood, and cramer-rao bound,” Acoustics, Speech

and Signal Processing, IEEE Transactions on, vol. 37, no. 5, pp. 720–741, 1989.

[54] SF Yau and Y. Bresler, “Worst case cramer-rao bounds for parametric estimation of superimposed

signals with applications,” Signal Processing, IEEE Transactions on, vol. 40, no. 12, pp. 2973–2986,

1992.

[55] H.P. Kao and AS Verkman, “Tracking of single fluorescent particles in three dimensions: use of

cylindrical optics to encode particle position,” Biophysical journal, vol. 67, no. 3, pp. 1291–1300,

1994.

[56] S. Ram, J. Chao, P. Prabhat, E.S. Ward, and R.J. Ober, “A novel approach to determining the

three-dimensional location of microscopic objects with applications to 3d particle tracking,” in Proc.

SPIE, 2007, vol. 6443, pp. 64430D–1.

[57] M.J. Mlodzianoski, M.F. Juette, G.L. Beane, and J. Bewersdorf, “Experimental characterization

of 3d localization techniques for particle-tracking and super-resolution microscopy,” Optics express,

vol. 17, no. 10, pp. 8264–8277, 2009.

[58] S.R.P. Pavani and R. Piestun, “Three dimensional tracking of fluorescent microparticles using a

photon-limited double-helix response system,” Optics express, vol. 16, no. 26, pp. 22048–22057,

2008.

[59] S.R.P. Pavani, M.A. Thompson, J.S. Biteen, S.J. Lord, N. Liu, R.J. Twieg, R. Piestun, and WE Mo-

erner, “Three-dimensional, single-molecule fluorescence imaging beyond the diffraction limit by using

a double-helix point spread function,” Proceedings of the National Academy of Sciences, vol. 106,

no. 9, pp. 2995, 2009.

[60] G. Grover, S.R.P. Pavani, and R. Piestun, “Performance limits on three-dimensional particle local-

ization in photon-limited microscopy,” Optics letters, vol. 35, no. 19, pp. 3306–3308, 2010.

– 50 –

[61] M. Capitanio, R. Cicchi, and FS Pavone, “Position control and optical manipulation for nanotech-

nology applications,” The European Physical Journal B-Condensed Matter and Complex Systems,

vol. 46, no. 1, pp. 1–8, 2005.

[62] A.R. Carter, G.M. King, T.A. Ulrich, W. Halsey, D. Alchenberger, and T.T. Perkins, “Stabilization

of an optical microscope to 0.1 nm in three dimensions,” Applied optics, vol. 46, no. 3, pp. 421–427,

2007.

[63] K. Kim and O.A. Saleh, “Stabilizing method for reflection interference contrast microscopy,” Applied

optics, vol. 47, no. 12, pp. 2070–2075, 2008.

[64] G.F. Franklin, J.D. Powell, A. Emami-Naeini, and J.D. Powell, Feedback control of dynamic systems,

vol. 2, Addison-Wesley Reading, MA, 1994.

[65] G. Jekely, Eukaryotic membranes and cytoskeleton: origins and evolution, vol. 607, Springer Verlag,

2007.

[66] J. Vandekerckhove and K. Weber, “At least six different actins are expressed in a higher mammal:

an analysis based on the amino acid sequence of the amino-terminal tryptic peptide,” Journal of

molecular biology, vol. 126, no. 4, pp. 783–802, 1978.

[67] P.H. Harvey and M.D. Pagel, The comparative method in evolutionary biology, vol. 239, Oxford

university press Oxford, 1991.

– 51 –

Summary

Array Signal Processing Based 3D Multi-color Super Resolution

Fluorescence Microscopy

본 연구에서는 기존 광학계의 회절 한계를 뛰어넘는 초고해상도 다색 삼차원 형광영상 기법을 제안한

다. 형광 현미경은 면역형광법 개발과 형광단백질 발견을 통한 형광프로브에 관한 폭넓은 연구덕분에

세포의 형태와 동작원리를 밝히는 데 유용한 툴로 자리잡았다. 하지만 기존의 형광 현미경은 회절한계

로 인하여 발생하는 형광 파장의 절반 이하의 작은 세포 내 구조를 구별하는 것이 불가능 했다. 최근 10

년간 기존의 회절한계를 뛰어넘는 원거리 형광 영상 기법이 폭 넓게 연구되었다. 대표적인 예로 STED

는 뾰족한 유효 점 퍼짐 함수를 만들기 위해 형광물질의 유도방출을 이용하였다. 또한 SSIM은 여러

개의선형무늬조명을통해초고해상도영상을얻었다. 또다른방법으로 STORM/PALM은단일분자

영상기법과광학적으로키고끌수있는형광프로브를이용하여기존의회절한계를뛰어넘는초고해상

도 영상을 얻었다. 하지만 이런 기술들은 각자 다른 한계점들을 가지고 있다. 예를 들어 STED와 SSIM

은 세포에 유해할 수 있는 강한 조명과 그에 따른 제한된 형광프로브 그리고 정교하고 복잡한 광학계의

설치가필요하다. 또한 STORM/PALM은광학적으로키고끌수있는형광물질을기반하므로일반적인

형광물질을 사용할 수 없다. 이러한 제약은 초고해상도 형광영상 기법의 대중화를 저해하는 요소가 된

다. 따라서기존의형광현미경의기법을사용할수있는초고해상도영상기법의개발이요구되었다. 본

연구에서는 스페클조명과 배열신호처리기법을 이용하여 초고해상도 다색 삼차원 영상을 복원하였다.

제안한 방법은 초고해상도 영상복원을 여러 신호원의 위치 추적문제로 전환하여 기존의 형광 현미경

에서 사용하던 형광물질을 모두 사용 가능한 초고해상도 형광영상기법이다. 자세히는 스페클조명을

통해 각 형광물질이 시간에 따라 무작위로 발광되며 이를 부분공간 기반 배열신호처리 기법 중 하나인

MUSIC 방법을 통해 초고해상도 영상을 복원하였다. 추가적으로 다색 초고해상도 영상복원을 위해

초점면을 유지시키는 방식을 개발하였다. 제안한 방법은 미소관, 미토콘드리아 그리고 액틴이 염색된

HeLa 세포에 적용하여 성능을 평가하였고 다색 삼차원 초고해상도 영상이 최대 수평 해상도 65nm로

복원됨을 보였다.

핵심어: 초고해상도, 3차원 영상, 다색 영상, 스페클, 배열신호처리, 초점면 안정화

– 52 –

0ô à8Ÿ 0Łdt 3 äÉ àt ł › uploads › 2 › 5 › 6 › 7 › 25679421 › 22... ·...

Documents