Copyright © 2009 by Lew Brown, Fluid Imaging Technologies, Inc.1

Imaging Particle Analysis: Resolution and Sampling ConsiderationsLew Brown

Technical DirectorFluid Imaging Technologies, Inc.

I. WhatDoWeMeanby“Resolution”?

The term “resolution” as it applies to imaging is often misused, and is easily misunderstood. As a start, consider this definition specific to imaging from the Merriam-Webster online dictionary:

6 a: the process or capability of making distinguishable the individual parts of an object, closely adjacent optical images, or sources of light b: a measure of the sharpness of an image or of the fineness with which a device (as a video display, printer, or scanner) can produce or record such an image usually expressed as the total number or density of pixels in the image <a resolution of 1200 dots per inch> (1)

As can be seen from the above, “resolution” is used in two very different ways: definition “a” refers to the ability of a system to capture information (input resolution), while definition “b” refers to the ability of a system to output information. These are completely independent concepts, although the “input resolution” of an image can certainly affect its ability to be properly output. For example, many people wrongly assume they can print something they see on a computer screen on paper and see the same quality output at the same size. What they fail to realize is that in this case, the image on the screen only has a resolution (input) of 72 pixels/inch, whereas the equivalent print (output) resolution is 300 pixels (dots)/inch. For the purposes of this paper, we are only concerned with definition “a”, the input resolution.

The resolution that can be captured by any imaging system is limited by two distinct factors: the optical system and the

sensor. An easy way to think about this is to remember your last visit to the eye doctor, reading a letter chart on the wall: the various lenses that the doctor puts between your eyes and the letter chart are the optical system, and your eyes are the sensor (which remains constant). As the doctor changes the lenses, the image seen by the eye can become sharper or more blurry; the sensor (eyes) remains constant. Your ability to “resolve” objects is determined by your ability to recognize the letters on the chart correctly.

Although we perceive the world as “continuous tone” (all colors and shades of colors merge smoothly into their neighbors), the human eye is actually a discrete sensor composed of rods and cones on the retina. There are approximately 120 million rods on the retina (sensitive only to gray scale) and 6-7 million cones (which are sensitive to color) (2). In digital imaging terminology, we can think of the eye as having a 120 Megapixel black and white sensor and a 6-7 Megapixel color sensor. This is an extremely high-resolution sensor! Despite the fact that the eye is indeed a discrete sensor, we never see any “pixilation”, because the eye as a sensor is also connected to the most powerful computer

Figure 1: Most particle analyzers give a distribution of particle size only as shown by the graph on the left. Imaging particle analysis yields size, shape and gray-scale information, enabling the automated characterization of different particle types in a heterogeneous sample. However, the ability to make these differentiations based upon particle shape requires a certain minimum level of image resolution, which is the topic of this paper.

Abstract: Imaging particle analysis represents an exciting method of particle analysis which combines the speed of automated particle analyzers with the discrimination found in optical microscopy. However, in order to discriminate shape differences, it is limited to particles of a certain size or larger (see Figure 1) . This paper discusses the primary factors that lead to this limitation, caused by the optical system and the sensor. In the end, diffraction limits of the optical system and sampling limits incurred at the sensor limit this technique to particle counting of particles 1µm in Equivalent Spherical Diameter (ESD) and larger, and particle characterization of particles 2µm in ESD and larger. Due to this limitation, imaging particle analysis of submicron particles is limited to non-optical techniques such as electron microscopy.

Copyright © 2009 by Lew Brown, Fluid Imaging Technologies, Inc.2

known, the human brain. The brain processes the signals from the eye to make the world appear as completely “continuous tone”.

Using definition “a” above, the most common terminology used to describe “spatial resolution” comes from photography, where the resolution of a system is described in its ability to distinguish closely spaced lines in an image. The unit of measure used is “line pairs per millimeter”, where a “line pair” is a pair of two parallel black lines separated by an equal width white line. In film photography, a resolution target consisting of groups of bars of increasing numbers of line pairs/millimeter is imaged, and the largest bars that the imaging system can not discern are considered the limit of the system’s resolving power. The most commonly used target has been the 1951 USAF Resolution test Target (Figure 2)

In film systems, given identical optics, the resolution limit would be determined by the microstructure of the film itself, or the film “grain” size. Essentially this equates to the mean size of the silver grains which are laid down on the film emulsion during manufacturing. The smaller the grains, the more detail that can be resolved in the test target (the trade-off is that finer grained film also requires more light to be “exposed”). In general, film has much higher resolution than most reasonably priced digital systems (although as with most digital systems, the sensor prices are dropping precipitously while the pixel density continues to increase).

II. Whatis“Resolution”inaDigitalSystem?

We described above how resolution in a film system is measured using a resolution target. In the film system, the sensor is the film, and its spatial resolution is generally limited by the

granularity of the film. In a digital system, the sensor is a discrete array of “picture elements” or pixels arranged on a rectangular grid. Each pixel is a photosensitive site that outputs a signal based upon the amount of light striking it. Although the actual “signal” produced by the photosite is “continuous”, the signal is immediately converted by an analog to digital converter (A/D) into a discrete digital number. In a black and white system, this number is usually an 8-bit number ranging from 0 to 255, where 0 is black and 255 is white. Color systems (with the exception of multiple sensor systems or Foveon chip systems) use three separate photosites to measure Red, Green and Blue intensity, which are combined to produce color (16.7 million colors can be created from 8 bits each of red, green and blue data). For ease of discussion, we will work with a monochrome system here.

Since the sensor in a digital system has a discrete number of photosites, or pixels, the resolution of the sensor itself is usually the prime decider of the spatial resolution limit of the overall system. As with film, one of the easiest ways to determine the resolution limit is simply to image a resolution test target to determine the maximum number of line pairs per millimeter that the system can distinguish.

III.CompositionofaDigitalImage

As discussed above, a digital image is composed of a two-dimensional matrix of picture elements, or pixels. Each pixel has a value associated with it, typically an 8-bit number where 0 is black, 255 is white, and the numbers in between represent varying shades of gray (recall that we are going to limit our discussion to monochrome images for

Figure2:USAF1951ResolutionTestTarget

OpticsObject Space

Image (Sensor) Space

Optical Axis

Sensor Pixel Dimensions:5µm X 5µm

Sensor Pixel Dimensions,Projected onto Object:

2.5µm X 2.5µm

Sensor Geometry Projected onto Object

Figure3:ProjectingSensorGeometryBackontoObject

Copyright © 2009 by Lew Brown, Fluid Imaging Technologies, Inc.3

simplicity’s sake, color images are just an extrapolation of the monochrome). Because the image sensor is a discrete 2D array of pixels, the resolution of the sensor is fixed. If the overall magnification of the optical system is known, we can actually predict the overall resolution of the system by projecting the sensor forward through the optical system onto the target (see Figure 3, preceding page).

Although the optical magnification of the system can be deduced mathematically, it is more generally simply measured by imaging a target of known dimensions. This could be as simple as imaging a ruler; in imaging particle analysis the common method is to image calibrated spheres which are traceable to a known standard (typically NIST), and to calculate the images’ Equivalent Spherical Diameter (ESD). As an example, if a ruler is imaged, and it is found that it takes ten pixels to cover 10mm, then the resolution is 10 pixels = 10mm or 1 pixel = 1 mm. The calibration of the system can then be

expressed by the size of one pixel projected onto the target plane, in this case 1mm/pixel. All other distances can now be measured merely by multiplying the number of pixels to cover the object by the calibration factor, so a 25 pixel long object in the image would be 25mm long.

So far this seems pretty straightforward, but we need to remember that there is another dimension to each pixel, the gray scale value. In order to make measurements using the

technique above, we need to reduce each pixel to a binary value that says that either the pixel is part of what you want to measure, or it is not. In imaging particle analysis, what this really says is that a pixel can only either be “particle” or “not particle”. The reduction of the image to a binary image is accomplished through a simple “gray scale threshold”. First, a “background” image is recorded that represents the gray

scale value for each pixel in the sensor when no particles are present. Then, for each image acquired when sample is present, the background pixel value is subtracted from the incoming value for the same pixel in the sensor array, yielding a “difference” value. If the difference value is 0, then the pixel is the same as the background, and no particle is present. If the difference is greater than (or less than) 0, then something is present. At this point, the software makes a decision as to whether this

pixel is “particle” or “not particle” based upon a user-supplied threshold value for the difference. Figure 4 shows how this binarization would look on some sample objects at a very coarse level.

It is extremely important to realize that the pixel density of the projected sensor onto the object will have an enormous impact upon what the thresholded image looks like, and its measurements (Figure 5).

The Effect of Thresholding

Theshold

Gray-Scale Image

Binary Image

Figure4:Gray-ScaleThresholdingtoProduceaBinaryImage

The Effect of Resolution

Theshold

Gray-Scale Image

Binary Image

Resolution: 1 pixel = 1 unit area Resolution: 4 pixels = 1 unit area

Size (ESD) = 2�(4 pixels/π) = 2.26 units

Size (ESD) = 2�(2 pixels/π) = 1.60 units

Size (ESD) =2�[(12 pixels/4)/π] = 1.95 units

Size (ESD) =2�[(6 pixels/4)/π]= 1.38 units

Figure5:Gray-ScaleThresholdingtoProduceaBinaryImage

Copyright © 2009 by Lew Brown, Fluid Imaging Technologies, Inc.4

It is an absolute axiom in all digital imaging that more resolution in the sensor always yields more accuracy in measurements and a more faithful rendition of the object given all other things in the system are constant (and that diffraction limits are not reached).

To make matters a bit more complex, we need to realize that there are a couple of types of error that can be introduced during the thresholding process. The first type could be

called “quantization error”, and can be introduced by setting the threshold differently when looking at the same objects (Figure 6).

The second type of error that can be introduced during thresholding is “positional error”, essentially the fact that the overlap of the projected sensor onto the object can actually produce different results for the same size object with the same threshold (Figure 7).

IV.SamplingTheoryandNyquistLimits

The process of digitizing is the process of converting an analog (or continuous) signal or object into a discrete set of points or samples. This process is also known as “sampling”. One of the most important tenets of sampling theory is the Nyquist-Shannon sampling theorem. Many good references can be found that address this theorem in detail (3,4,5), and a detailed discussion is beyond the scope of this paper.

The basic conclusion of the Nyquist-Shannon sampling theory is that “in order to get an accurate reproduction of a continuous signal with a particular frequency the sampling frequency must be at least the double of that number. The theorem refers to units that must be translated to the particular case of digital imaging. The theorem says you need at least 2 samples per cycle, and this means two pixels per line pair” (6) To put this into microscopic terms, if you wish to resolve one line pair/micron on an object, you must sample that object with at least 2 pixels/micron, or a system

calibration of 0.5 microns/pixel. Once again, this calibration value is based upon the projection of the system’s image sensor onto the object through the optics. So, in this example, if the optics are 10X magnification, then the sensor must have a photosite density of 5 microns/pixel (or smaller) in order to resolve one line pair per micron on the sample.

Current state-of-the-art industrial digital video cameras with a resolution of 1024x768 pixels have a pixel size somewhere

Gray Scale Image

Binary Image

100

100

100

100 100

100 100

100

50

100

100 100 100

100 100

100 100

50

Threshold <100 Threshold ≤150

Quantization Error by Threshold

Size (ESD) = 2�(6 pixels/π) = 2.80 unitsSize (ESD) = 2√(1 pixels/π) = 1.12 units

Threshold

Figure6:SizeErrorCausedbyDifferentThresholds

Gray Scale Image

Binary Image

100

100

100

100 100

100 100

100

50

200

200 50 50

200 50

200 200

50

Threshold <100 Threshold <100

Positional Error

Size (ESD) = 2�(4 pixels/π) = 2.20 unitsSize (ESD) = 2�(1 pixels/π) = 1.12 units

Threshold

Figure7:SizeErrorCausedbyDifferentProjectionontheSensor

Copyright © 2009 by Lew Brown, Fluid Imaging Technologies, Inc.5

between 4 to 5 microns/pixel, so this is within the range desired in the example above. However, a Nyquist sampling frequency of 2 samples per cycle is a theoretical minimum sampling rate to resolve an object of 1 cycle in size. In reality, many more samples are usually necessary to actually “resolve” the object. Most microscopists use a sampling rate of between 3-10 samples per object as a rule of thumb (7). This would mean that in the above example (trying to resolve 1 line pair/micron), they would want to have a system calibration somewhere in the area of between 0.33 microns/pixel and 0.10 microns/pixel. At 10X magnification, this represents a photosite density on the sensor of between 3.3 and 1.0 microns/pixel. With current technology, anything below 4 microns/pixel is not only very expensive in a sensor, but is also extremely prone to noise due to the small size of the photosite.

The final thing to remember in this discussion is that, so far, we have been talking only about resolving “line pairs”, which are very simple objects. For the purposes of imaging particle analysis, resolving a line pair can be looked at as the minimum unit for measurement of Equivalent Spherical Diameter (ESD). So, continuing the example above, if we can resolve 1 line pair/micron, we should be able to measure spherical particles 1 micron and larger in diameter. However, the primary strength of imaging particle analysis over other more common techniques (electrozone sensing, laser diffraction, etc.) is that it can be used to measure particle shape attributes beyond ESD. These higher order measurements can then be used to differentiate particles from

one another. A common example would be to measure a particle’s “circularity” by comparing its actual perimeter to its perimeter based upon ESD. However, if we are sampling only to resolve ESD, the circularity measured will be the same for all particles of that size regardless of shape (see Figure 8).

This means that in order to measure higher order shape attributes such as perimeter on a particle, the particle has to be sampled at a much higher sampling rate than would be suggested by the Nyquist limit! Finally, because light is a wave-based phenomenon, the absolute limit of any instrument forming an image by wave interference is half the wavelength of the wave used to form the image (in this case, light). In other words, using a 550nm light source, the theoretical maximum resolution that could be achieved would be 0.275µm.

V.DiffractionLimits

All of the above discussion was made making a very simple (and very incorrect) assumption: that the optical system is “perfect”, meaning that it performs perfectly according to the mathematics associated with it. In reality, there is absolutely no such thing as a “perfect” optical system, and the closer one tries to get to “perfect”, the faster the cost rises! All lenses have defects referred to as aberrations; some of the common ones are astigmatism, distortion, field curvature, and coma. These defects result in a loss of image quality in the projected image onto the sensor. Many of these defects can be “corrected” by design and materials, and some can also be eliminated by post-processing of the digital image.

However, there is one further limit on image quality (sharpness) for which there is no “fix”, diffraction. Diffraction is caused by the wave-like nature of light and what happens to those waves when they encounter objects (such as an aperture) or changes in the material the wave is travelling through (such as the change in refractive index when travelling from air into glass). All optical imaging systems can be characterized by a “Numerical Aperture” which is a direct indication of how well the optics will be able to resolve fine detail. Numerical Aperture is defined as NA = n sinθ where n is the index of refraction the lens is working in (1.0 for air) and θ is the half-angle of the maximum cone of light that can enter or exit the lens with respect to a point P (focal plane) (see Figure 9) (8).

Gray Scale Image

Binary Image

Loss of Information atLow Sampling Frequency

Size (ESD) = 2�(9 pixels/π) = 3.40 unitsPerimeter = 12 units

Size (ESD) = 2�(9 pixels/π) = 3.40 unitsPerimeter = 12 units

Threshold

(Both Particles are assigned the same Circularity due to Under-Sampling)

Figure8:Information(Detail)LossduetoUndersampling

Copyright © 2009 by Lew Brown, Fluid Imaging Technologies, Inc.6

The size of the finest detail resolvable by an optical system is proportional to λ/NA, where λ is the wavelength of the illumination. For a constant λ, the higher the NA of the system, the more light the lens gathers and the higher level of detail it can resolve. Going back to the definition of NA, since air has an index of refraction of 1.0, the theoretical maximum NA for a lens working in light is 1.0. For this reason, in very high resolution microscopy, oil immersion lenses are used where the oil can have an index of refraction in the 1.5 range. Finally, since the resolution is proportional to λ/NA, it is important to note that shorter wavelengths of light will give higher theoretical resolution for the same NA (see Figure 10) (9).

Wavelength(Nanometers)

Resolution(Micrometers)

360 0.19400 0.21450 0.24500 0.26550 0.29600 0.32650 0.34700 0.37

In a system using “white light” illumination, this means that the resolution limit is different for objects of different wavelengths. If we choose the middle wavelength of 550 nm, then we can see that our theoretical maximum resolution for a 0.95 NA objective is 0.29 µm. Once again, this result is calculated using many assumptions which are very unlikely to occur in the real world. Also, recall from the previous discussion, that when we talk about “resolving” an object here, we are referring to a simple theoretical line pair, not some complex organic shape.

An optical system that can actually produce the theoretical maximum angular resolution is said to be “diffraction limited”. In the real world, most optical systems have enough additional defects so as to be significantly lower resolution than the diffraction limit.

VI.WhatDoesthisMeantoMicroscopicImagingParticleAnalysis?

So we have now seen that there are several factors which enter into and can limit the “resolution” of a digital imaging system: the density of the photosites on the digital sensor, sampling artifacts, Nyquist limits and finally diffraction. Let us now look at how this affects imaging particle analysis. From our discussion on diffraction limits, we know that, putting aside any other considerations for now, any optical sensing system will be at very minimum limited by diffraction. Diffraction tells us that even with a “perfect” optical system, the best we could ever possibly resolve in a microscope system would be on the order of around 0.30µm, or 3.3 line pairs per micron. Now, add in what we learned from sampling theory, and that resolution is cut by at least a factor of 2 (or halved). Finally, add in the fact that our optical system will never be “perfect” (it will have aberrations), and we begin to see that actual resolution for merely “counting” particles in an optical system will be on the order of minimum 1µm spheres (as no shape data can be measured or inferred at this level). As previously stated, the important difference in imaging particle analysis lies in shape discrimination, so realistically we can only really talk about discerning “low level” shape constructs (10) at particle sizes of 2µm ESD or larger.

At this point, a “real world” example should greatly help to put all of this “theory” into perspective! A sample containing particles smaller than 10µm in ESD was run through the FlowCAM®, a continuous-imaging particle analysis system manufactured by Fluid Imaging Technologies, Inc. (Yarmouth, ME). This particular sample was a parenteral drug sample, although for looking at particles below 2 µm, the actual sample does not much matter (as will be evident from the images).

Once the particle images were acquired by the FlowCAM, the instrument’s VisualSpreadsheet© software was used to filter and isolate particles having an Equivalent Spherical Diameter (ESD) equal to 1µm. The first image below shows 9 particle images that have an ESD = 1µm. Note the summary statistics associated with these particles. The actual images are too small to see any real detail on when displayed at 1:1 (actual pixels) (see Figure 11, following page). The second image

θP

Figure9:DefinitionofHalf-AngleforNumericalAperture

Figure10:WavelengthVersusResolutionatFixedNA=0.95

Copyright © 2009 by Lew Brown, Fluid Imaging Technologies, Inc.7

shows the same nine particle images zoomed by a factor of 64X, so that one pixel in the original image is now shown using a 64x64 pixel array on the screen (see Figure 12). At this magnification, the actual pixels are easily seen because they are now represented as “blocks” of data. This type of digital zoom

is known as “pixel replicated”, because each pixel is made larger simply by replicating it. The final image is the same but with the addition of the binary overlay to see the actual pixels that the measurements were made from (see Figure 13).

Figure11:ParticleswithESD=1µmatActualScale

Figure12:ParticleswithESD=1µmat64XZoom

Figure13:ParticleswithESD=1µmat64XZoomwithBinaryOverlay

Copyright © 2009 by Lew Brown, Fluid Imaging Technologies, Inc.8

Compare the images above of 1µm ESD particles with the following two particles from the same sample having a 4µm ESD (Figures 14, 15: these images are also zoomed by a factor of 64X as per the previous images):

It can be clearly seen from these images of the 4µm ESD particles that far more detail is now seen in the images. In the two particle images above, for example, we can now clearly distinguish that one of these particles is spherical in shape whereas the other is “rod-like”. We can also now begin to collect “higher order” measurements like “circularity” at this point, which we were incapable of doing with the 1µm ESD particles.

VII. Conclusions

Clearly there are many factors that affect the ability of an imaging particle analysis system to resolve detail in very small microscopic particles. As pointed out in the introduction, we stated the resolution of the imaging particle analysis system

is limited by two different factors, the optical system and the sensor. The optical system is, in the best case, limited by diffraction and the wavelength of light being used to image the particles. Adding to this, the sensor further limits the optical system due to sampling constraints (Nyquist limit) and also by physical limitations on the size of the actual photosite that can be produced on the sensor.

All of these factors combined, along with the example shown, lead us to the following basic conclusions:

1.) Particle counting in an imaging particle analysis system should be limited to particles having an ESD of 1µm and greater.

2.) “Simple” particle characterization (i.e. “round” versus “rod-like”) in an imaging particle analysis system should be limited to particles having an ESD of 2µm and greater.

3.) “Higher level” particle characterization (i.e. differentiation based upon higher order measurements such as circularity) in an imaging particle analysis system should be limited to particles having an ESD of 4µm or greater.

For submicron particles, imaging particle analysis requires the use of non-optical imaging techniques, such as electron microscopy. Unfortunately, these techniques are even more limited than optical microscopy from the standpoint of requiring extensive laboratory set-up (and expensive equipment). This means that imaging of statistically significant numbers of particles of this size is not possible, only very small samples can be observed.

VIII. References

1.) Merriam-Webster Online Dictionary: http://www.merriam-webster.com/dictionary/resolution

2.) HyperPhysics web site, Light and Vision, Georgia State University Department of Physics and Astronomyhttp://hyperphysics.phy-astr.gsu.edu/hbase/vision/rodcone.html

3.) Wikipedia entry on Nyquist-Shannon sampling theoremhttp://en.wikipedia.org/wiki/Nyquist-Shannon_sampling_theorem

Figure14:ParticleswithESD=4µm,64XZoom

Figure15:ParticleswithESD=4µm,64XZoomwithBinaryOverlay

Copyright © 2009 by Lew Brown, Fluid Imaging Technologies, Inc.9

4.) An Introduction to Sampling Theory, Thomas Zawistowski & Paras Shahhttp://www2.egr.uh.edu/~glover/applets/Sampling/Sampling.html(contains an interactive Java applet demonstrating aliasing caused by sampling)

5.) Digital Signal Processing : Principles, Algorithms, and Applications, by J. Proakis and D. Manolakis, New York: Macmillan Publishing Company, 1992

6.) “Do Sensors “Outresolve” Lenses?”, Rubén Osuna and Efraín García, Luminous Landscape Web Sitehttp://luminous-landscape.com/tutorials/resolution.shtml

7.) Microscopy Today (Microscopy Society of America), Volume 14 - Number 6, November 2006, Netnotes, “Image Analysis - object size”, pages 63-66

8.) Wikipedia entry on Numerical Aperturehttp://en.wikipedia.org/wiki/Numerical_aperture

9.) Nikon MicroscopyU on the web, Concepts and Formulas/Resolutionhttp://www.microscopyu.com/articles/formulas/formulasresolution.html

10.) “Particle Image Understanding - A Primer”, Lew Brown,Fluid Imaging Technologies Web Sitehttp://fluidimaging.com/imaging-particle-analysis-white-papers.aspx