X-ray imaging for measuring holes in cheeseby

Lorenzo Venturini (Queens’ College)

Fourth-year undergraduate project inGroup F, 2015/2016

Supervised by Dr. Joan Lasenby

I hereby declare that, except where specifically indicated, the worksubmitted herein is my own original work.

Signed: Date:

Technical abstract

X-ray imaging for measuring holes in cheeseLorenzo Venturini, Queens’ College, Group F, 2015/2016

Swiss-type cheeses like Emmental and Leerdammer are characterised by the presenceof large and prominent holes, which give these cheeses their characteristic look and flavour.As consumers expect a certain size and distribution of holes in their Swiss cheese, ensur-ing that these characteristics meet their standards has become economically important.Quality control of these cheeses, to ensure that they have the desired characteristics, isthus an important step. Traditionally, this has been performed by a technique known ascore sampling, an inherently destructive method which cuts through and visually exam-ines holes along a column cut through the cheese. Not only does this damage the cheese,it also shears the holes along the column, distorting them and providing an inaccuratesample of the internal structure. It is also limited in scope, providing data only on thoseholes which happen to lie along the column cut.

This project attempts to develop a new, nondestructive technique to obtain a betterpicture of the size and distribution of the holes in this cheese. Specifically, the aim of theproject is to be able to reconstruct a 3-dimensional map of the cheese from the minimumnumber of X-ray projections of the cheese, obtaining an accurate and economically com-petitive quality-control technique. This is to be designed for commercial applications, andtherefore the cost and competitiveness of the techniques are more important than perfectaccuracy.

It was attempted to obtain this 3D reconstruction from just 2-dimensional X-ray pro-jections from different angles. This is insufficient to obtain an accurate 3D reconstructionusing traditional techniques (such as those used for medical imaging), and therefore acustom method was designed to reconstruct the structure by incorporating the knownprior information about the characteristics of the holes in cheese. Individual holes werefound in each 2D projection, and then simple geometric techniques were used to matchholes in different projections to each other and reconstruct their 3D positions and lengths.

To obtain a ground truth to compare reconstructions to, a proxy cheese, with similarstatistics to real Emmental cheese, was designed using MATLAB and parametric CADand then manufactured by 3D printing. X-ray projections of this were then taken andanalysed to obtain reconstructions.

An objective function was devised and an optimisation technique to minimise it wasdeveloped by matching ellipsoidal kernels to individual 2D projections. The kernels wereinitialised at uniform spacing and they were translated and stretched until the rms errorof the reconstructed image using them was minimised. This was done in two different 2Dprojections, taken at a relative angle of 90º to each other, and the set of kernel locations,brightnesses and eccentricities was compared between the two projections. A similarity

2

score was calculated between each pair of converged kernels, and the most similar kernelsin the two projections were matched to each other.

This matching, plus knowledge of the geometry of the system, provided enough infor-mation to obtain a full three-dimensional reconstruction of the structure of the cheese.The centres of the holes were simply found via geometric principles from their positionsin each view. The length along each axis was similarly obtained, using the assumption oforthographic projection to make recovery of the lengths easier and less prone to noise.

The locations of the centres of the holes which were correctly reconstructed in this waywere found with good accuracy, with an rms error on the position of the centre of 4.5mm.Reconstruction of the lengths and volumes of the holes was somewhat less accurate, withaverage deviations of 12% and 19%, respectively. This is nonetheless far better accuracythan offered by core sampling methods.

The execution time of the method on high-resolution X-ray images was found to be veryslow, running on the order of an hour on a standard PC for a single projection. This canbe improved by subsampling high-resolution images, but the optimisation method devisedis inefficient and has scope to be made faster. Using a method with a variable step size orpreloading the reconstruction to compute rms error could substantially improve the speedof the technique. Another relatively simple change that could be made to improve theaccuracy of this technique is to change the optimisation technique to correctly fit holesat the edges of the cheese, or holes with a substantial amount of overlap. It should alsobe possible to automatically detect the edges of the cheese in the image and make theprocess invariant to translation.

Overall, it was deemed that the technique developed offers better performance thantraditional core sampling. It is rapid (with only two X-ray projections), cost-effective,nondestructive, and gives a reasonably accurate image of the location and sizes of theholes within the cheese. Additional steps were proposed to improve the accuracy andefficiency of the algorithm outlined, and possible future applications for this techniquewere also specified.

3

Acknowledgments

I would like to thank my supervisor, Dr. Joan Lasenby, for all the guidance and supportshe has given me throughout the project. She has followed my progress throughout theyear and her advice on how to proceed has proved invaluable.

I would also like to express my gratitude to Cheyney Design for providing the ideafor this project and allowing me to use their X-ray equipment. In particular, I wouldlike to thank Richard Parmee for being my main point of contact and providing valuableresources; Jonathan Cameron for teaching me to use the X-ray machine and providingme with X-ray images; and Patrick Roux for arranging the delivery of real cheese.

Mifroma provided the cheese that was used in this project and their help has made itpossible to test the methods developed here in the real world.

I would also like to thank Prof. Jim Haseloff of the Cambridge University Departmentof Plant Sciences for allowing me to use his lab’s 3D printer for my unusual and time-consuming cheese print, as well as Mihails Delmans in his lab for printing the proxy.

Finally, I would like to thank Konstantinos Kyriakopoulos, who gave me advice withmy code and helped me greatly speed up my implementation, as well as being an excellentfriend.

4

Contents

1 Introduction 71.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.2 Problem definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.3 Structure of report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Theory and design of the experiment 92.1 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2 Cheese specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3 X-ray imaging principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.4 Reconstruction techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.4.1 Traditional methods . . . . . . . . . . . . . . . . . . . . . . . . . . 122.4.2 Hough transform-based methods . . . . . . . . . . . . . . . . . . . 12

2.5 Geometric principles and three-dimensional reconstruction . . . . . . . . . 13

3 Techniques implemented 143.1 3D printed proxy design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.1.1 Proxy design considerations . . . . . . . . . . . . . . . . . . . . . . 143.1.2 3D printing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.1.3 Image preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.2 Histogram adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3 Traditional techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.4 Hough transform-based methods . . . . . . . . . . . . . . . . . . . . . . . . 19

4 First-principles ellipse fitting 204.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.2 Description of technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.3 Ellipsoidal kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214.4 Specification of an objective function . . . . . . . . . . . . . . . . . . . . . 22

4.4.1 Description of terms in objective function . . . . . . . . . . . . . . 234.5 Optimisation method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.5.1 Choice of asynchronous update . . . . . . . . . . . . . . . . . . . . 264.6 Determining the appropriate number of kernels . . . . . . . . . . . . . . . 26

5 Three-dimensional fitting 275.1 Matching projections to each other . . . . . . . . . . . . . . . . . . . . . . 27

5.1.1 Determination of similarity scores . . . . . . . . . . . . . . . . . . . 275.1.2 Determination of correct matches . . . . . . . . . . . . . . . . . . . 29

5.2 Recovery of 3D position and parameters . . . . . . . . . . . . . . . . . . . 305.2.1 X-ray scaling factors . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5

5.3 Assumption of orthographic projection . . . . . . . . . . . . . . . . . . . . 345.4 Surface and intersecting holes . . . . . . . . . . . . . . . . . . . . . . . . . 35

6 Results and discussion 356.1 Accuracy and efficiency of 2D optimisation . . . . . . . . . . . . . . . . . . 35

6.1.1 Efficiency of 2D optimisation . . . . . . . . . . . . . . . . . . . . . 366.1.2 Comparison with past results . . . . . . . . . . . . . . . . . . . . . 38

6.2 Three-dimensional fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396.3 Agreement with ground truth . . . . . . . . . . . . . . . . . . . . . . . . . 41

7 Conclusion 427.1 Future directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

7.1.1 Possible applications . . . . . . . . . . . . . . . . . . . . . . . . . . 43

References 45

A Risk assessment retrospective 48

6

1 Introduction

1.1 Motivation

Cheese with holes, such as Emmental or Leerdammer, is a very popular and tasty additionto the dinner table. Production of Emmental cheese in Switzerland and France aloneexceeds 270000 tons a year [1]. One of the distinguishing features of these cheeses is thepresence of large and prominent holes (or “eyes”), caused by colonies of bacteria whichconsume the cheese during maturation and produce carbon dioxide [2]. As the densityand size of holes changes the chemical makeup and flavour of the cheeses, manufacturersstrive to maintain a consistent size and distribution of holes in their product.

With such a popular and profitable product, it is surprising that the techniques usedto find the holes within samples of cheese and conduct quality control have remainedthe same for centuries. In fact, the traditional technique to inspect holes, known as coresampling, simply involves extracting a cylindrical sample from a wheel of cheese using atool known as a cheese trier [3], and manually inspecting the holes present along it.

While this can provide a rough picture of the characteristics of the holes along thatsample, it has some clear limitations. This technique is semi-destructive, lowering thevalue of the wheel of cheese. Furthermore, it only gives information about the holes onthe sample - it will not, for example, be able to find any asymmetries in the distributionof holes in the cheese.

1.2 Problem definition

The aim of this project is to design a nondestructive technique to faithfully find the 3Dpositions and other characteristics of the holes within a cheese. This should be at leastas accurate as existing (destructive) techniques, as well as cost-effective. X-ray imagingwas chosen as it is a relatively cheap and established technology that can provide a good-quality picture of the holes within the cheese.

X-ray imaging is not the only method which can be thought of to provide a three-dimensional map of the cheese: several other common imaging techniques, such as ul-trasound or MRI, could conceivably be used instead. X-ray was chosen due to the lowcost, high resolution and low image acquisition time of the technique, which makes it aneasy option for this project. MRI was discarded due to its high cost relative to X-rayimaging and high acquisition time. Ultrasound, though cheap, requires a gel to be spreadover the cheese first to diminish reflection of the ultrasound signal. This was deemedto be unpalatable to consumers, potentially lowering the value, and therefore X-ray waspreferred.

To produce a three-dimensional image, multiple two-dimensional projections mustbe taken and combined; one part of this project is focused on obtaining a good three-

7

Figure 1: Two X-ray images of a cheese proxy, imaged at a relative angle of ∼ 25º. Therelative change in position of the holes in the projections suggest that it is possible toextract 3D information from them.

dimensional image with all desired information, from 2D projections (see Figure 1).This project is aimed primarily at commercial applications, trying to find a technique

that is more cost-effective and accurate than current methods. This means that anymethod developed does not need to be perfectly accurate: it just needs to be able toprovide a better picture than established techniques, with only a small error. It alsomeans that low cost is a priority: as part of this tradeoff, it is desirable to take as few2D projections as possible. The aim therefore is to take as few as two projections, whilemaintaining an appropriate level of fidelity.

Cheyney Design, a Cambridge-based company specialised in X-ray imaging, providedtechnical advice as well as X-ray equipment to help with this project; Mifroma, a Swisscheese manufacturer, provided cheese samples.

1.3 Structure of report

This report is split into seven sections, corresponding to the different aspects of theproblem that were examined.

Section 2 provides background to the problem, giving a review of the literature tacklingsimilar problems and a technical background to the methods and techniques used in theproject. Section 3 describes the work that was done to prepare images for analysis,including design of a 3D-printed proxy, image preprocessing, and the performance ofcommon techniques. A custom method to extract holes in individual 2D projections isoutlined in section 4, describing the process, structure of the kernels and components ofthe cost function. Two projections are then combined to obtain a single 3D reconstruction,and the technique used to find that is described in section 5. Section 6 gives an overviewof the performance and efficiency of each of the stages implemented in this report, with a

8

quantitative analysis of the quality of the results and suggestions to improve the method.Conclusions and reflections on future directions and possible applications are given insection 7.

2 Theory and design of the experiment

2.1 Literature review

There are many examples in the literature of X-ray imaging being used for quality controlof foods, including cheese. Brosnan and Sun (2004) [4] offer a review of computer visionapplications in food inspection, including several applications for X-rays. These includeusing X-ray imaging to detect bones in chicken and fish [5] and to find split pits in peaches[6].

These techniques are well-established, and several companies exist which offer X-rayapplications for food inspection, including Cheyney Design, which was heavily involvedin this project. However, past efforts have mostly focused on detecting foreign objectswithin the cheese, such as fragments of metal or stone.

The only attempt that could be found in the literature to characterise holes usingX-ray imaging techniques was made by Kraggerhud et al. [7]. They sought to find thepositions and dimensions of holes in cheese using an X-ray scan, and therefore provide astatistical indication of the quality of the cheese. However, their attempts are limited toa single 2-dimensional projection - they did not attempt to build a 3D reconstruction ofthe cheese. Additionally, the technique they developed is only valid on spherical holes;in reality, however, the holes are ellipsoids with substantial eccentricity. It was thereforedeemed that this project would make a novel contribution to the state of the art.

Other attempts exist in the literature to extract holes from photographs of cheese [8],ultrasound [9] and even MRI [10]. However, due to the different imaging equipment havingdifferent capabilities and limitations, the techniques developed with these two methodsare not directly applicable to X-ray imaging.

2.2 Cheese specifications

The cheese used for the purposes of this project was a sample of Emmental cheese providedby Mifroma, a Swiss cheesemaker.

It was aged for 3 months under commercial conditions and X-rayed at Cheyney Design.This sample has dimensions of 345mm × 265mm × 55mm, though only a section of it(210mm x 100mm x 55mm) was examined by X-ray, for practical purposes. The techniquesdeveloped however should, in principle, be applicable to cheeses of any size.

9

Figure 2: Left, a projection of a block of cheese imaged with visible light. Right, aprojection of a larger block imaged with X-rays.

2.3 X-ray imaging principles

Most materials in everyday life (including cheese) are opaque to visible light. This meansthat when they are struck by photons of visible light, they will tend to absorb all, ornearly all, of their photons near their surface. This is why holding a block of cheese infront of a light is not in itself sufficient to identify whether there are any holes withinit - the material’s attenuation coefficient to visible light is simply too high to let anyappreciable number of photons through its thickness, leading to a uniformly dark image(see Figure 2, left).

Clearly this is not a viable technique to locate holes inside a cheese. However, whilecheese is opaque to visible light, it is much more transparent to other frequencies of theelectromagnetic spectrum.

X-rays are the region of the light spectrum with wavelength 0.01-10 nm (or energies∼100 eV - 100 keV per photon). This is a far higher energy range than visible light,and crucially most biological materials (including cheese) are much more transparent tothis range of frequencies than to visible light. This leads to well-known applications inmedicine, where the relative transparencies of different types of tissue can allow for aphotographic image of the patient to be constructed by shining X-ray radiation throughthem. This can also be used in cheese, which is mostly (but not entirely) transparent toX-ray radiation, allowing an image of the holes present within it to be obtained (Figure2, right).

The attenuation of radiation through a material is given by the Beer-Lambert law [11],

n

n0

= exp (−µx)

where n is the number of photons exiting the material, n0 is the number of photons

10

entering the material, µ is the material’s attenuation coefficient1 and x is the thickness.The X-ray attenuation coefficient for cheese (approximated by that of water [12]) per

unit volume is larger than that of air by a factor of approximately 103, which confirmsthat it should be easy to draw contrasts between solid cheese and holes in an X-ray image.

X-rays are typically produced by X-ray generators. These consist of vacuum tubeswhere a hot cathode releases electrons, which are accelerated by high voltages (50 kV inthe model used in this report[14]) onto an anode. The energy released by the electrons’collision with the anode is in the form of X-ray radiation, whose energy is limited bythe voltage (ie. the radiation produced by this setup cannot exceed 50 keV). The outputdistribution of X-ray frequencies is a combination of the generator materials’ characteristicspectra and a smooth curve, known as bremsstrahlung [13]. To prevent the emission ofradiation at low energy, this output is then filtered by a thin layer of material (berylliumin the generator used). This is known as beam hardening.

The cheese has a relatively small attenuation coefficient with respect to X-rays (µ ≈5 × 10−2cm−1 at the X-ray energy used) which means that cheeses with thickness of upto several centimetres can be usefully imaged with a reasonable amount of light passingthrough. A quick back-of-the-envelope calculation shows that the contrast of a hole ofdiameter d cm within a cheese of thickness t is (assuming air within the holes to have noattenuation)

Contrast =(brightness through hole centre)

(brightness through region with no holes)

=exp(−µ(t− d))

exp(−µt)= exp(µd).

This means that even a relatively small hole with diameter d = 1cm presents a contrastratio of roughly 1.05:1, which should be noticeable when the hole is placed in a piece ofotherwise uniform cheese. This contrast can also be artificially enhanced to make theholes appear more obvious.

2.4 Reconstruction techniques

Several 3D reconstruction techniques exist to find the internal structure of a 3D objectfrom 2D projections. Many are widespread, being used very commonly in medicine toperform CT scans.

It is possible to take a different approach, focusing instead on finding individual holesin a 2D projection, and then combining them with different projections to reconstruct a

1The coefficient µ is depends on the sum of several nuclear interactions and therefore has a complexrelationship with frequency - nonetheless it can be approximated to be constant across the X-ray spectrumfor the purposes of this report.

11

3D image. These are also investigated.

2.4.1 Traditional methods

Computed tomography (CT) is a very widespread and well-established technique to re-cover a 3D reconstruction from single 2D X-ray projections. Since the technique’s devel-opment in the 1970s, it has been used in hospitals around the world and is now part ofmany fairly routine medical procedures.

Filtered backprojection [15] is the principal mathematical technique by which CTworks. It uses a mathematical technique known as the Radon transform to attempt toinvert the projection process, using many (on the order of 100) individual 2D projectionsof the area of interest, from many angles around the object (approximately a 180º range).Each of these is then projected back along the line where the points with high attenuationcould lie in a 3D image, and filters are applied to reduce any of the resulting “streaking”artifacts. With large numbers of projections and angles, this can lead to a very accuratereconstruction which is appropriate for use in sensitive medical applications like surgeryor radiotherapy.

A different technique which is increasingly being used to recover 3D images in medi-cal CT is iterative reconstruction [15]. There are many different approaches to iterativereconstruction, but fundamentally they attempt to build a 3D reconstruction by findingthe most likely attenuation coefficient for each voxel by iterative optimisation techniques.This is increasingly popular in medical devices but it typically requires much more com-putational power than backprojection to obtain a reconstruction.

2.4.2 Hough transform-based methods

A different approach could be to find the locations of individual holes within 2D projec-tions, and combine the information of two projections to find a three-dimensional position.This would involve finding ellipses in a two-dimensional image.

There are several techniques to find ellipses within 2D images [16]. Of these, a popularmethod is the Hough transform.

The Hough transform can find imperfect instances of a desired feature within an image[17, 18]. Typically, this is used in computer vision to find straight lines or circles withinan image. More recently, techniques extending the Hough transform to find other featuressuch as ellipses in 2D images have been investigated [18].

The memory usage involved with the Hough transform is heavily dependent on the di-mensionality of the features being matched. The complexity of the transform is O

(nd−2

),

where n is the number of pixels in the image and d is the dimensionality of the featuresearched for.

12

Figure 3: Left, a traditional computer vision setup with two cameras (or a rotatingcamera). This is equivalent to the setup on the right, where the cheese is rotated instead.

2.5 Geometric principles and three-dimensional reconstruction

Standard computer vision techniques can be used to find the 3D position of points from2D images at different angles.

Any point in an image at one angle must lie along a line in the other image known asthe epipolar line. This is the image in the other camera of the line joining the point andthe optical centre of the other camera. With a known point in one image and knowing thepositions and internal parameters of the different cameras, it becomes possible to narrowdown the position to a single straight line in the other image2, making it easier to findpoint matches. From the calibration of the two cameras it becomes relatively easy to thenfind the 3D positions of point matches.

Figure 3 shows two imaging setups. Observing a cheese using a stereo rig of twocameras at opposite angles is in fact equivalent to observing the cheese with a singlecamera and rotating it instead. This is a cheaper and easy setup and therefore preferable,while maintaining all the stereo vision techniques described above.

Real sensors and cameras operate under perspective projection, where the 3D imageof the cheese is somewhat distorted along its thickness - the part of the cheese which isnearer to the X-ray source appears larger than that further away. If the cheese is placedfar enough away from the X-ray source (and so its thickness is negligible compared to thedistance from the source) it becomes possible to approximate the perspective projection toan orthographic projection [19]. This is mathematically easier and requires less calibrationthan perspective projection, so it would be useful if this is a good approximation.

2In fact, due to nonlinear distortion from the sensor and the X-ray generator not being a point source,it is narrowed down to a narrow region around a line instead of a single line.

13

Volume (mm2) 150×100×35Hole proportion (by vol.) 20%Hole radius (mm, mean) 8Hole radius (mm, std. dev) 2Stretch (mean) 1Stretch (std. dev) 0.25

Table 1: Parameters used in generating holes for the 3D-printed proxy.

3 Techniques implemented

3.1 3D printed proxy design

In any cheese, the distribution of holes is initially unknown and cutting the cheese toattempt to determine that will tend to deform the holes themselves. Therefore, it isimpossible to obtain an accurate ground truth to which any attempted reconstructioncan be compared.

To address this, a “fake cheese” proxy was designed. It was designed to have similarhole dimensions and statistics to real cheese, and to thus be a good model for real cheeseto an X-ray scanner. A real piece of Emmental cheese was inspected to find the size anddistribution of the holes. The distributions obtained were then randomly sampled with aMATLAB program to create a unique cheese model, which was then 3D printed.

Any reconstruction could then be compared to the CAD “ground truth” to find andrectify any errors.

3.1.1 Proxy design considerations

A pack of store-bought Emmental cheese was inspected to find the distribution of positionsof the holes in it, as well as the distribution of their sizes. This was then used to informthe design of the proxy cheese.

The parameters for the proxy cheese designed are shown in Table 1. The centres ofholes were assumed to be uniformly distributed throughout the volume of the cheese,taking up approximately 20% of the volume. Intersecting holes were allowed3.

The holes themselves are well approximated by ellipsoids. The ellipsoids were codedin MATLAB to be spheres with different levels of stretch along each axis. The radii of theholes were set to be normally distributed, with a mean of 8mm and a standard deviationof 2mm. The stretch along each axis was also normally distributed around 1, with astandard deviation of 0.25. If anything, this somewhat overstates the eccentricity of theellipsoidal holes in real cheese - this was not thought to be a significant problem.

3This is a slight deviation from what was observed in real cheese samples. In fact, real holes in cheesetend to have a thin layer of cheese separating them, even if they would overlap. This was thought to haveno impact on imaging.

14

Figure 4: A wireframe view of the CAD model of the proxy.

Another design consideration for the proxy was its size and the size that the manufac-turing process could allow. A typical industrial block of cheese measures approximately300mm x 200mm x 70mm. However, the maximum dimensions of any object that couldbe printed on the 3D printers available for use was 223mm × 223mm × 205mm [20].Besides, it was found that printing a proxy with the same volume as an industrial blockof cheese would take more than one spool of 3D printer filament, raising other practicalproblems. Therefore, it was finally decided to scale down the dimensions of the proxy bya factor of 2 along each dimension.

A MATLAB script was written to randomly generate hole positions and parameters forthe proxy based on this specification. These were then imported into a CAD programmefor design of the physical object.

3.1.2 3D printing

OpenSCAD [21] was then used to develop a CAD model of the proxy, which was then 3Dprinted. A top wireframe view of the CAD model can be seen in Figure 4.

Most 3D printers work by a process known as additive manufacturing : a thin filamentof an appropriate type of plastic is heated to its melting point (typically above 200 ºC)and deposited onto a surface, where it quickly hardens again. When filament is addedbeside already-manufactured filament, it fuses, creating a single solid mesh structure.Thus meshes of filament can form solid layers, and new layers can be printed on top ofexisting ones to produce 3D structures.

One benefit of this process is that it makes it possible to create fully-enclosed voids.It is desirable for the proxy, much like real cheese, to contain holes which are completelysurrounded by material and not visible to the naked eye. These could now be manufac-tured.

Additive 3D printing also introduces a few restrictions and artifacts in the manufac-turing process. The regular mesh created by the 3D printer is visible and can create

15

Figure 5: Left, the CAD model of the 3D printed proxy. Right, a photograph of the finalproduct.

noise in the X-ray measurements. Additionally, the printing process draws the contoursof shapes (such as holes or the cheese’s edge) before filling it in. This creates noticeableartifacts around the edges of holes and in their X-ray images, which must be taken intoaccount.

The 3D printer used was an Ultimaker 2 [20] present at the Department of PlantSciences, University of Cambridge. The filament used was heated to 215 ºC and had adiameter of 100 µm, the minimum allowed by the printer, to reduce mesh artifacts whileimaging as much as possible.

The material used for printing was polylactic acid (PLA), a standard 3D printer ma-terial. This is a biologically-derived plastic normally synthesised from sugarcane, and itis non-toxic and biodegradable [22]. As it is an organic compound, it was assumed thatits X-ray attenuation coefficient is similar to that of real cheese.

3.1.3 Image preprocessing

The resulting 3D-printed proxy (shown in Figure 5) subjectively looks very similar tocheese, but does present a few differences in its X-ray images. Due to the additive man-ufacturing process, the regular plastic mesh which the solid parts of the proxy are builtfrom is visible in the X-ray images: this presents a significant source of high-frequencynoise, which has the potential to add confusion to the results. Additionally, the entireX-ray image appears very dark and it is difficult to make out any higher-order featureswithin the cheese (such as holes). To counteract this, all pixel values were normalised tohave a maximum at 255, which made the image much brighter with no loss of information.

To eliminate the high-frequency mesh from the image, filtering was used. Both aGaussian filter and a median filter were considered for use. As shown in Figure 6, theycan both effectively blur out the mesh, with the median filter providing marginally bettersubjective sharpness than the Gaussian filter, and slightly slower performance. As a result,the median filter was chosen.

16

Figure 6: Left, the original image. Centre, filtered using a Gaussian blur, and right, usinga median filter.

Figure 7: The original histogram of an X-ray image of the proxy cheese, taken withCheyney equipment. Right, the new histogram that the image’s intensity was adaptedto.

3.2 Histogram adaptation

Figure 7 shows the histogram of an X-ray image taken of the proxy. This shows a promi-nent peak around the pixel values of 50-70, and relatively few pixels outside those values.Subjectively, the effect is to cause the output image to look washed-out and uniform, anddifficult to find holes in (especially smaller and shallower ones).

Therefore, histogram adaptation was used to make any holes present in the imagemore prominent. Histogram equalisation was originally applied to the image, increasingthe contrast between holes and regions without holes, with results as can be seen in Figure8.

However, it was noticed that this technique had its own problems. A simple equali-sation emphasised small changes in brightness in uniform regions (due to the texture ofthe proxy or even noise) and flattened brighter regions with one or several holes present.

17

Figure 8: A detail of an X-ray image of the proxy. Left, the original image output fromthe X-ray. Centre: the same detail after histogram equalisation. Right, the same detailwith custom histogram adaptation.

This made it difficult to distinguish between overlapping holes, and occasionally led tospurious holes being found in areas with no holes at all.

Therefore, a different form of histogram adaptation was designed. A section of aGaussian curve was selected as giving empirically the best results, with good contrastbetween holes and background and between overlapping holes.

This is shown in Figure 8. The form of the histogram chosen was found, by trial anderror, to be best at

H(i) ∝ exp

(− i2

20000

)where H is the height of the histogram at a given pixel intensity i. This was divided into64 bins which each pixel was assigned to. This was found to give the best (lowest-cost)matches to the objective function described in section 4.4 below.

3.3 Traditional techniques

Common techniques to extract 3D reconstructions from 2D projections, such as filteredbackprojection and iterative reconstruction, are widely used in medical imaging to obtainhighly accurate 3D images. These techniques, however, routinely require a large numberof individual projections from a large range of angles.

Unfortunately, filtered backprojection is not an effective technique to obtain a re-construction of the structure of the cheese under the constraints required in this project.Figure 9 shows simulated reconstructions (from 1D projections) of an X-ray slice of cheese,operating under the constraints of the problem. With either a limited number of projec-tions or a limited angular range of projections, this method fails to provide an image of

18

Figure 9: Left to right: An X-ray projection of cheese; attempted reconstructions fromfiltered backprojection using (a) few (10) 1-dimensional projections of the image and (b)limited angles (within ±50°).

any quality and must therefore be discarded.Iterative reconstruction was judged to proceed too slowly to be effective (estimating

several hours on a standard PC to converge) and was terminated. Nevertheless, it isexpected to suffer from the same problems.

A reason for the poor performance of backprojection is that it does not assume any-thing about the internal structure of the cheese. This leaves it with insufficient informationto reconstruct an accurate picture from just two projections. However, in reality muchis known about the internal structure of the cheese: it is an approximately uniform sub-stance with constant attenuation, with ellipsoidal holes of varying sizes within it. Differenttechniques that use this information can therefore be expected to perform better.

3.4 Hough transform-based methods

A different proposed approach is to find individual holes in 2D projections, and thencombine different projections to find a full 3D reconstruction.

Section 2.4.2 introduced the Hough transform, which attempts to find imperfect in-stances of a given feature within a 2D image. This was applied to search for holes in anX-ray projection.

At first, a circular Hough transform was used. The dimensionality of a feature Houghtransform is O

(nd−2

), where n is the number of pixels in the image and d is the dimen-

sionality of the feature; since circles have d = 3 dimensions (given by the x- and y-positionof the centre, plus the radius) this reduces to O (n), which computes very quickly on anormal workstation. An implementation of the circular Hough transform is a MATLABlibrary function and this was used. The left-hand side of Figure 10 shows the result ofthis attempt. While some of the holes in the cheese are correctly identified, the algorithmfails to find many, and finds other spurious matches and errors. This is believed to be dueto the fact that the holes in the X-ray projection are in fact ellipsoidal, and only poorly

19

Figure 10: Left, the output of a circular Hough detector, right, the output of an ellipticalHough detector, decimated so it could run in less time.

approximated by spheres.A different approach that can be taken is computation of an elliptical Hough transform,

finding ellipses in the projection directly. This should provide a better match to the holes.However, ellipses have d = 5 dimensions (the same three as circles, plus an eccentricityterm and an angle term) and therefore the complexity of an elliptical Hough transform isO (n3). This makes it impossible to evaluate this transform directly on a normal computer,as it requires about 1TB of memory for a test image. Data-reduction techniques wereemployed to make this possible, including subsampling and edge detection to speed upthe runtime.

The results of this are shown on the right-hand side of Figure 10. It appears thatthe scale of data reduction necessary made it impossible to obtain a reconstruction ofany accuracy from the algorithm, and in fact it seems to perform worse than the circularHough transform. Therefore it was decided to abandon any further investigation of Houghtransforms, and to seek different methods to find ellipses in 2D images.

4 First-principles ellipse fitting

4.1 Motivation

All the techniques described in the literature and attempted above proved inadequate forthe problem of finding holes in cheese. Therefore, a novel technique was developed, usingan iterative approach to optimise an objective function from first principles. This methodattempts to fit a number of 2D ellipsoidal kernels to an image, each of which is supposedto correspond to a hole. The final positions and characteristics of the optimised kernelsshould match those of the real holes in the X-ray projection.

4.2 Description of technique

A set of ellipsoidal kernels is specified within an image. As an initial “guess”, a relativelylarge number of kernels is initialised in the image, equally spaced and with identical size

20

Figure 11: A contour plot of the kernels used for optimisation. This one is spherical withr = 50 and no eccentricity.

and no eccentricity. The kernel positions and sizes are then optimised asynchronously tominimise an objective function. Kernels are moved individually until each reaches a localminimum in the objective function. This is repeated for all kernels in the image.

To ensure that there are a correct number of holes in the final converged image, ateach iteration of the convergence sequence a set of tests are applied to ensure that allkernels are fit to a hole, and are not caused by overfitting the data. Any kernels that failthis test are eliminated from the optimisation procedure.

This process is then repeated until all the kernels converge, or the change in theobjective function is negligible.

4.3 Ellipsoidal kernels

The fitting technique relies on fitting ellipsoidal kernels onto the data given. Since theyneed to be fitted onto a single projection, the kernels themselves must be 2D projectionsof ellipsoids - brighter in the (thicker) centre and tapering off towards the sides, as shownin Figure 11.

Figure 11 shows a contour plot of a spherical kernel C with radius r = 50 used in theoptimisation process. The intensity of such a kernel (with radius r) is given by

C(x, y) = max

{0,

√r2 − ‖(x, y)− (r, r)‖2

}.

This comes from a simple application of Pythagoras’ theorem. The kernel can be thoughtof as a projection of a sphere, centred around the point (r, r). The intensity of the kernel

21

Figure 12: The C(x, y) function for a spherical kernel is simply given by the height from(x, y) to the surface of the sphere.

at any position is proportional to the vertical thickness of the sphere from that position.Any point outside the sphere is outside the kernel, so its intensity should be set to 0.

Figure 12 provides a visual representation of this argument. It is clear that the inten-sity of the kernel can thus be obtained from Pythagoras’ theorem.

When the kernel is not spherical, it is still relatively easy to compute. Any ellipsoidcan be described as a sphere that has been stretched along the x-axis by a factor of sxand along the y-axis by a factor of sy. To find the intensity of the ellipsoid at any point,it is therefore relatively easy to project it back onto a sphere and proceed from there,

C(x, y) = max

0,

√r2 −

∥∥∥∥( x

sx,y

sy

)− (r, r)

∥∥∥∥2 .

This is the form of kernel that was used in this investigation.

4.4 Specification of an objective function

Optimisation is carried out by an iterative method, changing the positions and parametersof the kernels to minimise an objective function. Therefore the choice of objective function,and the terms within it, is very important to achieve a good fit.

The initial objective function used was simply the rms error between the projectionand the 2D reconstruction, with a fixed number of holes. An example of a stable fitobtained by minimising this objective can be seen in Figure 13.

The first hole to be fitted seems to attempt to fit all the holes present in the image,and any subsequent holes which are added result only in the fitting of the intersectionsand additional noise. This is clearly unphysical and does not provide any information onthe holes present in the image. This is relatively stable, and small deviations from thisequilibrium only result in the cost function growing.

22

Figure 13: Left, a detail of an X-ray scan of the 3D-printed proxy (with dc componentremoved). Right, the reconstruction obtained by a simple least-squares objective function.

Coefficient Component Value Threshold Valuea major axis 2 a0 3b number of holes 0.6 None N/A

Table 2: The coefficients chosen for each of the terms in the objective function. Thesewere chosen by trial and error and led to seemingly the best fits.

4.4.1 Description of terms in objective function

Additional terms were added to the cost function to prevent such incorrect fits.The terms added penalised for several common misfit artifacts observed while trying to

find an appropriate form for the objective function. It was found that “misfit” holes weretypically very large (covering multiple holes) and had large eccentricity, as they tendedto fit holes which were next to each other. To address this, it was decided to add a termto the cost function, adding a penalty to holes with a large major axis to prevent misfits.Another term was also added, representing the quantity of kernels in the image, to avoidoverfit:

CMA =∑i

max (li,1 − a0, li,2 − a0, 0)2

Cn = n,

where li,1and li,2 are the lengths of the axes of hole i. CMA penalises the square of themajor axis (the longer axis of the elliptical kernel) beyond a0, applying a quadraticallyincreasing cost (see Figure 14). The value of a0 can be found in Table 24. Cn avoids overfitby placing a penalty on the overall number of holes fitted in the projection, ensuring thatany “overfit” kernels which do not fit any real holes are eliminated.

4Lengths are expressed in multiples of 100 pixels, for ease of manipulation.

23

Figure 14: The change cost added into the function from the constraint on major-axislength, with the change in the length of the major axis itself.

Figure 15: The components of the cost function over the course of the optimisation of setof kernels.

Therefore the added terms apply no extra cost up to a certain threshold (which almostall true holes within the cheese satisfy) and then add a cost which increases linearly withdistance from the threshold. The overall objective function is therefore

C = RMSE + aCMA + bCn,

where a and b are scaling coefficients that place the correct weight on each component toreach a good fit. The coefficients chosen for the cost function can be seen in Table 2.

Figure 15 shows the change in the cost function. Typically this choice of cost functionresults in no added cost in the converged fit from the added parameters to the function,as the characteristics of real holes place them below the thresholds chosen for added cost.However, these can add fairly significant costs in intermediate steps of the optimisation,

24

Figure 16: The reconstruction of the same area of the scan obtained using the final versionof the objective function.

and prevent the establishment of local minima which do not correspond to the groundtruth.

4.5 Optimisation method

After the initialisation of the kernels, a very simple iterative technique is used to minimisethe objective function. This is specified as follows:

1. Select a kernel

2. Translate the kernel slightly to the left and the change in the objective is measured.

(a) If the error decreases, (2) is repeated until the objective no longer decreases.

(b) Otherwise, it is moved to the right until the objective increases.

3. Repeat same procedure as (2) for y-axis translation, intensity, and stretches alongthe x- and y-axis.

4. Return to step (2) and repeat until the rms error remains constant between itera-tions.

5. Select another kernel and return to (2) until convergence.

Each optimisation is done on the basis of the results of the previous one - that is, optimisedpositions, stretches etc. are stored and used to compute the optimisation for the nextparameter. This method was implemented using MATLAB [23] as a platform.

Rotation of the kernels was not included in this method, after it was found that itslowed down convergence substantially (by adding a degree of freedom) without increasingthe accuracy of the final fit.

25

Figure 17: Comparison of the time-evolution of the cost function for asynchronous andsynchronous update rules.

4.5.1 Choice of asynchronous update

Using a synchronous update rule was also considered. Under this alternative scheme, eachparameter is updated based on the others’ values at the previous iteration, and only afterall parameters have been optimised are the new values stored.

Figure 17 shows the convergence of the objective function under the two differentschemes with a test image of part of the proxy. The synchronous update rule clearly seemsto converge somewhat slower, even though both techniques eventually reach essentiallythe same solution.

Additionally, a synchronous update rule no longer guarantees that the cost does notincrease at any iteration, while the asynchronous rule does. This is seen quite clearlyin Figure 17. This can cause problems in the optimisation, including infinite loops be-tween two states which never converge. These were in fact observed in some test images.Therefore, it was decided that an asynchronous update was the better choice, both forconvergence speed and practical reasons.

4.6 Determining the appropriate number of kernels

The number of holes in a projection is not known in advance. This can cause difficulties inchoosing the appropriate number of kernels to fit to each projection - too few, and severalholes will not be detected; too many, and overfitting can occur, leading to phantom “holes”which do not match anything, or several kernels fitting the same hole.

To prevent this, the algorithm is initialised with a number of kernels deemed signifi-cantly larger than the number of holes likely to be present in it. A number of methods arethen implemented to ensure that any additional holes can be eliminated. Any kernel whichis deemed to be “extra” in this way is discarded and removed from the reconstruction,

26

leaving all the others unchanged.A term is added to the cost function representing the number of kernels to be fitted.

This scales linearly with the number of kernels, adding 0.6 (a small but not insignificantcost) to the cost of a given fit per kernel present in it. At each iteration, kernels areremoved one by one and the fit is attempted without each kernel present. If this resultsin a better (lower-cost) fit, then the kernel is discarded.

If any of the holes are too small or too faint, they are also deemed to be overfit. Atest for this is performed and any kernels with brightness or size below defined thresholdsare similarly removed.

Additional kernels may be removed at the 3D fitting stage if no corresponding kernelsare found. The procedure for this is explained in detail below.

5 Three-dimensional fitting

Once holes have been found in 2D projections, each hole in one projection must be matchedto the corresponding hole in another projection. Three-dimensional information must thenbe extracted from these matches to obtain a full 3D reconstruction.

5.1 Matching projections to each other

The first step in this process is finding matches between holes in 2D projections.Figure 18 shows X-ray projections at two different angles of a detail of the 3D printed

proxy, and the corresponding detected 2D positions of the holes (found as describedabove). The holes on the right of the images are known to be near the top surface ofthe proxy, while those on the left are nearer to the bottom. This can be seen in theprojections, as there is a great deal of relative movement of the holes between them. This3D information now needs to be used to extract the positions of the holes themselves.

Note also that in both images, a hole is detected in the top-left corner. This is due toa different hole being cropped out to find a sample image.

5.1.1 Determination of similarity scores

The first step taken is to match each detected hole in one projection with its correspondinghole in the other. This is done by calculating a dissimilarity score Sij for each pair ofdetected holes i and j in different projections. A lower score indicates similar holesaccording to number of measures, which are therefore more likely to match to each other.Therefore, if N holes are detected in one projection and M holes in another, an N ×Mmatrix S stores the dissimilarity scores of each.

The dissimilarity score Sij is obtained from a number of different metrics, including theposition of the centre (x, y), the area of the holes, their eccentricity and their brightness

27

Figure 18: Top, two X-ray images of a detail of the proxy at two different angles. Bottom,the reconstructions of the hole positions from the projections.

(or thickness along the direction of the projection). These component scores are given by

Sx(i, j) = max {|xi − xj| −∆x, 0}

Sy(i, j) = max {|yi − yj| −∆y, 0}

SA(i, j) = |li,1li,2 − lj,1lj,2|

Secc(i, j) = max

{∣∣∣∣ li,1li,2 − lj,1lj,2

∣∣∣∣ , ∣∣∣∣ li,2li,1 − lj,2lj,1

∣∣∣∣}Sint(i, j) = |Ii − Ij| .

The scores Sx and Sy correspond to the absolute distance between the centres of theholes, with some tolerance ∆x and ∆y5. The tolerance ∆x is deliberately chosen to be

5This means that any pair of holes within distances ∆x and ∆y of each other have no distance costs,

28

Coefficient Component Valueα Hole area 100β Eccentricity 100γ Intensity (depth) 50

Table 3: The coefficients corresponding to different components of the dissimilarity scoreS. Intensity has a relatively low coefficient because there is only a weak correlationbetween intensities from different angles.

large, as it represents the epipolar constraint. According to this constraint, a point in oneimage must lie along a line (known as the epipolar line) in the other, representing a raybetween the camera’s optical centre and the point itself. In the geometry of this setup,the epipolar line is purely horizontal (due to the camera’s rotation being purely in thex-axis) and therefore the range of ∆x must be large. The epipolar line can, however, benarrowed down to a relatively short line segment representing the thickness of the cheese.

In practice, nonlinear distortion means that the correspondence is found along a thinband around the epipolar line rather than the line itself. This is given by ∆y.

The area and eccentricity components are expected (and seen) to be fairly constantbetween different projections, and therefore no tolerances are added. The final score S isgiven by

S = Sx + Sy + αSsz + βSecc + γSint

where α, β and γ are coefficients added on to the area, eccentricity and intensity terms toreach a similar magnitude to the x- and y-components.

The intensity coefficient γ is given a relatively small value, as the observed thicknessof a hole can vary substantially between projections. The coefficients used can be foundin Table 3.

Figure 19 shows a visual representation of the S matrix in this case, with darker pixelsrepresenting a lower dissimilarity score.

The correct matches here are quite distinct from incorrect matches, with each correctmatch for each hole scoring several times lower than the lowest incorrect match. Thegeneral diagonal trend along the matrix is to be expected, as the holes are in generallysimilar positions in both projections and the kernels are always generated in the samepositions (eg kernel 1 is always initialised near the top-left corner, etc).

5.1.2 Determination of correct matches

In this instance, the correct matches are all quite clear from the matrix. However, theremay be other cases where matches may be ambiguous, where more holes are detected in

and the cost increases linearly with distance from those thresholds.

29

Figure 19: Visual representation of the matching costs between the kernels in the two testimages. Brighter cells represent higher costs.

one projection than in another, or where two holes in one projection seem to match tothe same hole in the other.

Therefore, an algorithm to provide the best guess for a correct fit was devised.Figure 20 shows a flowchart describing the algorithm used. The global minimum Sij

of the matrix S is found, and the holes (i, j) are deemed to be a match to each other. Toavoid any other matches to the same holes, all elements in row i and column j of S arereplaced with very large values. This is repeated as many times as necessary, with newminima (and matches) being found at each iteration.

A threshold score is set, and any pair of holes scoring above this threshold is rejected.This threshold was calibrated, but is set relatively high to allow for the possibility of errorsin the 2D matching stage. If the minimum found is above this threshold, the algorithmdeems that all matches have been found and exits. This acts as another mechanism ofnoise rejection, removing any “overfitting” holes which do not correspond to any holesin the other projection, and allowing there to be different numbers of kernels in eachprojection, M 6= N .

5.2 Recovery of 3D position and parameters

Once the holes have been matched to each other, it is possible to combine the informationfrom the two projections to achieve a three-dimensional reconstruction.

Figure 21 shows a diagram of a hole being projected onto the detector surface fromtwo beams, each at 45º to the surface. The coordinates of the first projection (x1, y1) are

30

Figure 20: A flowchart showing the process by which kernels are matched in two projec-tions, from the dissimilarity scores in the matrix S.

Figure 21: The positions of the projections of the centre of a 3D hole will diverge by anamount proportional to the height of the hole.

31

given by

x1 = X − Z tan(θ)

y1 = Y

and those of the second projection (x2, y2) are

x2 = X + Z tan(θ)

y2 = Y.

Therefore the 3D position of the centre of the hole can be simply obtained with

X =x1 + x2

2

Y = y1 = y2 = y1, y2

Z =x2 − x12 tan(θ)

=x2 − x1

2,

where y1, y2 is the arithmetic mean of y1 and y2 (to reduce noise). As long as the X-raysource is aligned correctly relative to the centre of the hole, this is true regardless of thedistance to the source.

Finding the dimensions of each hole is somewhat more susceptible to the geometryof the system. If the X-ray source is assumed to be very far away (ie, all incoming raysare parallel and the projection reduces to an orthographic projection) the length in eachdirection can be given by

lX =(lx1, lx2

)cos(θ) + cL

(I1, I2

)sin(θ)

lY = ly1 = ly2 = ly1, ly2

lZ =(lx1, lx2

)sin(θ) + cL

(I1, I2

)cos(θ),

where cL is a scaling factor (measured from known thicknesses) relating the brightness ofa hole in the X-ray projection to its thickness6.

However, this is only a good approximation if the distance between the X-ray generatorand the surface of the cheese is much greater than that between the surface of the cheeseand the detector. If it is smaller, the divergent beams from the generator cause holes atthe surface to appear larger than those near the bottom.

32

Figure 22: Left, a photographic detail of the 3D-printed proxy. Right, an X-ray image ofthe same detail.

5.2.1 X-ray scaling factors

Pixel positions on the X-ray detector (u, v) are linearly related to the true world positionof the projection (x, y) by a scaling and translation operation. This can be expressed as

[u

v

]=

[ku 0 u0

0 kv v0

]xy1

where ku and kv are the scaling factors along the x- and y- directions, respectively, and(u0, v0) is the pixel position of the point (x = 0, y = 0) (which can be set arbitrarily).The factors ku and kv were estimated by inspecting known distances and lengths in theproxy (as shown in Figure 22) which shows the same detail of the proxy (whose lengthsare all known by design) in a photograph and in an X-ray image. It was found that, forthe X-ray detector used, ku = kv ≈ 13.9 px/mm. However, this does vary depending onthe specific X-ray detector used.

Additionally, a scaling factor cL was used above to relate the brightness of a detectedhole in a projection to the hole’s depth in the direction of the X-rays. This is not inprinciple a linear operation: the Beer-Lambert law suggests an exponential variationin contrast with hole thickness, and the added image preprocessing further distorts therelationship. However, a linear approximation was found to perform well in this situation7.A reasonable value cL = 1.8 was selected experimentally, to minimise the reconstructionerror with respect to the proxy.

6In fact, by the Beer-Lambert law, the relationship between brightness and thickness is exponential.Since all the holes are relatively small, however, linearity is a good approximation.

7This is due to the relatively small thickness of the holes, meaning that the first-order linear approxi-mation of the exponential characteristic is relatively accurate.

33

Figure 23: A diagram showing the difference between the true length of a hole and itsmeasured length (under perspective projection).

5.3 Assumption of orthographic projection

The reconstruction is obtained using orthographic projection (ie, assuming that all in-coming rays are parallel). This makes the geometry used for reconstruction considerablyeasier, but it is a simplification and a potential source of error.

The X-ray source can be treated as a point source, and its rays all diverge fromthe source and cross the cheese at different angles before impacting the detector. Thisdivergence leads to holes, especially those near the top surface, appearing magnified withrespect to their real size. A visual representation of this discrepancy is given in Figure23. The magnification is given by

M =lmeas

lreal=lSDlSH

where lSD is the distance between the source and the detector, and lSH is the distancebetween the detector and the hole measured.

In the setup used, the distance between source and detector is lSD = 400mm, and thecheese lies on top of the detector. A hole lying at distance d = 30mm from the bottom ofthe cheese is therefore magnified by a factor of

M =lSDlSH

=lSD

lSD − d=1.081.

34

which leads to a hole appearing roughly 8% longer in both directions , and thus appearingto have a volume (M2 − 1) ≈17% larger than the real volume of the hole8. This is asignificant error, but it is still deemed to be smaller than any of the errors introduced byother sources and therefore acceptable.

Higher order errors are also introduced by this assumption which are more difficult tonegotiate. The 2D kernels used for fitting are all orthographic projections of ellipsoids,and somewhat differ from perspective projections of ellipsoids. The effect was deemed tobe small and difficult to correct for.

5.4 Surface and intersecting holes

The technique described here assumes that all holes are fully spherical and containedwithin the cheese. Therefore, it has difficulty reconstructing holes which do not meetthose requirements, such as holes which are “cut off” and visible from the outside (likethose shown in Figure 22) or even internal holes which intersect others.

While this does lead to potential problems reconstructing the holes present in theproxy and in slices of cheese, it should not present any issues when dealing with wheels ofreal cheese. A wheel itself does not have any exposed holes at the surface, and all of itsholes are entirely internal. This ensures that this technique remains useful for that case.

6 Results and discussion

6.1 Accuracy and efficiency of 2D optimisation

To find the 3D position of holes, their positions in each 2D projections were first found.The optimisation process described above always led to a converged stable equilibrium

of the fitted hole positions in a 2D projection.Figure 24 shows the final converged fit for a top-view X-ray projection of the 3D-

printed proxy. The final fit had an rms error of RMSE = 32.03 (in a range of 0-255, fromthe error in pixel values). All fits made by the algorithm resulted in similar performance,with rms error in the range 25-35.

Many holes are matched perfectly, or nearly perfectly, by the algorithm. It performsparticularly well on fully segmented holes with no overlap with any others in the image,but it can also find accurate reconstructions of holes with notable overlap. The holesnear the top of the image, for example, are matched very well. However, the algorithmdoes perform relatively poorly on holes with substantial overlap and little variation inbrightness between them, such as the holes near the centre of the proxy. This can lead tosome holes missing during the 3D fitting stage.

8The length in the z-direction should not be affected by the assumption of orthographic projection.

35

Figure 24: Left, an X-ray image of the proxy. Right, the 2D reconstruction obtained usingthe algorithm designed.

Holes which are partially outside of the image also present a problem for 2D fitting.Due to the design of the routine, all matched holes must lie entirely within the image.This leads to relatively poor fits near cut-off holes at the sides of the X-ray image. Thisshould not present a problem with real cheese, as a wheel entirely contains all of its ownholes.

Figure 25 shows an X-ray image of the cheese described in Section 2.2, and its finalconverged fit. The holes which are clear to the human eye can be seen to be matched quitewell. There is also a fairly large number of unclear holes, which are matched generouslyby the algorithm, finding more holes than are probably present. Any holes which do notcorrespond to anything are removed later, in the 3D reconstruction stage. The rms errorof this fit is very low, with RMSE = 17.48.

6.1.1 Efficiency of 2D optimisation

The 2D optimisation stage ran very slowly due to the iterative method used. At each stepof the optimisation process, the cost function was evaluated. This involves finding therms error of the reconstruction, which requires rendering of the proposed reconstruction.Reconstructing a proposed fit adds all holes individually and then finds the rms error ofeach pixel, so the reconstruction process of a u × v image has complexity O(uv × N),where N is the number of holes to be fitted to the cheese. This is then looped throughall N holes, and therefore the overall computational complexity of the approach taken for2D reconstruction is O(uv ×N2).

The very large number of loop iterations and the rendering of the reconstruction atevery step made the process run very slowly. The 2D optimisation for a projection of the

36

Figure 25: Left, an X-ray image of a real piece of cheese. Right, the 2D reconstructionobtained.

entire proxy shown in Figure 24 computed in approximately one hour on a normal PC.The need to re-render the reconstruction at every step, and the presence of this step atthe bottom of four nested loops, means that the algorithm spends a large proportion ofits time evaluating proposed fits. It would be possible to speed up execution substantially(if not change the overall complexity) by saving a render for the N − 1 holes not beingoptimised at the current iteration, and then simply adding the proposed fit for the kernelbeing fitted before recomputing the cost.

An even simpler way to reduce running time is by subsampling high-resolution images,which results in large computational savings with a relatively small decrease in perfor-mance. This was done, and as predicted, subsampling by a factor of 2 along each axisreduced the running time of the optimisation by a factor of approximately 4.

Due to theO(uv×N2) complexity of the optimisation process, there are also significantexpense savings to be had if the image can be segmented into smaller sub-images, andeach sub-image only uses a fraction of the N kernels. This is not straightforward to do asmany holes overlap with each other and a naive segmentation would cut holes across thesub-images and result in incorrect fits. It is, however, possible to identify lines along whichthe image can be segmented (such as the divide between the upper and lower sections ofthe fit in Figure 24) and perform optimisation along those.

Another area where changes could be made to lower the running time of the process(and bring marginal improvements to the fit) is the optimisation technique itself. Thealgorithm as implemented uses a fixed step size to translate and stretch kernels around theprojection to minimise the error. Varying the step size based on the previous change couldresult in many unnecessary iterations being skipped and a much faster optimisation perkernel [24, 25]. As the derivative of the objective function with any of the parameters being

37

Figure 26: Left, the sample image used by Kraggerhud et al. Right, the reconstructionobtained by them (reproduced from [7])

used is not apparent, derivatives must be computed numerically [26] or a “derivative-free”optimisation method should be used instead [27].

It was also found that the final converged fit showed some dependence on the startingconditions for the optimisation. Varying the spacing of the initial kernel setup, or offset-ting the kernels, led to slightly different converged fits (for example, fitting or skippingholes that a different starting setup did not do). All converged fits nonetheless presenteda similar rms error.

6.1.2 Comparison with past results

It is difficult to directly compare the results obtained in this report to other resultsavailable in the literature, as nobody has attempted to find the three-dimensional locationof holes in cheese through X-ray imaging.

However, Kraggerhud et al. [7] attempted to use X-ray imaging to monitor the growthof holes in cheese over time. They proposed a method to find holes in individual 2-dimensional X-ray projections of cheese. This started from the assumption that holes ina single X-ray projection are well-approximated by uniform circles. They then scannedthe image with a set of uniform circular templates of different radii to obtain a cross-correlation image for each [28]. They then took the maximum cross-correlation valuefor each pixel, thresholded the image and found the centroids and radii of the resultingpatches as the radii of the holes within the cheese. The results they obtained from asample image (reproduced from the paper) are shown in Figure 26.

This is clearly a different approach from that taken in this project. Apart from thedifferent optimisation technique used, the 2D matching process used in this project didnot assume uniform circular holes, but rather used kernels which were projections of anellipsoid and allowed them to deform.

The converged 2D fit obtained for the same data used by Kraggerhud et al. usingthe algorithm developed in this project is shown in Figure 27. This subjectively seemsto perform somewhat better than Kraggerhud et al.’s system, finding several holes which

38

Figure 27: Left, the sample image used by Kraggerhud et al [7]. Right, the reconstructionobtained by the algorithm developed.

Figure 28: Left, a detail (in wireframe view) of the ground truth for the proxy showingsurface holes; right, its reconstruction.

their technique did not correctly identify. It is still not perfect, and tends to overfitsomewhat and finds more holes than are truly present in the image. This is to an extentintentional in the fitting process, however, as these phantom holes can later be removedin the 3D matching stage with reasonable accuracy and relatively little inaccuracy9. Thus(as far as the two methods are comparable) the method employed for two-dimensionalmatching here performs as well as, or better than, the state of the art.

6.2 Three-dimensional fitting

The two-dimensional fits obtained from each projection were then matched to obtain athree-dimensional reconstruction.

Figure 28 shows a three-dimensional reconstruction of a detail of the 3D-printed proxy,using the algorithms detailed above. Although there are some errors in the reconstruction

9It would not be helpful in this instance to compare the rms error of the two reconstructions, as thematching proposed by Kraggerhud et al. does not explicitly produce a reconstruction.

39

Error type rms errorCentre location 4.5mmSide length 12%Hole volume 19%

Table 4: The rms magnitudes of the reconstruction errors observed in the reconstructionof the proxy.

of each hole, it is clearly a fairly faithful reconstruction of the original.The spurious hole which has been identified in the top-left corner of the reconstruction

is in fact an artifact due to cropping the 2D images: part of a larger hole remainedpresent and was matched (as is visible in Figure 18, which shows the projections of thesame detail). Two real holes which are present on the left, on the other hand, are notreconstructed on the right. This is due to the nature of those holes: one is almost entirelyoutside of the proxy and is invisible in X-ray images, and the other has a very higheccentricity and is only visible in one of the projections. The other holes, with goodvisibility in both projections, are reasonably reconstructed.

The reconstructed position of the centre of the holes presented an rms error of 4.5mm(and the error seems to be approximately equal along all axes). This leads to very goodpositioning of all the holes, and agreement with the ground truth.

The quality of reconstruction of lengths and volumes about each axis of the holes wassomewhat worse. The rms difference between the actual and reconstructed length of eachaxis averaged 12%, while the rms difference between actual and reconstructed volume was19%. These errors are all shown in Table 4. However, this overstates the true error in thereconstruction. This relative inaccuracy is largely due to the inaccurate reconstructionof surface holes, which are reconstructed to be somewhat smaller than they are in theoriginal due to the low contrast measured by the X-ray. Therefore in reality this error issomewhat immaterial.

It must also be noted that the traditional technique of core sampling does a generallypoor job of estimating the lengths and volumes of the holes cut across. The mechanicalstrains induced by core sampling lead to the distortion of the holes cut across [29], andit therefore becomes difficult to obtain an accurate impression of the lengths and thevolumes of the holes.

There was no significant change in volume discrepancy between holes at the bottom ofthe cheese (nearer the detector) and holes nearer the top surface. Therefore it was deemedthat the assumption of orthographic projection used to perform 3D reconstruction wasnot a significant source of error.

Figure 29 shows the 3D reconstruction of the cheese analysed above. Even thougheach 2D projection fitted the holes quite generously, finding many matches in a singleprojection, only a few of those were recognised to be real holes in the 3D matching stage.

40

Figure 29: The 3D reconstruction of the real cheese shown in Figure 25.

Figure 30: Left, the original proxy design (in wireframe view); right, the reconstructedproxy.

As there is no ground truth available for the cheese, it is not known how accurate thisreconstruction is, but it appears to be reasonable.

6.3 Agreement with ground truth

Figure 30 shows a comparison of the original 3D-printed proxy and the reconstructionobtained after using the 3D fitting process.

The algorithm developed only matches 21 of the original 83 holes. Thus it appears(and it seems clear from images of the reconstruction) that many of the holes in theproxy are missing. However, the “missing” holes are largely holes cut-off near the edgesof the cheese (the method’s weaknesses in matching those are described above) and theheavily overlapping holes near the centre of the proxy. Non-overlapping (or marginally

41

overlapping) holes far from the edges were generally matched and reconstructed quitewell. This is a substantial weakness of the reconstruction, but one that is less significantin real cheeses. In real cheeses, the holes do not extend to the edge of the wheel and theygenerally do not overlap [30].

7 Conclusion

This project was established with the aim to use as few X-ray images as possible to obtaina faithful three-dimensional reconstruction of the internal structure of cheese with holesin it. The outcome was to be aimed at a commercial application, so it needed to below-cost, rapid and an improvement over the destructive methods currently used in thequality-control industry [3].

This was split into several parts. To obtain a known ground truth for calibration andquantification of errors, a proxy was designed and manufactured (with a 3D printer) andmade to resemble the statistics of real Emmental cheese. This proxy, as well as a fewsamples of real cheese, was imaged using an X-ray generator from several angles.

An algorithm was developed to find the locations of individual holes in a single two-dimensional X-ray projection. This was achieved by specifying a cost function to beminimised to obtain the most accurate reconstruction, and optimising it by translatingand scaling a set of ellipsoidal kernels corresponding to individual holes.

The characteristics of the kernels from two X-ray projections at different angles werethen matched and combined to produce a single 3-dimensional image. A simple similarityscore was computed to match the found holes in one 2D projection to those of the other,and geometry was used to obtain 3D positions of the centres of the holes. Assumingorthographic projection, the length of each hole along each axis was also reconstructed,and a full 3D image was obtained from the positions.

Both the 2D and 3D matching steps proved somewhat prone to errors. The 2D match-ing stage, while correctly matching many of the holes, had difficulty matching holes onthe sides of the image or holes which overlapped a lot with others. This is a particularproblem with the 3D-printed proxy used - real cheese shows less overlap between holes sogenerally produces higher quality matches.

The 3D reconstruction stage matched holes in different projections correctly withgood accuracy, with no incorrect matches (which would have been judged as incorrect bya human) observed. 3D reconstruction did a very good job of correctly finding the 3Dpositions of the holes, with an rms error of just 4.5mm. On the other hand, the length ofeach side of the hole was matched somewhat worse, with a 12% rms error on the lengthin each direction and a 19% rms error on the volume. Nevertheless, this is a better resultthan offered by core sampling, which itself distorts the length and volume of holes byshearing them.

42

One problem that remains in the implementation used is the speed of the 2D optimi-sation. It runs very slowly on high-resolution X-ray images of cheese, taking roughly anhour on a standard PC. Subsampling improves this, but reduces accuracy. There remainmany computational savings to be had by optimising the code.

Obtaining a 3D reconstruction of the holes in cheese from X-ray images is a problemwhich had never been specifically tackled in the literature. The method proposed in thisproject, while imperfect and in need of improvements, represents the first viable solutionto this problem.

7.1 Future directions

There remain many improvements that can be made to the process described in thisproject to make it more accurate and efficient.

The main problem encountered is the low speed of the 2D optimisation step. Asdesigned, it takes a very long time to converge to a stable solution, on the order of anhour for a high-resolution X-ray image. While it is possible to speed it up by subsamplingthe image to a lower resolution, this lowers the accuracy of optimisation.

A better optimisation technique with a variable step size would substantially reducethe running time of the method, and could even lead to more accurate solutions. Even arelatively simple algorithm, such as steepest-descent (with numerical derivatives) [24, 26]could improve performance by a great deal. Since holes are fitted one at a time, it wouldalso be possible to preload a reconstruction with all the other holes, so that the objectivefunction does not need to obtain a new reconstruction and compute its rms error at eachiteration.

One major source of inaccuracy in optimisation which would be relatively easy to fixis the 2D fitting of cut-off holes at the side of the image. These are not fit correctly asfit holes must remain entirely within the image. This could be fixed relatively easily byallowing kernels to be centred beyond the edges of the X-ray image, and improving thequality of the fit.

One other feature which would improve the technique outlined here is a system torecognise the edges of the cheese in the image and crop the image to contain only thecheese. This could be done with simple edge detection [31], followed by cropping androtation, to make reconstruction invariant to cheese positioning.

7.1.1 Possible applications

The techniques developed in this report can potentially be used in more areas than thequality control of cheese. The computational methods used are in no sense limited toX-ray images, or cheese itself, and are able to find ellipsoidal objects in any image.

One area where this can potentially be of use is biological microscopy, where accurate

43

automated cell counting is an active area of research [32]. Apart from providing a count ofthe cells, the techniques developed here could be used to find the specific location and sizeof cells in a sample and even produce a three-dimensional reconstruction of the sample ata microscopic scale.

44

References

[1] Bachmann H.-P., U. Bütikofer, D. Isolini. Encyclopedia of Dairy Sciences. FAM,Swiss Federal Dairy Research Station; August 2001.

[2] Sherman J.M. “The Cause of Eyes and Characteristic Flavor in Emmental or SwissCheese”. Journal of Bacteriology, 6, no. 4 (1921): pp. 379-393.

[3] “Cheese trier.” U.S. Patent 2,362,090, issued November 7, 1944.

[4] Brosnan T., D-W. Sun, “Improving quality inspection of food products by computervision – a review”. Journal of Food Engineering, 61, no. 1 (January 2004): pp. 3–16.

[5] Jamieson V., “Physics raises food standards”. Physics World, 15, no.1 (2002): pp.21–22.

[6] Han Y.J., S.V. Bowers, R.B. Dodd, “Nondestructive detection of split–pit peaches”.Transactions of the ASAE, 35, no.6 (1992): pp. 2063–2067.

[7] Kraggerhud H., Wold, J. P., Høy, M. and Abrahamsen, R. K. , “X-ray images forthe control of eye formation in cheese”. International Journal of Dairy Technology,62 (2009): pp. 147–153.

[8] Caccamo M., Melilli, C.M. et al. “Measurement of gas holes and mechanical opennessin cheese by image analysis”. Journal of Dairy Science, 87, no. 3 (April 2004): pp.739-48.

[9] Eskelinen J.J., A.P. Alavuotunki et al., “Preliminary Study of Ultrasonic StructuralQuality Control of Swiss-Type Cheese”. Journal of Dairy Science, 90, no. 9 (Septem-ber 2007): pp. 4071–4077.

[10] Mussea M., S. Challois et al., “MRI method for investigation of eye growth in semi-hard cheese”. Journal of Food Engineering, 121 (January 2014): pp. 152–158.

[11] Swinehart D. F. “The Beer-Lambert Law”. Journal of Chemical Education, 39, no.7(January 1972): p. 333.

[12] ICRU, “Tissue Substitutes in Radiation Dosimetry and Measurement”, Report 44 ofthe International Commission on Radiation Units and Measurements (1989).

[13] Koch H. W., and J. W. Motz. “Bremsstrahlung cross-section formulas and relateddata.” Reviews of Modern Physics, 31, no. 4 (1959): p. 920.

[14] Cheyney Design, “X-ray Generator High Performance Air-Cooled Monoblock” CDDXG datasheet, November 2015.

45

[15] Hsieh J. Computed tomography: principles, design, artifacts, and recent advances.Bellingham, WA: SPIE—The International Society for Optical Engineering; 2003.

[16] Wong C.Y., Lin, Ren and Kwok, “A Survey on Ellipse Detection Methods”. IEEEInternational Symposium on Industrial Electronics, May 2012

[17] Duda, R. O., P. E. Hart, “Use of the Hough Transformation to Detect Lines andCurves in Pictures,” Comm. ACM ,15 (January 1972): pp. 11–15.

[18] Nair P., A. Saunders et al., “Hough transform based ellipse detection algorithm,”Pattern Recognition Letters, 17, no. 7 (1996): pp. 777–784.

[19] Hartley R. I., P. Sturm, “Triangulation”. Computer Vision and Image Understanding,68 (November 1997): pp. 146–157.

[20] Ultimaker B.V., “Ultimaker 2 User Manual” Ultimaker 2 datasheet, November 2014.s

[21] OpenSCAD Release 2015.03, The OpenSCAD Developers.

[22] Drumright, R.E., P.R. Gruber, and D.E. Henton. “Polylactic acid technology.” Ad-vanced materials, 12, no. 23 (2000): pp. 1841-1846.

[23] MATLAB Release 2015a, The MathWorks, Inc., Natick, Massachusetts, UnitedStates.

[24] Battiti, R. “First-and second-order methods for learning: between steepest descentand Newton’s method.” Neural computation, 4, no. 2 (1992): pp. 141-166.

[25] Jacobs, R.A. “Increased rates of convergence through learning rate adaptation.” Neu-ral networks, 1, no. 4 (1988): pp. 295-307.

[26] Nocedal J., and S. Wright. Numerical optimization. Springer Science & BusinessMedia, 2006.

[27] Rios L.M. and Sahinidis, N.V. “Derivative-free optimization: a review of algorithmsand comparison of software implementations”. Journal of Global Optimization, 56,no.3 (2013): pp. 1247-1293.

[28] Pratt W. K. Digital Image Processing, New York, NY: John Wiley & Sons, Inc. 2ndedn (1991), pp 651–668.

[29] Casiraghi E.M., Bagley E.B., Christianson D.D. “Behavior of mozzarella, cheddar andprocessed cheese spread in lubricated and bonded uniaxial compression”. Journal ofTexture Studies, 16, no. 3 (September 1985):pp. 281-301.

46

[30] Clark, W.M. “On the formation of “eyes” in Emmental cheese.” Journal of DairyScience, 1, no.2 (1917): pp. 91-113.

[31] Canny, J. “A computational approach to edge detection.” IEEE Transactions onPattern Analysis and Machine Intelligence, 6 (1986): pp. 679-698.

[32] Benes, F.M. et al. “Two-dimensional versus three-dimensional cell counting: a prac-tical perspective”. Trends in Neurosciences, 24 , no. 1 (2001): pp. 11-17

47

A Risk assessment retrospective

The main risk present in this project, and pointed out in its risk assessment, is handlingX-ray imaging equipment. This was an essential part of the project, as its aim is to create3D reconstructions of cheese from X-ray images, and could not be avoided.

X-ray radiation, like any ionising radiation, is potentially harmful, and carries a smallrisk of genetic damage, including increased risk of cancer and inheritable defects, as wellas (in very high doses) the possibility of causing skin burns, hair loss and cataracts. For-tunately, the dose required for imaging is very small and cannot cause noticeable damageto skin or other tissue. I received proper instruction on how to handle the equipment andall of my use of it was monitored by Cheyney Design. Additionally, the equipment itselfwas designed so that it could not function unless the (radiation-shielded) door was closedand all X-rays were contained within the small chamber. This all makes me confidentthat all use of X-ray equipment was safe and any risks were effectively eliminated.

As this project is essentially a computational project, most of my work for it tookplace on a computer. This comes with a few risks, including possible long-term healthproblems due to improper posture.

Appropriate measures were taken to mitigate this, such as taking regular breaks andadjusting the chair and screen height to obtain an ergonomic position. Therefore, noissues were encountered with computer use for the duration of this project.

No other risks were encountered which were not explicitly stated in the risk assessment.Overall, it was deemed that this project was handled safely and that any potential riskswere sufficiently addressed.

48