Statistics with a human faceHow do you analyse a face? Adrian Bowman’s statistics help children with cleft palates and show the symmetry behind the smile.

One of the fascinating aspects of being a statistician is the amazing variety of applications where statisti-cal methods are essential to fi nd out what is going on. Medicine, the environment, genetics, the fi nancial world—statisticians are crucial participants in all of these areas and many more. However, of all the kinds of data that can be collected, few are as personal and emotive as the human face.

Quantitative analysis in this context often brings to mind issues of facial recognition—a very topical area given the increasing preoccupation with security sys-tems. Indeed, this was the topic of an earlier Signifi cance article by Nick Fieller1. However, another important area is in surgery, and medicine more widely, where it is sometimes important to identify the presence of any unusual facial features and to quantify their extent.

In the UK, one of the most common congenital craniofacial abnormalities is cleft lip or cleft lip and pal-ate. Th is is where the soft tissue of the upper lip, some-times including the palate beneath, has a break or cleft. Th ere are about 650 cases of this each year in the UK and children born with the condition usually undergo surgery when they are around 3 months old. Th ese surgical repairs are very successful but it is possible that some unusual facial patterns may remain. It is helpful to have a quantitative assessment of this, not only to be able to assess the eff ectiveness of the surgery, but also to provide information that may be useful in later surgical planning, should that prove necessary.

Photographic images are an obvious place to start when working with facial patterns. However, important information may well be lost if only two dimensions are recorded. Th ree-dimensional image capture systems are now increasingly available, which have opened the door to modelling of the full facial shape of patients.

Laser scanning systems have probably been the most common technology (see Philip Treleaven’s article on the size and shape of whole bodies2 in Signifi cance, September 2007). However, stereo-camera systems can also be used, which do have some advantages, not least of which is that the speed of a camera shot is suffi ciently fast to reduce the diffi culties in keeping very young chil-dren still! In addition to recording a computer model of the three-dimensional facial shape, stereo-cameras also record the coloured photographic texture, which can be particularly helpful when identifying “landmarks” and other key anatomical features. A high resolution stereo-camera system has been developed by Dr Paul Siebert and colleagues in the Department of Comput-ing Science at the University of Glasgow. When two cameras look at the same object, the resulting photo-graphs will be slightly diff erent because of the diff erent viewing positions. Th e two images can be matched and stereo-photogrammetry used to work out how far away things are—essentially what the brain does when our two eyes view an object.

Each image is a highly complex object based on a “range map”, consisting of the three-dimensional loca-tions of a very large number of points on the underlying surface. An obvious fi rst step is to extract some rela-tively simple, but useful, information as a starting point. Well-defi ned anatomical landmarks are the natural place to begin. It would be good to have automatic ways of identifying these, and some progress on that is being made—see Andrea Cavagna’s article on stereo photo-graphs of starling fl ocks in this issue for one automated solution; but at the moment this involves the labori-ous business of inspecting and manually marking the three- dimensional surface on a computer screen. From there, richer information can be extracted, for example

A 3-D image of a face is a highly complex object; analyzing it at

present involves tedious manual

marking of points on a computer

screen

74 june2008

75june2008

by tracking along the surface between land-marks to identify important anatomical curves.

Professor Ashraf Ayoub and colleagues at the Glasgow Dental School have used this system to capture images of cleft lip children3. Once data have been extracted from samples of patients, including control children with non-cleft lip and palate, standard statisti-

cal questions arise. What is the evidence for diff erences between cleft cases and controls, where do these diff erences occur and how large might the diff erence be? Where data have been collected over time, what are the patterns of growth in cleft and non-cleft lip populations and are these diff erent? However, what makes these questions hard to answer in this setting is the unusual nature of the data. Each obser-

vation is not a single number but a set of points in three-dimensional space.

Th e good news is that quite a lot of work has already been done in developing methods for handling data on shape. A relatively recent book by Ian Dryden and Kanti Mardia4 gives a very helpful introduction to the thinking re-quired. An informal defi nition of shape is what is left once the eff ects of location, orientation and scale have been removed. Where a face is positioned, how it is rotated and how big it is do not really aff ect its intrinsic shape, which is determined by the relative positions of the points within each face. A method called Pro-crustes registration can be used to take samples of landmark confi gurations and to match them up by removing the location, orientation and scale eff ects.

Figure 1(a) shows a sample of data from 6-month-old infants after Procrustes regis-tration. Th e grey circles, representing cleft cases, show much more variation than the

black crosses, representing uncleft cases. Th e underlying patterns are seen more clearly by examining the means of each landmark posi-tion, illustrated in Figure 1(b). Th e fi ve up-permost landmarks refer to positions across the eyes and the bridge of the nose. Th e fi ve lowermost landmarks track the position of the upper lip. Th e nose, in the middle of the fi gure, has one side a little lower than the other, and the absence of the “Cupid’s bow” showing a lit-tle dip in the middle of the upper lip, are both recognisable characteristics of cleft shapes.

In repairing a cleft, one of the aims is to make the facial shape as symmetric as possible. Asymmetry seems to be something to which the human perception system is quite sensi-tive. A particularly simple way of approaching facial data is therefore to construct asymmetry scores and to analyse these in more standard statistical models. Th is raises the interesting question of how to measure asymmetry. An object is symmetric if the shape of its mirror image matches up perfectly with the original.

One of the aims is to make the repaired face as symmetric as possible. Asymmetry seems to be something to which the human perception system is quite sensitive

Figure 1. (a) Landmark confi gurations from 6-month-old infants, recorded after Procrustes registration and (b) means of each landmark position

(a)

(b)

76 june2008

A natural measure of asymmetry is therefore to quantify the degree of mismatch between an object and its mirror-image. Mardia, Bookstein and Moreton used this idea in a recent paper to analyse the mean asymmetry of populations5. In the cleft lip study3, considerable eff ort went into collecting data on the same children over the fi rst few years of life, enabling the calcula-tion of an individual asymmetry score for each

child at each time point, thereby tracking any changes. Th is is easily done by refl ecting each set of fi xed landmarks and using Procrustes registration to line this set-up with the original one. Th e average of the distances between the corresponding landmarks gives the asymmetry score.

Figure 2(a) shows the longitudinal pat-tern for the cleft lip, cleft lip and palate, and

control children. Th e eff ect of surgery be-tween 3 and 6 months is clear, with a sharp reduction in asymmetry. However, a marked degree of asymmetry remains in the cleft cases, compared with the controls, and the size of this eff ect does not seem to change with further growth. Quantifying the asymmetry therefore allows us to make useful statements about what is happening. Further thought al-lows us to break down the asymmetry scores into separate components corresponding to diff erent regions of the face, or diff erent ways in which asymmetry can arise. Figure 2(b) locates the major components of asymmetry in the expected places, namely the nasal base and the upper lip.

Although the average degree of asym-metry is of interest, it is equally valid to inves-tigate the nature of asymmetry for individual cases. Figure 3 shows the landmark confi gura-tion for a single cleft case. Arrows have been added to mark the halfway point between each landmark and the corresponding landmark on the matched mirror-image, which indicates how each landmark could be moved to create a symmetric shape. Th is does not necessarily off er a sensible prescription for further surgery, as symmetry is not the sole feature of interest, but it does provide a useful way of indicating the source and nature of the asymmetry in an individual patient.

It was commented earlier that anatomi-cal landmarks are a natural, but rather simple, place to begin in extracting data from images. Anatomical curves provide richer data, which come closer to expressing the complexity of the underlying surface. Sarah Barry, a research student in statistics at the University of Glas-gow, is working on methods for modelling this kind of data6. Th e fi rst step is to construct a suitable description of a three-dimensional curve. Th ere is a strong connection here with another major growth area in statistics: functional data analysis. A convenient way to parameterise a curve is through a collection of three functions, x(t), y(t) and z(t), which describe how the curve co-ordinates change as we travel from one end of the curve to the other (indexed by the dummy variable t). Each of these three functions can then be fi tted by splines or some other fl exible but convenient method. Th e curve can then be identifi ed by its spline coeffi cients. Th is gives an economical way of characterising each curve.

Th ese curve coeffi cients can then be used as the data in fi tting and analysing suitable models. Sarah has investigated longitudinal models, where the curves for each child are

Figure 2. (a) Longitudinal pattern of asymmetry for the cleft lip (UCL,1), cleft lip and palate (UCLP,2), and control, 3 children and (b) breakdown of asymmetry scores for separate components (nbc, nasal base; nrim, nasal rim)

Squa

re ro

ot o

f asy

mm

etry

Squa

re ro

ot o

f asy

mm

etry

(a)

(b)

77june2008

tracked across time and there is interest in the nature of the patterns of growth. Th ese models are complex to fi t, but they have the advantage of giving a unifi ed description of the full data set, where simpler models would

provide only a collection of separate analyses at each time point. Figure 4 shows the results of fi tting such a model to the anatomical mid-line curve that runs down the ridge of the nose to the upper lip. A short upper lip

and its deviation from the mid-line are ap-parent in the mean curve for the cleft cases at 3 months, before surgery. Th e eff ects of the corrective action, and the further improve-ment over time, are apparent as the mean curve shape for cleft and control cases become closer as age increases.

Th ese diff erent methods allow facial shape, and in particular the eff ects of the cleft condition, surgical correction and subsequent growth to be quantifi ed and the evidence for diff erences and change over time to be as-sessed. Current modelling challenges include extending the representation of shape from points and curves to full three-dimensional surfaces and developing the analytical and sta-tistical tools to accompany this. However, the existing statistical models and methods have already brought considerable insight to ques-tions of facial shape and there is clearly scope for applying them to good eff ect in a number of other medical settings, for instance in surgi-cal procedures required for a wide variety of other craniofacial conditions.

In every sense, there is clearly a bright future for statistics with a human face!

References 1. Signifi cance (2004) Computer-assisted

facial identifi cation. Signifi cance, 1, 2.2. Treleaven, P. (2008) How to fi t into your

clothes: busts, waists, hips and the UK National Sizing Survey. Signifi cance, 4, 113–117.

3. Ayoub, A. F., Wray, D., Moos, K. F., Siebert, P., Jin, J., Niblett, T. B., Urquhart, C. and Mowforth, R. (1996) Th ree-dimensional modelling for modern diagnosis and planning in maxillofacial surgery. International Journal of Adult Orthodontics and Orthognathic Surgery, 11, 225–233.

4. Dryden, I. and Mardia, K. V. (1998) Th e Statistical Analysis of Shape. New York: Wiley.

5. Mardia, K. V., Bookstein, F. and Moreton, I. (2000) Statistical analysis of bilateral symmetry of shapes. Biometrika, 87, 285–300.

6. Barry, S. J. E. and Bowman, A. W. (2006) Linear mixed models for longitudinal shape data with applications to facial modelling. Technical Report, University of Glasgow, Glasgow. (Available from www.stats.gla.ac.uk.)

Adrian Bowman is a Professor in the Department of Statistics at the University of Glasgow. His research in-terests cover a variety of aspects of statistical methods and include the modelling of three- dimensional shape in particular. The work described in this article has been carried out in conjunction with colleagues in the Dental School and the Department of Computing Science at the University of Glasgow, notably Profes-sor Ashraf Ayoub and Dr Paul Siebert.

Figure 3. Landmark confi guration for a single cleft case: arrows mark the halfway point between each landmark and the corresponding landmark on the matched mirror-image

Figure 4. Example of how longitudinal models can be used to track curve coeffi cients for individual children over time