1356 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 54, NO. 7, JULY 2007

A Novel System for the 3-D Reconstruction of the HumanSpine and Rib Cage From Biplanar X-Ray Images

F. Cheriet, C. Laporte, S. Kadoury*, H. Labelle, and J. Dansereau

AbstractThe main objective of this study was to develop a 3-D X-ray re-construction system of the spine and rib cage for an accurate 3-D clinical as-sessment of spinal deformities. The system currently used at Sainte-JustineHospital in Montreal is based on an implicit calibration technique basedon a direct linear transform (DLT), using a sufficiently large rigid objectincorporated in the positioning apparatus to locate any anatomical struc-ture to be reconstructed within its bounds. During the time lapse betweenthe two successive X-ray acquisitions required for the 3-D reconstruction,involuntary patient motion introduce errors due to the incorrect epipolargeometry inferred from the stationary object. An approach using a new cal-ibration jacket and explicit calibration algorithm is proposed in this paper.This approach yields accurate results and compensates for involuntary mo-tion occurring between X-ray exposures.

Index TermsExplicit calibration, nonlinear optimization, spinal defor-mities, X-ray images, 3-D reconstruction.

I. INTRODUCTION

The 3-D reconstruction of the spine and rib cage from multipleX-rays is currently done at Sainte-Justine Hospital for the evaluationof the 3-D scoliotic deformities. The reconstruction is based on a directlinear transformation (DLT) technique [1] requiring a calibration ob-ject with known 3-D coordinates. Given the significant extrapolationerror of this technique [2], the calibration object was built large enoughto accommodate any structure to be reconstructed within its limits.The technique was shown to be adequate for the 3-D reconstructionof manually identified anatomical landmarks on the spine. The 3-Dreconstruction of the rib cage is less robust because repeatably iden-tifiable anatomical landmarks are scarce. Correspondence betweenviews must, therefore, be established based on the epipolar geometryinferred from the stationary calibration object. Error is introduced byinvoluntary patient motion with respect to the calibration object duringthe delay of approximately 25 seconds between X-ray exposures.In previous work [3], we described a nonlinear method for the 3-Dreconstruction of the coronary arteries from biplane X-ray angiograms

Manuscript received October 13, 2005; revised October 21, 2006. Asteriskindicates correpsponding author.

F. Cheriet is with the cole Polytechnique de Montral, Department of Com-puter Engineering, Montreal, QC H3C 3A7, Canada. He is also with the Re-search Center Sainte-Justine Hospital, Montreal, QC H3T 1C5, Canada (e-mail:farida.cheriet@polymtl.ca).

C. Laporte is with the McGill University, Centre for Intelligent Machines,McConnell Engineering Building, Montreal, QC H3A 2A7, Canada (e-mail:cathy@cim.mcgill.ca).

*S. Kadoury is with the cole Polytechnique de Montral, Department ofComputer Engineering, P.O. Box 6079, Succursale Centre-ville, Montreal, QCH3C 3A7, Canada. He is also with the Research Center Sainte-Justine Hospital,3175, Cote-Sainte-Catherine, Montreal, QC H3T 1C5, Canada, (e-mail: samuel.kadoury@polymtl.ca).

H. Labelle is with the Research Center Sainte-Justine Hospital, 3175 Cote-Sainte-Catherine, Montreal QC H3T 1C5, Canada (e-mail: hubert.labelle@recherche-ste-justine.qc.ca).

J. Dansereau is with the cole Polytechnique de Montral, Department ofMechanical Engineering, Montreal, QC H3C 3A7, Canada. He is also withthe Research Center Sainte-Justine Hospital, Montreal, QC H3T 1C5, Canada(e-mail: jean.dansereau@polymtl.ca).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TBME.2006.889205

Fig. 1. X-ray acquisition system setup with calibration apparatus.

not involving a calibration object. This approach explicitly solvedthe nonlinear equations relating a 3-D structure with its perspectiveprojections, with respect to the geometrical parameters of the biplaneimaging system. A subsequent study [4] showed this approach to befeasible for the 3-D reconstruction of the spine and rib cage. Thecontributions of this work are twofold: first, a new calibration objectconsisting of a jacket worn by the patient. The jacket undergoes thesame displacement as the patient between exposures, allowing cali-bration to compensate for this displacement. Second, a new explicitcalibration algorithm that avoids the extrapolation errors intrinsicto the DLT approach. The paper is organized as follows. Section IIdescribes the image acquisition and calibration methods. Experimentalresults are presented in Sections III (simulations) and IV (clinicalstudy).

II. MATERIALS AND METHODS

A. Image Acquisition

Two postero-anterior X-rays (the standard PA-0 and the 20 angleddown PA-20) and one lateral (LAT) X-ray are acquired from the setupshown in Fig. 1 for the 3-D reconstruction of the spine and rib cage. Anautomated rotary platform brings the patient from the LAT to the PA po-sition. The calibration object is a jacket with 16 encrusted radio-opaquemarkers worn by the patient during exposures. The setup includes anexternal calibration object made of 6 coplanar radio-opaque pellets ofknown 3-D coordinates which define a global reference plane for the3-D reconstruction. All calibration markers are automatically detectedand matched using the method presented in [5].

B. Explicit Calibration Algorithm

The proposed approach involves explicit use of the description ofthe calibration matrices with the geometrical parameters of the radio-graphic system. The goal of the explicit calibration algorithm is to es-timate the geometrical parameters of the radiographic setup leading tooptimal 3-D reconstruction of the spine and rib cage. The method usedis based on the iterative nonlinear optimization process described in[4]. The criterion to be minimized is

" ()=

N

n=1

2

i=1

d [(xmni; ym

ni) ; (xni(i; pn()); yni(i; pn()))]

2

+

L

l=1

d [(Xl; Yl; Zl) ; pl ()]2 (1)

where d[] denotes Euclidean distance, (xm; ym) are the fixed, mea-sured projection coordinates of theN points from the calibration jacket

0018-9294/$25.00 2007 IEEE

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 54, NO. 7, JULY 2007 1357

Fig. 2. Effect of rotation about the X, Y, and Z axes on distance between the actual corresponding point and the computed epipolar line.

on the biplane images, and (x(; p); y(; p)) are the coordinates of theanalytical projection of the object point p obtained from standard per-spective transformation formulas. p() is the 3-D reconstruction of anobject point obtained by stereo-triangulation from its measured biplaneprojections and the current estimate of the geometrical parameters .The second term of (1) represents the distance between the known 3-Dpositions of the L = 6 pellets describing the reference plane and thosecomputed from current parameter estimates. In this work, the constraint(1) is applied to the PA-0 LAT, and PA-0 PA-20 pairs. The Lev-enberg-Marquardt algorithm [6] is used for optimization, iterating untilthe correction to the geometrical parameters becomes negligible.

III. SIMULATION OF PATIENT MOTION

To estimate the geometrical parameters, the proposed method relieson radio-opaque markers that move with the patient, rather than a sta-tionary calibration object. This makes the epipolar geometry inferredfrom the calibration object consistent with the 2-D anatomical data re-trieved from the images, and affects the accuracy of the resulting 3-Dreconstruction. Computer simulations were performed to evaluate therobustness of the explicit calibration algorithm and the effects of patientmotion on the establishment of stereo-correspondence. A simulated3-D model of the human trunk was obtained from a DLT reconstructionof an X-ray dummy marked with radio-opaque pellets. Typical geomet-rical parameters (determined from previous DLT reconstructions) were

used to compute analytic PA-0 and LAT projections of the 3-D model,markers and calibration pellets. Small amounts of Gaussian noise wereadded to the projected points to account for landmark identificationerrors. To simulate patient motion between X-ray exposures, the 3-Dmodel and markers were transformed between the PA-0 and LAT pro-jections. Assuming that bony structures undergo a motion similar tothat of the external trunk (i.e., the jacket) [7], the same geometricaltransformation was applied to both the 3-D model and 3-D positions ofthe markers. Three types of patient movements were considered: lateralbending, frontal bending and shoulder rotation. These were modelled asrotations of the trunk about theX,Y and Z axes, respectively, using thecenter of the fifth thoracic vertebra as a center of rotation. The quality ofthe stereo-correspondences obtained for the rib cage points was mea-sured by the distance between the points of the LAT view known tocorrespond exactly to those of the PA-0 view, and the epipolar linescomputed from the estimated geometrical parameters. The results areshown in Fig. 2 for varying amounts of motion and compared to thoseobtained with the DLT technique. The results show that, irrespective ofmotion, stereo-correspondences are established with constant accuracywhen the explicit calibration method is used. With the implicit DLTmethod, the quality of the stereo-correspondences steadily degrades asthe magnitude of the movement increases. The explicit calibration al-gorithm appears to be most robust to rotation about the X and Y axes.Interestingly, rotation about the Y axis is also the most commonly ob-served type of inter-exposure patient motion [8].

1358 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 54, NO. 7, JULY 2007

IV. CLINICAL VALIDATION

Clinical trials were conducted in order to assess the validity of the3-D reconstructions yielded by the proposed system. The data consistedof pairs of X-rays of 20 children. The study group was comprised of15 girls (75.0%) and 5 boys (25.0%). The mean age was 13:63 2:81(range 918) years old. All the children in the group had scoliosis, withan average Cobb angle on the frontal plane of 22, (range 751). Thespine and rib cage of each subject were reconstructed in 3-D using boththe DLT and explicit calibration techniques. To reconstruct the spine,six anatomical landmarks per vertebra (centers of superior and inferiorvertebral endplates and the tips of both pedicles) are manually identifiedand matched on the PA-0 and LAT views by an expert. Cubic splinesare fitted to 11 arbitrary, manually identified points along the center lineof each rib on the PA-0 and PA-20 views, and automatically matchedbased on epipolar geometry to reconstruct the rib cage. The simulationspresented earlier showed the effect of patient rotation on the accuracy ofthe epipolar line. This measure represents the distance of a specific land-mark selected on an X-ray image to its corresponding epipolar line ob-tained from the 3-D reconstruction. The first clinical experiment com-puted the same epipolar accuracy measure for the 20 patients in thisstudy. The mean distance between the corresponding points and theepipolar lines for all 20 patients was 0:971 0:676mm, compared to amean distance of 1:4011:541mm for the standard DLT method. Thisconfirms that the proposed method is well suited for clinical use becauseof its robustness to involuntary patient displacements, which representa pitfall of the straightforward DLT method. A second clinical experi-ment collected a series of clinical indices from the 3-D reconstructionsobtained with the DLT and explicit calibration methods. The absolutedifferences between the Cobb angles obtained with the DLT and explicitcalibration methods were 0:3 0:428. As for the frontal and sagittalbalance, the absolute differences were0:150:151, and0:370:259

respectively. These results prove that the proposed method recovers thesame overall geometry as the standard DLT technique. Overall, the pro-posed explicit calibration approach offers many advantages over the im-plicit DLT technique. First, the small size of the calibration apparatusallows the patient to adopt a more relaxed posture during X-ray acqui-sition. Moreover, the explicit calibration method is suitable for manycircumstances under which the DLT is not, including the case of pa-tients in wheelchairs. Finally, its robustness to patient motion betweenexposures is very appealing.

V. CONCLUSION

The new system provides the clinician with an accurate tool to eval-uate spinal deformities and allows the patient to adopt a normal attitudewithout any constraint, compensating for her displacement between ex-posures. The validation study assessed the accuracy of the new methodand showed that the clinical indices derived from reconstructions ob-tained with the new method are similar to those obtained with the stan-dard DLT technique. An extension to this work would be a self-cali-bration technique that allows the 3-D reconstruction of the spine andrib cage from uncalibrated X-rays.

ACKNOWLEDGMENT

The authors would like to thank P. Labelle and J. Joncas of theSainte-Justine Hospital Research Center for handling and processingclinical data that was used in several experiments.

REFERENCES

[1] J. Dansereau and I. A. F. Stokes, Measurements of the three-dimensional shape of the rib cage, J. Biomech., vol. 21, pp. 893901,1988.

[2] G. A. Wood and R. N. Marshall, The accuracy of DLT extrapolationin three-dimensional film analysis, J. Biomech., vol. 19, no. 9, pp.781785, 1986.

[3] F. Cheriet and J. Meunier, Self-calibration of a biplane X-ray imagingsystem for an optimal 3D reconstruction, Int. J. Comp. Med. Imag. andGraph, vol. 23, pp. 133141, 1999.

[4] F. Cheriet et al., Towards the self-calibration of a multiview ra-diographic imaging system for the 3D reconstruction of the humanspine and rib cage, Int. J. Pattern Recognit. Artif. Intell., vol. 13,no. 5, 1999.

[5] S. Kadoury and F. Cheriet, X-ray image restoration with adaptive PDEfilter for an accurate 3D reconstruction of the human spine, presentedat the 20th Conference of CARS, Osaka, Japan, 2006.

[6] D. W. Marquardt, An algorithm for least-squares estimation of non-linear parameters, J. Soc. Ind. Appl. Math., vol. 11, no. 2, pp. 431441,1963.

[7] C. Bergeron et al., Prediction of scoliotic spinal curve from three-dimensional trunk surface using support vector regression, Eng. Appl.Artif. Intell., vol. 18, no. 8, pp. 973983, 2005.

[8] C. Bellefleur et al., Evaluation of the Efficiency of Patient StabilisationDevices for 3D X-Ray Reconstruction of the Spine and Rib Cage.Amsterdam, The Netherlands: IOS Press, 2000.