ieee transactions on image processing,...

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 7, JULY 2006 1871

A Computationally Efficient Approach to theEstimation of Two- and Three-Dimensional

Hidden Markov ModelsDhiraj Joshi, Student Member, IEEE, Jia Li, Senior Member, IEEE, and James Z. Wang, Senior Member, IEEE

Abstract—Statistical modeling methods are becoming indispens-able in today’s large-scale image analysis. In this paper, we explorea computationally efficient parameter estimation algorithmfor two-dimensional (2-D) and three-dimensional (3-D) hiddenMarkov models (HMMs) and show applications to satellite imagesegmentation. The proposed parameter estimation algorithm iscompared with the first proposed algorithm for 2-D HMMs basedon variable state Viterbi. We also propose a 3-D HMM for volumeimage modeling and apply it to volume image segmentation using alarge number of synthetic images with ground truth. Experimentshave demonstrated the computational efficiency of the proposedparameter estimation technique for 2-D HMMs and a potential of3-D HMM as a stochastic modeling tool for volume images.

Index Terms—Hidden Markov models (HMMs), maximum like-lihood estimation, parameter estimation, three-dimensional (3-D)HMM, Viterbi training, volume image processing.

I. INTRODUCTION

OVER THE last decade, we have see an explosion of digitalimage usages in a number of application domains. Each

day, large quantities of multidimensional images are beingacquired digitally. Scanners in satellites produce enormousamounts of data for surveillance and reconnaissance purposes.Multispectral and hyper-spectral images contain additionalinformation in the form of a third spectral dimension. Moderntelescopes are capable of producing high-resolution pictures ofdistant stars and galaxies. Magnetic resonance imaging (MRI)and computed tomography (CT) scanners in hospitals producehigh-resolution three-dimensional (3-D) images of the humanbrain or the human body so that physicians and radiologists canlook inside their patients noninvasively. At airports, 3-D CTscanners are being installed to monitor luggages checked in bytravelers in order to detect dangerous substances. The amount

Manuscript received January 4, 2005, revised August 25, 2005. The Websitehttp://riemann.ist.psu.edu provides more information related to this work.This work was supported in part by the U.S. National Science Foundationunder Grants IIS-0219272, IIS-0347148, and ANI-0202007; in part byThe Pennsylvania State University; in part by the PNC Foundation; andin part by SUN Microsystems under Grants EDUD-7824-010456-US andEDUD-7824-030469-US. The associate editor coordinating the review of thismanuscript and approving it for publication was Dr. Thierry Blu.

D. Joshi is with the Department of Computer Science and Engineering,The Pennsylvania State University, University Park, PA 16802 USA (e-mail:[email protected]).

J. Li is with the Department of Statistics and the Department of ComputerScience and Engineering, The Pennsylvania State University, University Park,PA 16802 USA (e-mail: [email protected]).

J. Z. Wang is with the College of Information Sciences and Technology andthe Department of Computer Science and Engineering, The Pennsylvania StateUniversity, University Park, PA 16802 USA (e-mail: [email protected]).

Digital Object Identifier 10.1109/TIP.2006.877039

of digital image information being generated is so enormousthat it has become inevitable for computers to help analyze,segment, and classify these images. Because multidimensionalimages are typically stored in image databases, it is now pos-sible for computers to extract patterns or semantic connectionsbased on a large collection of annotated or classified images.Such automatically extracted patterns can also be used bycomputers to classify new images.

In the recent past, researchers in the image analysis commu-nity have successfully used statistical modeling techniques tosegment, classify, and annotate images. Particularly, variationsof hidden Markov models (HMMs) have been developed andsuccessfully used for image and video applications. The keyissue in using such complex models is estimation of parameterswhich is usually a computationally expensive task. In practice,often a trade off is accepted between accuracy of estimation andrunning time of the estimation algorithm.

In this paper, we introduce a new model for studying collec-tions of volume images (3-D HMM). We also present a compu-tationally efficient parameter estimation algorithm for two-di-mensional (2-D) and 3-D HMMs. As discussed in Section IV-D,the estimation algorithm can be efficiently scaled from its ver-sion for 2-D HMMs to be used for 3-D HMMs. The proposed al-gorithm is polynomial-time in number of states and linear-timein the number of pixels of an image, for both 2-D HMM and 3-DHMMs. When applied to 2-D HMM-based supervised learning,the running time of the algorithm is found to be much lesscompared to the estimation algorithm for 2-D HMMs whichwas proposed in [24]. The techniques have been implementedand tested with a large number of images. Promising resultshave been obtained. The algorithms can potentially be appliedto other analysis tasks involving high-dimensional data.

A. Extending Conventional HMMs

Over the years, HMMs have been explored as efficient statis-tical modeling techniques in speech recognition and image andvideo understanding [1], [17], [23], [36].

HMMs are a special case of graphical models in statistics[13]. Observed signals are modeled through a first order Markovprocess as follows. Suppose the observed signal is a sequence

, where could be multivariate. An HMMassumes that each observation in the sequence exists in one ofa set of hidden states at each position . Let the correspondinghidden state sequence be , .It is usually assumed that the state sequence follows a firstorder Markov chain. That is,

1057-7149/$20.00 © 2006 IEEE

1872 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 15, NO. 7, JULY 2006

. Furthermore, the time homogeneity of the Markovchain is normally assumed, i.e., the transition probabilities,

, are invariant over . Given thestate , the observation is conditionally independent of thestates and observations at the other positions. The conditionaldistribution of given is usually modeled parametricallywith parameters varying with the conditioned state. The mostwidely used distribution for multivariate data is the multivariateGaussian distribution which is parameterized by a mean vectorand a covariance matrix.

To effectively account for the 2-D statistical dependence, 1-DHMMs have been extended to pseudo 2-D HMMs and pseudo3-D HMMs for face recognition [11], [18]. Another line of re-search is to extend the underlying Markov chain assumed for se-quential data in a 1-D HMM to a Markov mesh for spatial data[24] (for 2-D images, the observations reside on a 2-D grid.).

There is a surge of research to analyze volume images. Con-ventional 2-D image modeling paradigms may not be effective Inthe past, 3-D Markov random fields (MRFs) have been appliedto medical image segmentation [27]. Given the great potentialdemonstrated by the HMM paradigm in various applications, itseems most natural to extend 2-D HMM to 3-D HMM for volumeimage analysis. In this paper, we construct a 3-D HMM thatcaptures the 3-D statistical dependence in volume images.

In 3-D modeling, a volume image is represented by featurevectors on a 3-D grid. An image may be divided into cubeswhich could be overlapping. In such a case, every cube corre-sponds to one position in the grid. In our experiments, the size ofthe cube is 1 and each voxel represents a distinct grid point. Theobservation at each position comprises features extractedfrom the respective cube. It is assumed that each position existsin an underlying state and the observation is condi-tionally independent of the other positions when is known.The Markovian property that governs the transition of states in2-D HMM is extended to one in 3-D. In particular, the transi-tion probability takes into account the neighboring state alongthe extra dimension.

For the same number of states, the dependence in the addi-tional dimension leads to a significant increase in the numberof transition probabilities to be estimated. To robustly estimatethese parameters, a model regularization approach is explored.Another challenge faced in the extension is the computationalcomplexity of the estimation algorithm. The Viterbi training [24]approach is used here. The key computational issue is to searchfor the combination of states with the maximum a posteriori(MAP) probability conditioned on all the observations under aset of given parameters. A computationally efficient algorithmis developed to achieve a locally optimal solution by iterativelyseeking the best states in a subset with the other states fixed.Our algorithm appears to extend to the case of Markov meshrandom fields on higher dimensional finite rectangular lattices.

B. Related Work in Modeling

One of the key challenges of mathematical and statisticalmodeling has been to understand and model the neuro-physi-ological processes that form the basis of human visual cogni-tion. Empowering a machine with such capabilities would haveplausible applications in many areas such as robotics, medicaldiagnostics, and automated surveillance.

There is a rich resource of prior related work on statisticalimage modeling in computer vision, image processing, and ma-chine learning. Space limitations do not allow us to present abroad survey. Instead we try to emphasize some of the work thatis most related to what we propose. The references below are tobe taken as examples of related work, not as the complete list ofwork in the cited areas.

Theories and methodologies related to MRFs [6], [10], [12],[20] have played important roles in the construction of many sta-tistical image models which have widely been used for severalapplications in computer vision [26]. For a thorough introduc-tion to MRFs and their applications, see Kindermann and Snell[20] and Chellappa and Jain [6]. An MRF extends Markoviandependence from 1-D to a general graph [10], [12].

AsubstantialamountofworkhasbeendoneonapplyingMRFsto image segmentation and texture analysis [5], [7], [9], [12],[15], [29], [31]–[33], [35], [38]–[40]. Estimation and approx-imation techniques for MRF-based procedures have also beenwidely studied [3], [14], [16], [19], [34]. An interesting work onstationary MRFs has been presented in [4]. Particularly, MRFs ona finite rectangular lattice have been rigorously characterized.

Many applications use MRFs to model the pixel representa-tion of images, which sometimes may not be the best way tomodel the dependencies occurring in real images. In some realapplications (such as satellite image classification), certain di-rectional patterns are seen to be more pronounced than others. Insuch cases, HMMs, which do not impose isotropy, are expectedto capture dependencies, in a more precise fashion.

Inspired by HMMs, researchers have recently used MRFs tomodel the classes or states of image pixels rather than their di-rect values [2], [8], [22]. Pixel values are considered condition-ally independent given their classes or states. We discuss here anumber of such models. It is worthy to note that although thesemodels constitute an important part of existing statistical imagemodels, they represent, by no means, the complete set. Multi-scale representations of MRFs have also been studied [2], [28].Another multiresolution model based on HMMs is the modelproposed for wavelet coefficients by Crouse et al. [8], wherewavelet coefficients across resolutions are assumed to be gen-erated by one-dimensional HMMs with resolution playing therole of time, in the Markov chain.

C. Outline of the Paper

The remainder of the paper is organized as follows. Section IIintroduces the 2-D HMMs. Section III describes the constructionof the 3-D HMM. In Section IV, the parameter estimation methodand the computationally efficient algorithm are introduced.Sections V and VI discuss the experiments and their results. Wepresent our conclusions and suggest future research directions inSection VII. The Appendix includes a proof of an equation usedin the development of the computationally efficient algorithm.

II. TWO-DIMENSIONAL HIDDEN MARKOV MODELS

Two-dimensional HMMs and their multiresolution extension[23] have been studied, in the past and successfully applied tosupervised image classification, compression [21] and annota-tion [25]. In this section, we briefly introduce the assumptionsof a 2-D HMM with respect to 2-D image modeling.

JOSHI et al.: TWO- AND THREE-DIMENSIONAL HIDDEN MARKOV MODELS 1873

Conventionally, 2-D images are divided into square blocksand block-based features are extracted. Each block represents apoint in a 2-D grid and the features extracted from block atare represented by . In the limiting case when block size be-comes 1, each pixel represents a point in the 2-D grid. The ob-servation is usually a vector containing features computedfrom the pixel values in the block at . A lexicographic or-dering of points is defined as follows: if

or , .Under the assumptions of the model, the underlying states

are governed by a first order Markov mesh:, where , and

. That is, among the conditioned states, only states ofthe neighbor above and the neighbor to the left of the currentposition affect the transition probability. For points at bound-aries, which miss one of the two needed neighbors, dependenceupon only the states of the existing neighbor is assumed. Besidesthis, given the state of a point , the feature vectoris assumed to follow a multivariate Gaussian distribution with acovariance matrix and a mean vector determined by its state.

For the 2-D HMM, we need to estimate the transition prob-abilities , , , , the mean vectors , andthe covariance matrices for each state .

III. THREE-DIMENSIONAL HIDDEN MARKOV MODELS

In this section, we present the 3-D HMM in its general form.Before stating the statistical assumptions, we describe the fol-lowing terminologies which will be used throughout the paper.

• 3-D point: A point where , and are coordi-nates along the , and axes respectively. Typically,an observation is associated with a 3-D point. For a 3-Dimage, every voxel may correspond to a point. In certainapplications, a 3-D image may be divided into rectangularsolids and features are extracted for each solid. In suchcases, voxels in a rectangular solid are treated collectivelyas one point.

• 3-D grid: A 3-D array of finite and equally spaced 3-Dpoints (along , and axes) in space represented as

. Here is a finite positive integer. For brevity, wewill assume a cubic 3-D grid. A graphical illustration forthe 3-D grid is in Fig. 1.

• frame: We call a collection of points on a plane parallelto that of and axes a frame. A frame is indexed byits coordinate: .

A lexicographic order is defined among 3-D points as follows:if or , or ,

, . The observed feature vector of 3-D pointwill be denoted by and its state by . The number ofstates is preselected.

The model attempts to capture statistical dependence among3-D points. The assumptions of 3-D HMM are as follows.

1) whereis the set of

states and feature vectors of all points precedingin the lexicographic order. In addition, is thestate of the point at the same and coordinates in the

Fig. 1. Three-dimensional grid. Given the states of all the points that precedepoint (i; j; k), only the states of the three neighboring points affect thedistribution of the state at (i; j; k). These are denoted as N , N , and N inthe figure.

previous frame, and are thestates of the neighboring points in the same frame andone position ahead along and respectively. Givenany point , the three neighboring points that affectit among all those that precede it are denoted as , ,and in Fig. 1. For boundary points, dependence upononly the state of the existing neighbors is assumed. Forexample, if for a boundary point, the neighbor denoted as

is missing, then dependence upon neighbors andis assumed.

2) Given the state of a point , the feature vectorfollows a multivariate Gaussian distribution param-

etrized by a covariance matrix and a mean vector deter-mined by the state. For a state , denotethe corresponding covariance matrix and mean vector by

and . Recall that the probability density function ofa -dimensional Gaussian distribution is

3) If the state of point is known, its observed vectoris conditionally independent of the rest of the points

in the 3-D grid.For the 3-D HMM, the transition probabilities , , ,

, , the mean vectors , and the covariance ma-trices for each state need to be estimated.Subject to the constraint for any ,

, and , the transition probabilities comprise freeparameters. By the assumptions of our 3-D HMM model, givena previous frame, the states in the current frame follow a 2-DHMM.

IV. PARAMETER ESTIMATION

In this section, we present an algorithm designed to estimateparameters of two and 3-D HMMs. We present an efficient wayto extend the Viterbi algorithm to 3-D. Our approach may notbe unique or best but is computationally efficient as discussedlater. For clarity, we present our version of the algorithm for 3-DHMMs. The version of the estimation algorithm for 2-D HMMs


Fig. 2. Schematic of the Viterbi algorithm. Assume that the dashed path is the shortest path from t = 1 to t = 6.

follows naturally from its 3-D HMM counterpart, since a 2-Dimage can be thought of as a special case of a 3-D image withonly one frame.

A. Viterbi Training

The Viterbi training approach [37] used to estimate 1-DHMM is adopted here. In particular, we iterate the search forthe optimal combination of states and the update of parameters.From optimization perspective, the Viterbi training algorithmprovides a local maximization of the so-called classificationlikelihood [30], i.e., the likelihood of the observed featurevectors under a given set of parameters as well as a givenset of underlying states. The unknown states are regarded asparameters and are part of the estimation.

Consider the problem of predicting the optimal state sequencefor a 1-D HMM. In many cases, as in our

case, the classification likelihood, to be maximized, is separableand can be expressed in the form

Suppose, in the optimal state sequence , the state of theth position is , it is easy to see that maximization of

is equivalent to maximization of two separatefunctions

and

The expressions and represent terms in ,for and respectively, when the state at position is

. Also note that

The problem of finding the optimal state sequence can be seenas a shortest path problem as shown in Fig. 2. Here, a weight

is placed on a link from state at to state at

. The starting nodes have a weight for state . Suppose,the path denoted by a dashed line is the optimal one, then it iseasy to see that the dashed sub-path, ending at , is optimalamong the sub-paths ending at in state 2. Similarly, thedashed sub-path between to is shortest among thesub-paths starting at in state 2. Viterbi algorithm exploitssuch an optimal sub-structure in an optimization problem. Thesequential steps of the Viterbi algorithm are as follows.

1) At , record , whererepresents the number of states.

2) For , , set

.3) Finally, paths are formed each ending at a different

state at . Pick which maximizes and tracethe path backward from the last state .

Note that this procedure requires com-parisons and memory records. Hence, the compu-tational and storage complexities are and ,respectively.

Denote the collection of parameters of 3-D HMM collectivelyas . Mark the iteration step by a superscript. Suppose the initialparameter is . In Viterbi training, to update from

, the following two steps are applied.

1) Under parameter , the combination of states with theMAP probability conditioned on the observed vectors isidentified, as shown in the equation at the bottom of thepage. Details of this procedure are explained later.

2) The parameters are then computed by assuming statesare the true underlying states. If the true states were

known, the maximum likelihood estimation of the param-eters would be easy to obtain. The mean vector andthe covariance matrix would simply be estimated bythe sample mean and sample covariance matrix of all theobserved vectors whose states , shownin (1)–(4) at the bottom of the next page. The transi-tion probabilities are computed by the empiricalfrequencies as shown in (3). where is the indicatorfunction that equals 1 when the argument is true and 0


otherwise. The set.

Various methods can be used to initialize the Viterbi trainingalgorithm. In our experiment, -means clustering is used togenerate an initial set of . The initialization of -meansis random. The iteration then starts with step 2) above. The finalresult could be affected by the initial states. In -means clus-tering of points, directional dependencies, in the 3-D lattice areignored. In such a scenario, the generative model is a Gaussianmixture model, a special case of a HMM.

Due to the large number of parameters even with a moderate, we regularize the transition probabilities by a partial 3-D

dependence. In particular, if the dependence along the axis isignored, the model is reduced to a 2-D HMM and the transitionprobabilities are , where and .The maximum likelihood estimation for updated at iter-ation is given in (4).

The 3-D transition probabilities are regularized toward the2-D ones by a linear combination shown in (5). The value

controls the extent of 3-D dependence. correspondsto a pure 3-D model and , a 2-D model. It is shown in theexperiments that an intermediate value of is often preferable.For notation simplicity, we denote the transition probabilities ingeneral by even though (5) is used to compute them

(5)

B. Searching for Optimal States

The computational complexity of 1-D Viterbi algorithm hasbeen discussed in Section IV-A. The key computational issuein Viterbi training is to find the states ,with MAP probability, under a given set of parameters. For1-D HMM, the MAP sequence of states can be solved for bythe Viterbi algorithm. For 3-D HMM, there are possiblecombinations of states for the entire 3-D grid. When applied

directly to the frame process, the Viterbi algorithm enables usto avoid exhaustive search along the axis. However, it is stillnecessary to consider all the possible combinations of states inevery frame. In this method, the computational complexity ofsearching for the optimal set of states is at least .

To address the computational difficulty, we propose a locallyoptimal algorithm to search for the set of states with the MAPprobability. Define the set of points with a fixed and co-ordinate in a 3-D grid as a row, shown in Fig. 1. Denote a rowby . Let

. Denote a sequence of statesin row by , . Forbrevity, denote the sequence of observed vectors in row by

, . We also introduce alexicographic order for the rows: if or

, . We process the rows in the lexicographic order.The procedure to iteratively update the states in each row is asfollows, which is also illustrated by Fig. 3. Let the states in arow obtained at iteration be .

1) Initialize all the states , , in the3-D grid.

2) Let .3) Let , .4) Search for with the MAP probability given the ob-

served vectors in the entire 3-D grid and the states inall the other rows: , and ,

. Note that the conditioned states ofpoints in the other rows should always be their most up-dated values. Set .

5) Let .6) If , go back to step 4). Otherwise

a) set , ;b) if , go back to step 4). Otherwise

i) let ;ii) if stopping criteria is achieved, stop. Otherwise, go

to step 3).

(1)

(2)

(3)

(4)


Fig. 3. Process of updating the sequence of states in each row recursively. States and observed vectors in the shaded rows are included in the condition for solvingfor the sequence of states in a current row that has the MAP probability. After initialization, the scan through all the rows can be repeated in several passes.

In our experiments, we stop the iterations when the numberof points whose states are changed by the present iteration, isbelow a preselected threshold.

The objective function is guaranteed to decrease with eachiteration. It is a possibility that cycles could develop after cer-tain iterations. However, in such a case, the objective functionremains constant while states keep changing. In all our exper-iments, we noticed that states as well as the objective functionconverged after a finite number of iterations. The preselectedthreshold was very small which guaranteed that the algorithmhad achieved near convergence. The convergence could be toan optima different from the global optima. The initial states ofthe 3-D grid are obtained by a greedy algorithm. In particular,we scan the rows in the lexicographic order. For each row, thesequence of states with the MAP probability conditioned on thestates in all the preceding rows and the observed vectors in thecurrent row is selected. The algorithm is greedy because states ineach row are determined without looking forward on succeedingrows. The difference between the search for states in each rowin initialization and the search in step 4) above lies in the con-ditioned information. In initialization, only “past” information(in the lexicographic order) is conditioned on. During the ad-justment of each row, information about all the other rows isconditioned on. This difference is illustrated in Fig. 3. We nowoutline steps in initialization.

1) Let , .2) Search for with the MAP probability given the states

in preceding rows: , , and the ob-served vectors in the current row and the preceding rows:

, , .

3) Let .4) If , go back to step 2). Otherwise

a) set , ;b) if , go back to step 2). Otherwise, stop.

C. Local Optimization of States in a Row

Next, we present the Viterbi algorithm applied to search forthe sequence of states with the MAP probability given allthe observed vectors and the states in all the other rows, i.e.,

Note that the conditioned states (assumed given) are denotedby in order to distinguish from the states to be op-timized. This state sequence is needed in the row by rowadjustment of states after initialization in the proposed locallyoptimal algorithm. During initialization of states, as mentionedbefore, is defined as the MAP sequence conditioned onlyon states in the preceding rows. We describe the Viterbi algo-rithm in the context of row by row adjustment. Its usage in theinitialization case is similar. We, thus, omit detailed discussion.

For brevity, we assume the th row is not at the boundary.Modification for boundary rows is straightforward. It is provenin the appendix that is given by the (6) and (7), shown at thebottom of the next page. The first term in the summand is thelog likelihood of the observed vector given the state .The probabilities in the log function in the other three termsare transition probabilities of entering a state at point ,

, and respectively. Only states in the four


neighboring rows of , specifically, , , ,and , are relevant to the search of . For a point at thebeginning of a row, it has no preceding point along the axis.Correspondingly, in the above conditional probabilities, the con-ditioned state of the preceding neighbor along the axis isdropped. For brevity, we use a unified expression here.

The optimal state sequence can be solved by the fol-lowing recursion in the Viterbi algorithm.

1) Let for .2) For , compute the following.

a) , where

b) Let be the number fromthat maximizes the above function:

. In case morethan one candidates exist for , which maximizethe function, we choose one of them randomly, in thepresent implementation.

3) Let .4) For , let .

D. Computational Complexity

If the number of states of the model is , computa-tion and memory is required to find and store state com-binations which maximize the likelihood term involving the cur-rent point and its previous one. In order to search for the MAPsequence of states in the entire row of points, the computa-tion required is of the order of . For a 2-D grid containing

points (or rather rows), one iteration of the algorithmthrough the grid will take time. A 3-D grid con-taining points has a total of rows. Therefore, thecomplexity of one iteration of the proposed algorithm throughthe 3-D grid is . Thus, we see that the computationalcomplexity of the proposed algorithm is polynomial in numberof states and linear in problem size for both 2-D HMM and 3-DHMM. The problem size, here, represents the number of pixelsin an image (for 2-D HMM, the problem size is and for 3-DHMM, the problem size is ).

It was shown in [24] that the likelihood function for a 2-Dimage represented as a grid could be expressed as aproduct of separable functions of state sequences in diagonals. If

represents all the tuples in the grid, then the likelihoodfunction can be factorized as follows:

(8)

In the previous expression, denotes the state sequence ondiagonal . A path-constrained variable state Viterbi approachwas proposed, which restricted the search for state sequences ateach diagonal to the sequences with largest posterior proba-bilities. Each term in (8), can be represented as aproduct of likelihood terms corresponding to individual pointsin diagonal , owing to their conditional independence. If thereare points on diagonal , the unconstrained Viterbi approachwould normally involve searching all state sequences forthe sequences which yield highest probabilities. However,because of the conditional independence of points in diagonal, this search can be restricted to searching sequences com-

posed of states, which give highest probabilities at each point.For elaboration, suppose diagonal contains points (num-bered ) and the log likelihood of point being in state

is denoted as . The selection of nodes is equivalent tofinding state sequences, , with the largest

. As a consequence of conditional independence ofpoints on the diagonal the following equality holds:

(9)

Searching for is an operation (whereis the number of states of the model) and so finding the max-

imum sequence is an operation. This approach can beeasily extended to finding sequences with largest posteriorprobabilities. For more details, readers are referred to [24].

Under the path-constrained variable state Viterbi approach,an image is broken down into square subimages of sizeand each subimage is modeled as a separate 2-D HMM. Thus,for a 2-D image represented as a grid, separate 2-D

(6)

(7)


Fig. 4. (a)–(c) Three aerial images and their manual classification. Left: Original 8-bpp images, Right: Hand-labeled classified images. White represents man-madeand gray represents natural.

HMMs are estimated. The value of the subimage size is gen-erally selected as 8. Path constrained variable-state Viterbi algo-rithm, for each subimage, can be performed intime and so the computational complexity for the entire imageis roughly . It would be an interesting future workto extend this approach to be used for 3-D HMMs and comparewith the proposed algorithm. For the remainder of the paper, wewill refer to the estimation algorithm based on path-constrainedvariable state Viterbi as Algorithm and the estimation algo-rithm proposed in this paper as Algorithm .

V. EXPERIMENTS ON 2-D IMAGES

Our experiments with 2-D images were focused on com-paring the proposed estimation algorithm to the algorithmbased on variable state Viterbi . The goal of the experimentswas classification of aerial images into man-made and naturalregions. The experimental setup was similar to the setup in [24].Although classification results, for algorithm , are availablein [24], we performed all the experiments again in order tocorrectly compare the learning and classification times of thetwo estimation programs, when run on similar processors.

The six images used are 512 512 grayscale images with 8bits per-pixel (bpp). The images are the aerial images of the SanFrancisco Bay area provided by TRW [24]. The availability ofhand-labeled classification for the six images proved helpful inevaluating the performances. The manually classified imageswere used as the truth set for our experiments. The six im-ages and their respective hand-labeled classification are shownin Figs. 4 and 5.

The features extracted from images were the intra-block fea-tures used for experiments in [24]. No inter-block features wereused in the experiments performed. For clarity, we elaborate

the feature extraction process. The images were divided into4 4 blocks, and the discrete cosine transform (DCT) coeffi-cients or averages over them were calculated and used as fea-tures. If the DCT coefficients for a 4 4 block are denoted by

, the definitions of the six featuresused are

An initial -means clustering was performed. However, the number of states for each

class (man-made class and natural class) were variable. Forexample, if represents the total number of states of themodel, then states were assigned to the first classwhile were assigned to the second class. Theoptimization in Viterbi was performed by constraining thefeasible states to lie in the classes given by the training data.In case of multiple states per class, one of the designated stateswas randomly assigned to points, during initialization. Theparameters , and were varied during the experiments. Asix-fold cross validation was used to compare the classificationperformances of the algorithms. At each iteration, one imagewas used as test data and the other five were used as training


Fig. 5. (d)–(f) Three aerial images and their manual classification. Left: Original 8-bpp images, Right: Hand-labeled classified images. White represents man-madeand gray represents natural.

TABLE IPARAMETERS USED FOR ALGORITHM B FOR EACH ITERATION IN 2-D IMAGE

CLASSIFICATION EXPERIMENT. N AND N REPRESENT THE NUMBER OF

STATES FOR THE NATURAL AND MAN MADE CLASSES, RESPECTIVELY

data. The five training images were tiled from top to bottom, toform one composite training image.

A regularization of 2-D transition probabilities toward 1-Dprobabilities was performed, during estimation, in a fashionsimilar to (5). The parameters to be used for each iteration(number of states to model each class and the value of )were, in turn, estimated using a fivefold cross validation withineach training data set. In our case, each training set consistedof five images and at each iteration, four of them were usedfor learning the model and one image was used as testing themodel. The number of states per class was varied between 4and 10, and was varied between 0.0 and 1.0. At each step, theparameter combination yielding the best average performanceover five iterations, was selected, for testing. The parametercombination used at each iteration is shown in Table I.

In our algorithm, the training phase involved estimating theoptimal state configuration and the transition probabilities. Atthe testing phase, the learnt transition probabilities, means andcovariances were used to find the optimal state configurationand, hence, the misclassification error. In this paper, we reportthe average performance over all six iterations. If the man-made

class is assumed to be the target class, or the positive class, wedefine the following.

1) Sensitivity is the true positive ratio is the probability ofdetecting positive given that the truth is positive.

2) Specificity is the true negative ratio is the probability ofdetecting negative given that the truth is negative.

3) Predictive positive value (PVP) is the probability of beingtruly positive given a positive detection of the classifier.

4) Misclassification error is the probability of not de-tecting the correct class (positive or negative).

Fig. 6 compares the classification performances of algorithmsand for one particular image from the dataset. The original

and hand labeled images are also shown and the classificationerror rates are indicated in the figure. The classification perfor-mances for the aerial images shown in Figs. 4 and 5 is reportedin Table II. The values reported are the sensitivity, specificity,PVP and error rate averaged over all six iterations. All the ex-periments were performed on 2.6 GHz Xeon processors run-ning Linux. The CPU times required by Algorithms and forlearning and classification, respectively, are shown in Table III.

We notice from Tables II and III that as is increased for Al-gorithm , the error rate decreases but the model learning andclassification times go up. The average error rates usingalgorithms and are found to be comparable. However, therunning times for learning as well as classification using Algo-rithm are much less compared to Algorithm . These resultssupport the fact that the proposed algorithm has a lowercomputational complexity compared to algorithm .

VI. EXPERIMENTS ON 3-D IMAGES

The 3-D HMM has been applied to volume image segmen-tation in this paper. Experiments were performed using a large


Fig. 6. Compare the performances of Algorithm A and B. (a), (b) Original8-bpp image and the hand-labeled classified image. (c) Algorithm A withclassification error rate 15.8%. (d) Algorithm B with classification error rate15.3%.

TABLE IIPERFORMANCE OF ALGORITHMS A AND B FOR 2-D IMAGE CLASSIFICATION.

THE AVERAGE PERFORMANCE OVER SIX ITERATIONS IS SHOWN IN THE TABLE

TABLE IIIRUNNING TIMES OF ALGORITHMS A AND B FOR 2-D IMAGE CLASSIFICATION

pool of synthetic volume images. Each image contained a stan-dard mathematical 3-D shape centered about the image. Themathematical shapes used will be discussed later. The method ofimage generation was as follows. Points in the interior of the 3-Dshape were assigned black color while the rest were white. Eachcolor voxel, black and white , was perturbedby an additive Gaussian noise and the voxel valueswere truncated to lie in the interval . This per-turbed value formed the unidimensional feature for each voxel.

For the purpose of displaying images, voxel values in the in-terval were scaled to . The 3-D shape pa-rameters and the variance of noise were varied to form a largepool of images.

The term class denotes the classes into which an image isto be classified whereas states denotes the states of the 3-DHMM model. The number of classes is fixed by the classifica-tion problem definition whereas the number of states of the 3-DHMM model can be varied with experiments. When the numberof classes is known a priori, unsupervised segmentation using3-D HMM involves the following steps.

1) An initial clustering using -means algorithm is per-formed and each point is assigned a state depending uponits initial cluster. In case of more than one states percluster, one of them is assigned randomly. The number ofstates, fixed for each cluster is arbitrary and is a parameterfor the segmentation program. The number of clusterscorresponds to the number of classes, which is 2.

2) Initial parameters are estimated using ML criteria as-suming the initial states as true states.

3) 3-D HMM-based unsupervised learning is performed it-eratively. In other words, MAP states are assigned topoints, as discussed before, and parameters re estimatediteratively.

4) Final classes are given to points based on the states theyhave been assigned by the algorithm.

A. Geometric Equations of 3-D Shapes

The mathematical equations for the topological shapes usedfor the experiments were adopted from Mathworld1 and aregiven as follows.

1) Torus: A torus which is azimuthally symmetric about theaxis is represented by the following equation:

Here, is the radius from the center of the hole to thecenter of the torus tube while is the radius of the tube.A torus is depicted in Fig. 7(a).

2) Ellipsoid: An ellipsoid is a quadratic surface which isrepresented by the following equation:

Here, , and represent the lengths of the semi-axes.An ellipsoid is depicted in Fig. 7(b).

3) Hyperboloid: A hyperboloid is a quadratic surface whichis represented by the following equation:

Here, , , and represent the lengths of the semi-axes.A hyperboloid is depicted in Fig. 7(c).

B. Algorithm for Shape Generation

We used the following simple algorithm to vary the parame-ters , , and in order to generate images used for segmentation

1http://mathworld.wolfram.com


Fig. 7. Some 3-D shapes are illustrated in the figure. The pictures are adopted from Mathworld Website. The figures represent (a) Torus, (b) Ellipsoid, and(c) Hyperboloid.

Fig. 8. Segmentation of a torus with 3-D HMM. The figure shows three frames, one in each of the X-Y , Y -Z and X-Z planes. (a)–(c) Original image.(d)–(f) Segmented image. The parameters are a = 15, c = 35, and � = 0:6. The error rate is 1.2%.

Fig. 9. Segmentation of a torus with 3-D HMM. The figure shows three frames, one in each of the X-Y , Y -Z and X-Z planes. (a)–(c) Original image.(d)–(f) Segmented image. The parameters are a = 5, c = 25, and � = 0:5. The error rate is 12.8%.

study. For clarity, we restate that geometrically, the parameters, , and represent the length of the semi-axes for ellipsoids

and hyperboloids, while and represent the length of radii, fortorii. In the following pseudo-code, subscripts low, up, and steprepresent the lower, upper limits, and the step size used to varythe respective parameter.

1) ; ; .2) Generate 3-D image containing Shape centered

about the image containing Gaussian noise , as ex-plained before.

3) If ; then and goto step 2); else gotostep 4).

4) If ; then and goto step 2); else gotostep 5).

5) If ; then and goto step 2); elsestop.

For the parametric studies performed here, the sizes of theimages was fixed at 100 100 100 and the limits, and the stepsizes chosen were as follows:

• , , ; , ,; , , .

A total of 70 images can be generated with these parameter set-tings (10 for class torii, 30 each for classes ellipsoids and hy-perboloids) for a given noise parameter . The parameter was

varied from 0.2 to 0.7 to generate images having a wide rangeof Gaussian noise.

C. Parametric Studies, Results and Discussion

The main advantage of evaluation of 3-D HMM under thechosen experimental setup is the availability of ground truthfor every image. Moreover, performance over a large pool ofimages instills greater confidence in the suitability of the pro-posed algorithm for 3-D image segmentation. However, it mustbe mentioned that some of the resulting shapes are more diffi-cult to segment than the others if certain extreme parameter set-ting is used (for example if the semi-major axes or the radii aretoo small). Figs. 8–13 illustrate segmentation of a few 3-D im-ages containing torii, ellipsoids, and hyperboloids respectively.In each case, three frames of the original and segmented im-ages are shown. The parameters used are indicated in the figurecaptions.

Three separate parametric studies were performed with thepool of 70 images in order to determine the sensitivity of the 3-DHMM algorithm to the various parameters. These are explainedas follows.

1) The model parameter was varied between 0 and 1 insteps of 0.2 and segmentation performance noted for each

. The number of states were fixed at 3 for each class.Results for and are shown here. For


Fig. 10. Segmentation of an ellipsoid with 3-D HMM. The figure shows three frames, one in each of the X-Y , Y -Z and X-Z planes. (a)–(c) Original image.(d)–(f) Segmented image. The parameters are a = 35, b = 40, c = 30, and � = 0:6. The error rate is 6.5%.

Fig. 11. Segmentation of an ellipsoid with 3-D HMM. The figure shows three frames, one in each of the X-Y , Y -Z and X-Z planes. (a)–(c) Original image.(d)–(f) Segmented image. The parameters are a = 35, b = 25, c = 30, and � = 0:6. The error rate is 3.3%.

Fig. 12. Segmentation of a hyperboloid with 3-D HMM. The figure shows three frames, one in each of the X-Y , Y -Z and X-Z planes. (a)–(c) Original image.(d)–(f) Segmented image. The parameters are a = 25, b = 35, c = 30, and � = 0:6. The error rate is 3.1%.

Fig. 13. Segmentation of a hyperboloid with 3-D HMM. The figure shows three frames, one in each of the X-Y , Y -Z and X-Z planes. (a)–(c) Original image.(d)–(f) Segmented image. The parameters are a = 35, b = 25, c = 30, and � = 0:6. The error rate is 1.7%.

clarity, we have shown two tables (one each forand ) for torii, ellipsoids, and hyperboloids. Re-sults for torii are shown in Tables IV and V; results forellipsoids are shown in Tables VI and VII; while resultsfor hyperboloids are shown in Tables VIII and IX. Due tolack of space, we have shown results for six images foreach class and the respective shape parameters are alsoindicated. The best performance, which usually occurs atan intermediate value of , is indicated in bold. The pa-rameter represents the extent of 3-D dependence in themodel. A trade off between model complexity (complete3-D model, ) and ease of estimation (2-D model,

), is preferred in most cases, and the results supportthis hypothesis.

2) The Gaussian noise parameter was varied between 0.2and 0.7 in steps of 0.1. The number of states were fixedat 3 for each class and was fixed at 0.6. Segmentation

was performed for all 70 images for each . The best andmedian segmentation performances for different valuesof are shown in Table X. As is evident from the results,3-D HMM performs reasonably well segmentation evenfor large values of .

3) The number of states for each class were varied from 3 to9 in steps of 3. was fixed at 0.6. The best and medianperformances for noise parameter are shown inTable XI. Results show that models with less number ofstates per class perform better than models with highernumber of states per class.

We discovered that the best value of varied largely with theshape to be segmented. Based on the experiments performed,an intermediate value of , in the interval usually per-formed better than extreme values. In the future, we would liketo look into how we can theoretically characterize the perfor-mance of 3-D HMM model, with the smoothing parameter .


TABLE IVSEGMENTATION PERFORMANCES OF A FEW IMAGES OF CLASS TORII WITH

SHOWN PARAMETERS (SIZE 100� 100� 100 AND � = 0:5)AS THE PARAMETER � IS VARIED BETWEEN 0 AND 1.

THE BEST PERFORMANCE IS INDICATED IN BOLD

TABLE VPERFORMANCES OF A FEW IMAGES OF CLASS TORII WITH SHOWN

PARAMETERS (SIZE 100� 100� 100 AND � = 0:6) AS THE PARAMETER � IS

VARIED BETWEEN 0 AND 1. THE BEST PERFORMANCE IS INDICATED IN BOLD

TABLE VISEGMENTATION PERFORMANCES OF A FEW IMAGES OF CLASS ELLIPSOIDS

WITH SHOWN PARAMETERS (SIZE 100� 100� 100 AND � = 0:5)AS THE PARAMETER � IS VARIED BETWEEN 0 AND 1.


TABLE VIISEGMENTATION PERFORMANCES OF A FEW IMAGES OF CLASS ELLIPSOIDS



The running times of 3-D HMM segmentation program forimage sizes where takes values 50, 100, 150, and200 were found out to be 32, 280, 798, and 938 s respectively.These numbers support the fact that the complexity of the algo-

TABLE VIIISEGMENTATION PERFORMANCES OF A FEW IMAGES OF CLASS HYPERBOLOIDS



TABLE IXSEGMENTATION PERFORMANCES OF A FEW IMAGES OF CLASS HYPERBOLOIDS



TABLE XBEST AND MEDIAN SEGMENTATION PERFORMANCES OVER 70 IMAGES (SIZE

100� 100� 100) AS THE VARIANCE OF THE GAUSSIAN NOISE VARIES

TABLE XIBEST AND MEDIAN SEGMENTATION PERFORMANCES OVER 70 IMAGES

(SIZE 100� 100� 100 AND � = 0:6) AS THE NUMBER

OF STATES FOR EACH CLASS ARE VARIED

TABLE XIISEGMENTATION PERFORMANCE OF A 3-D IMAGE OF AN ELLIPSOID (SIZE

100� 100� 100, a = 40, b = 25, c = 30, � = 0:5) UNDER VARYING

DEGREES OF ROTATION. THE VALUE OF � USED IS 0.6

rithm is linear in the number of pixels in an image , whenother parameters are constant.

D. Studying Effect of Rotation

In order to study the effect of lexicographic ordering on unsu-pervised segmentation performance, a 3-D image of an ellipsoid


Fig. 14. Section of a 3-D image of an ellipsoid (size 100� 100� 100, a = 40, b = 25, c = 30, � = 0:5) under varying degrees of rotation. (a) 0 , (b) 10 ,(c) 30 , (d) 60 , and (e) 90 .

( , , , ), was subjected to indepen-dent rotations about the , , and axes. The angle of rotationwas varied from 0 to 90 , in steps of 10 .

The misclassification error rate for varying degrees of rotationis shown in Table XII. Fig. 14 shows a section of the original aswell as segmented image under different angles of rotation. Ascan be seen from the results, segmentation performance is quitestable over a range of rotations.

VII. CONCLUSION AND FUTURE WORK

In this paper, we proposed 3-D HMMs and implemented acomputationally fast parameter estimation technique which canbe used for both 2-D HMMs and 3-D HMMs. Supervised clas-sification of 2-D aerial images was performed using 2-D HMMswith the proposed estimation algorithm and compared with theestimation algorithm which had been proposed initially for 2-DHMMs. The classification performances were found to be com-parable while the running time of the proposed estimation al-gorithm was much faster than the old algorithm. Next, we per-formed unsupervised segmentation of a large number of syn-thetically generated 3-D images using 3-D HMM. Parametricstudies varying model and noise parameters were performed andresults were reported. Results show that 3-D HMM can be usedas a powerful tool for modeling 3-D images. We identify the fol-lowing issues for future work.

• It would be interesting to compare 3-D HMM-based seg-mentation with segmentation based on 3-D MRF algo-rithms.

• Multiresolution information has proved helpful in 2-Dimage classification. 2-D multiresolution HMMs (2-DMHMMs) have been successfully applied to 2-D image

modeling. In 2-D MHMMs, feature vectors are extractedfrom an image at multiple resolutions and spatially ar-ranged on a pyramid grid. Thus, a 2-D MHMM modelsboth intra and inter-resolution relationships betweenextracted feature vectors. The proposed estimation al-gorithm can be incorporated into 2-D multiresolutionHMMs for better computational efficiency. Extending3-D HMMs to 3-D multiresolution HMMs is also ofconsiderable interest to us.

• A plausible application of 3-D HMM could be inhyper-spectral image analysis and classification wherethe third dimension is the spectral dimension. Our 3-DHMM model can capture statistical dependence amonga pixel and its spatial or spectral neighbors and would,thus, be of use. Another application of our model couldbe in video modeling, where the third dimension is thetemporal dimension.

APPENDIX

We now prove (7), that is

(10)

is solved by (11), shown at the bottom of the page. Defineand

, which are the collections of indices for rows precedingand those succeeding , respectively (see the equation

shown at the bottom of the next page).The second equality is obtained by the chain rule of condi-

tional probability and the assumption that given the state of apoint, the observed vector is conditionally independent of any

(11)


other point. By deleting terms that are irrelevant to , we have

(12)

Since given the states of all the preceding points, the probabilityof a point being in a certain state only depends on the threeneighboring points that are one position ahead along each axis,the first term in (12) can be reduced to

(13)

The second term in (12) can be expanded by the chain rule ofconditional probability and the 3-D Markovian assumption onthe states

(14)

where . Note that the lastterm in (14) does not involve and, hence, can be droppedwhen searching for . Combining (12)–(14), we have

The last equality comes from expanding the conditional proba-bilities of states along the rows. For computational convenience,we can search equivalently the maximum of the log of the ob-jective function, which leads to (11).

ACKNOWLEDGMENT

The authors would like to thank anonymous reviewers of theinitial manuscript.

REFERENCES

[1] J. Boreczky and L. Wilcox, “A hidden Markov model frameworkfor video segmentation using audio and image features,” in Proc.IEEE Conf. Acoustics, Speech, Signal Processing, vol. 6, 1998, pp.3741–3744.

[2] C. A. Bouman and M. Shapiro, “A multiscale random field model forBayesian image segmentation,” IEEE Trans. Image Process., vol. 3, no.2, pp. 162–177, Mar. 1994.

[3] “Markov random fields with efficient approximations,” in Proc. Com-puter Vision Pattern Recognition, Santa Barbara, CA, Jun. 1998, pp.648–655.

[4] F. Champagnat, J. Idler, and Y. Goussard, “Stationary Markov randomfields on a finite rectangular lattice,” IEEE Trans. Inf. Theory, vol. 44,no. 7, pp. 2901–2916, Jul. 1998.

[5] R. Chellappa and S. Chatterjee, “Classification of textures usingGaussian Markov random fields,” IEEE Trans. Acoustics, Speech,Signal Process., vol. ASSP-33, no. 4, pp. 959–963, Aug. 1985.

[6] R. Chellappa and A. K. Jain, Markov Random Fields: Theory and Ap-plications. New York: Academic, 1993.

[7] G. R. Cross and A. K. Jain, “Markov random field texture models,” IEEETrans. Pattern Anal. Mach. Intell., vol. PAMI-5, no. 1, pp. 25–39, Jan.1983.

[8] M. S. Crouse, R. D. Nowak, and R. G. Baraniuk, “Wavelet-based sta-tistical signal processing using hidden Markov models,” IEEE Trans.Signal Process., vol. 46, no. 4, pp. 886–902, Apr. 1998.

[9] H. Derin, H. Elliott, H. Cristi, and D. Geman, “Bayes smoothingalgorithms for segmentation of images modeled by Markov randomfields,” in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing,San Diego, CA, Mar. 1984, pp. 682–685.

[10] P. L. Dobruschin, “The description of a random field by means of condi-tional probabilities and conditions of its regularity,” Theory Probl. Appl.,vol. 13, no. 2, pp. 197–224, 1968.

[11] S. Eickeler, S. Muller, and G. Rigoll, “Improved face recognition usingpseudo 2-D hidden Markov models,” presented at the Workshop Ad-vances in Facial Image Analysis and Recognition Technology in con-junction with ECCV, Freiburg, Germany, Jun. 1998.

[12] S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions,and the Bayesian restoration of images,” IEEE Trans. Pattern Anal.Mach. Intell., vol. PAMI-6, no. 6, pp. 721–741, Jun. 1984.

[13] Z. Ghahramani, “An introduction to hidden Markov models andBayesian networks,” Int. J. Pattern Recognit. Artif. Intell., vol. 15, no.1, pp. 9–42, 2001.

[14] M. I. Gurelli and L. Onural, “On a parameter estimation method forGibbs-Markov random fields,” IEEE Trans. Pattern Anal. Mach. Intell.,vol. 16, no. 4, pp. 424–430, Apr. 1994.

[15] F. R. Hansen and H. Elliott, “Image segmentation using simple Markovfield models,” Comput. Graph. Image Process., vol. 20, no. 8, pp.101–132, 1982.

[16] D. S. Hochbaum, “An efficient algorithm for image segmentation,Markov random fields and related problems,” J. ACM, vol. 48, no. 4,pp. 686–701, 2001.

[17] X. D. Huang, Y. Ariki, and M. A. Jack, Hidden Markov Models forSpeech Recognition. Edinburgh, U.K.: Edinburgh Univ. Press, 1990.

[18] F. Hulsken, F. Wallhoff, and G. Rigoll, “Facial expression recognitionwith pseudo-3D hidden Markov models,” in Proc. DAGM-Symp., Lec-ture Notes Computer Science, vol. 2191, 2001, pp. 291–297.

[19] F. -C. Jeng and J. W. Woods, “Compound Gauss-Markov random fieldsfor image estimation,” IEEE Trans. Signal Process., vol. 39, no. 3, pp.683–697, Mar. 1991.


[20] R. Kindermann and L. Snell, Markov Random Fields and Their Appli-cations. Providence, RI: Amer. Math. Soc., 1980.

[21] J. Li, R. M. Gray, and R. A. Olshen, “Joint image compression andclassification with vector quantization and a two dimensional hiddenMarkov model,” in Proc. Data Compression Conf., Snowbird, UT, Mar.1999, pp. 23–32.

[22] , “Multiresolution image classification by hierarchical modelingwith two dimensional hidden Markov models,” IEEE Trans. Inf. Theory,vol. 46, no. 5, pp. 1826–1841, May 2000.

[23] J. Li and R. M. Gray, Image Segmentation and Compression UsingHidden Markov Models. New York: Kluwer, 2000.

[24] J. Li, A. Najmi, and R. M. Gray, “Image classification by a two dimen-sional hidden Markov model,” IEEE Trans. Signal Process., vol. 48, no.2, pp. 517–533, Feb. 2000.

[25] J. Li and J. Z. Wang, “Automatic linguistic indexing of pictures by astatistical modeling approach,” IEEE Trans. Pattern Anal. Mach. Intell.,vol. 25, no. 9, pp. 1075–1088, Sep. 2003.

[26] S. Z. Li, Markov Random Field Modeling in Computer Vision. London,U.K.: Springer-Verlag, 1995.

[27] M. Lievin, N. Hanssen, P. Zerfass, and E. Keeve, “3D Markov randomfields and region growing for interactive segmentation of MR data,”in Proc. Medical Image Computing Computer-Assisted Intervention,Utrecht, The Netherlands, Oct. 2001, pp. 1191–1192.

[28] M. R. Leuttgen, W. C. Karl, A. S. Willsky, and R. R. Tenney, “Multiscalerepresentations of Markov random fields,” IEEE Trans. Signal Process.,vol. 41, no. 12, pp. 3377–3396, Dec. 1993.

[29] B. S. Manjunath and R. Chellappa, “Unsupervised texture segmentationusing Markov random field models,” IEEE Trans. Pattern Anal. Mach.Intell., vol. 13, no. 5, pp. 478–482, May 1991.

[30] G. J. McLachlan and D. Peel, Finite Mixture Models. New York:Wiley, 2000.

[31] D. E. Melas and S. P. Wilson, “Double Markov random fields andBayesian image segmentation,” IEEE Trans. Signal Process., vol. 50,no. 2, pp. 357–365, Feb. 2002.

[32] D. Panjwani and G. Healey, “Markov random field models for unsuper-vised segmentation of textured color images,” IEEE Trans. Pattern Anal.Mach. Intell., vol. 17, no. 10, pp. 939–954, Oct. 1995.

[33] W. Pieczynski and A.-N. Tebbache, “Pairwise Markov random fieldsand its application in textured images segmentation,” in Proc. SouthwestSymp. Image Analysis and Interpretation, Austin, TX, Apr. 2000, pp.106–110.

[34] H. Rue, I. Steinsland, and S. Erland, “Approximating hidden GaussianMarkov random fields,” J. Roy. Statist. Soc., vol. 66, no. 4, pp. 877–892,2004.

[35] C. S. Won and H. Derin, “Unsupervised segmentation of noisy andtextured images using Markov random fields,” Graph. Models ImageProcess., vol. 54, no. 4, pp. 308–328, 1992.

[36] L. Xie, S. F. Chang, A. Divakaran, and H. Sun, “Learning hierarchicalhidden Markov Models for video structure discovery,” ADVENT Group,Columbia Univ., New York, 2002, L. Xie, S. F. Chang, A. Divakaran, H.Sun, “Unsupervised discovery of multilevel statistical video structuresusing hierarchical hidden Markov models,” Proc. International Conf.Multimedia Expo., vol. 3, pp. 29-32, Baltimore, MD, Jul. 2003.

[37] S. Young, J. Jansen, J. Odell, D. Ollason, and P. Woodland, HTK—Hidden Markov Model Toolkit. Cambridge, U.K.: Cambridge Univ.,1995.

[38] J. Zhang, J. W. Modestino, and D. A. Langan, “Maximum-likelihood pa-rameter estimation for unsupervised stochastic model-based image seg-mentation,” IEEE Trans. Image Process., vol. 3, no. 4, pp. 404–420, Jul.1994.

[39] Y. Zhang, M. Brady, and S. Smith, “Segmentation of brain MR imagesthrough a hidden Markov random field model and the expectation-maxi-mization algorithm,” IEEE Trans. Med. Imag., vol. 20, no. 1, pp. 45–57,Jan. 2001.

[40] S. C. Zhu, X. W. Liu, and Y. N. Wu, “Exploring texture ensembles byefficient Markov chain Monte Carlo—Toward a Trichromacy theory oftexture,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 6, pp.554–569, Jun. 2000.

Dhiraj Joshi (S’02) received the Integrated Mastersdegree in mathematics and scientific computingfrom the Indian Institute of Technology (IIT),Kanpur, in 2002, where he worked on real-parameterevolutionary algorithms. He is currently pursuing thePh.D. degree in computer science and engineeringat The Pennsylvania State University (Penn State),University Park.

He is currently a Research Assistant at the Intelli-gent Information Systems Research Laboratory, PennState. He was a Research Intern at the Dalle Molle

Institute for Perceptual Artificial Intelligence (IDIAP), Switzerland, where heworked on social network analysis. His current research interests include statis-tical modeling, content analysis, and multimedia information retrieval.

Jia Li (S’95–M’99–SM’05) received the B.S. degreein electrical engineering from Xi’an Jiao TongUniversity, China, in 1993, and the M.Sc. degree inelectrical engineering, the M.Sc. degree in statistics,and the Ph.D. degree in electrical engineering fromStanford University, Stanford, CA, in 1995, 1998,and 1999, respectively.

She is an Assosciate Professor of statistics and bycourtesy appointment in computer science and engi-neering at The Pennsylvania State University, Uni-versity Park. She was a Research Associate in the

Computer Science Department, Stanford University, in 1999. She was a Re-searcher at the Xerox Palo Alto Research Center, Palo Alto, CA, from 1999 to2000. Her research interests include statistical classification and modeling, datamining, bioinformatics, image processing, and image retrieval.

James Z. Wang (S’96–M’00–SM’06) received theB.S. degree (summa cum laude) in mathematicsand computer science from the University of Min-nesota, Minneapolis, in 1994, the M.Sc. degrees inmathematics and computer science from StanfordUniversity, Stanford, CA, in 1997, and the Ph.D.degree in medical information sciences from theBiomedical Informatics Program and ComputerScience Database Group, Stanford University, in2000.

He is an Associate Professor at the College of In-formation Sciences and Technology and the Department of Computer Scienceand Engineering, The Pennsylvania State University, University Park. From2000 to 2006, he was the holder of the endowed PNC Technologies CareerDevelopment Professorship.

Dr. Wang is a recipient of an NSF Career award in support of his researchprogram.

ieee transactions on image processing,...

Documents