Detecting Chemical Plumes from Long-Wave

Infrared Hyperspectral Images

Jor-el Briones, Kevin Bui, Sihan Chen, Justin Sunu

August 9, 2013

Abstract

Chemical plumes come in many different shapes and diffusive patterns,and can occur in different environments. The goal of this paper is to im-prove previously available works onto newer data sets. We also focused onpre-processing the data before applying numerous clustering techniques.The two main cleaning techniques are Orthogonal Subspace Projection,which removes background signatures, and Manifold denoising, which re-moves noise. The clustering techniques we utilized were K-means, FuzzyK-means, Fuzzy C-means, Entropy based clustering, spectral clustering,and Nystrom extension. As an alternative to clustering methods, we uti-lized unmixing methods, in the form of L1 unmixing and Non-NegativeMatrix Factorization. The new distance metrics that we utilized wereSpectral Gradient Angle, Spectral Information Divergance measure, andthe Haussdorf metric. We compared the different distance metrics andclustering methods to better analyze the geometry of the data. It appearsthat in different situations, different pairs of clustering methods and dis-tance metrics work better. Both Orthogonal Subspace Projection andManifold denoising improved both the detection and clustering of plumesin even the most problematic of the data sets.

1 Introduction

Airborne toxin can potentially become a threat to a public safety since it can bea potential chemical weapon by terrorists or a result of an industrial accident.In order to contain these dangerous gases, we need to be able to detect themas soon as possible. To achieve this, we need to develop a detection algorithmthat best captures and locates the plume of airborne toxin.

Our goal in this paper is to be able to identify plumes of airborne toxinin hyperspectral images. Past research has been done to identify them throughvarious clustering techniques. In this paper, we will mainly continue the resultsmade from Gerhart et al.

Gerhart et al. has shown that k-means and spectral clustering have beensuccessful in identifying gas plume pixels, but they have encountered variousproblems, such as strong background signals and noises interfering with the

1

clustering results. Thus, in this paper, we will extend the results discussed inGerhart et al. to new datasets to determine if they are robust and also presentmethods that improve their results or solve the problems they encountered.

Our datasets are the hyperspectral images recorded at the Dugway ProvingGround in Utah. The cameras that recorded them are Romeo, Tango, andVictory, but in this paper, we will mainly focus on the images recorded byVictory and discuss the challenges encountered in the images by Romeo. Inaddition, there were four gas release experiments and they are the following:

aa12 contains gas releases from an ex-plosion

aa21 contains gas released from an ex-plosion, has a laser middle

R134a7 is a gas release sf6-32 is a gas release, is very faint

Figure 1: Types of gas releases done at the Dugway Proving Ground in Utah

The preprocessing of our data sets are based on the same methods doneby Gerhart et al.

2 Manifold Denoising

The technique of manifold denoising[11] is utilized to remove noise and to makethe data more ”compact.” The algorithm is described in Algorithm 1. It ba-sically performs the inverse diffusion process. An example of the process isseen in Figure 2. As seen in this figure, there is an increase in the distinctionbetween the gas plume and the background, allowing for a better clustering re-sult. Another benefit of the denoising process is the movement of the importanteigenvectors in spectral clustering. Figure 3 shows both the movement and thebetter separation that can be after applying manifold denoising on the data.

3 Orthogonal Subspace Projection

Orthogonal Subspace Projection is a method of background subtraction. Themain goal of orthogonal subspace projection is to eliminate the influence ofbackground signatures in an image and be left only with the target signature.

2

Algorithm 1 Manifold Denoising

Input: δt, k, raw data XInitialize X0 = XWhile condition not metCompute the graph Laplacian, ∆ with self tuning σSolve Xi+1 = (I + δt∆)−1Xi

Alternatively: solve the system of equation for Xi+1 in (I + δt∆)Xi+1 = Xi

Repeat

Figure 2: Example of Manifold Denoising being applied onto data with spectralclustering. The left shows the spectral clustering results before denoising. Theresults on the right show after the denoising process.

Before denoising 26th eigenvector After denoising 12 eigenvector

Figure 3: Example of Manifold Denoising being applied onto data with spectralclustering. The left shows the spectral clustering results before denoising. Theresults on the right show after the denoising process.

This method is especially useful in images with strong background signaturesthat overpower the target signature, or in images where the target signatureis so diffuse that the it does not contrast well enough from the backgroundfor clustering members to distinguish it from the background. We begin withthe assumption that pixels in an image are linearly mixed, including the targetpixel, so that each pixel rrr can be constructed as

rrr = α0ttt+

n∑i=1

αiuiuiui +nnn (1)

where ttt is the target signature, uiuiui are background signatures, αi are theirrespective abundance values, and nnn is a noise term. Both the target and thebackground signatures in this model are endmembers of the image. In otherwords, they are the pure, unmixed signals that, when mixed, result in the sig-

3

nals retrieved from the pixels of the image. Finding these endmembers is par-ticularly important in algorithms such as orthogonal subspace projection andnon-negative matrix factorization that utilize a linear mixing model in whicheach signal is assumed to be a linear combination of endmember signatures.

From there we obtain the orthogonal projection operator, P⊥, which wouldproject pixels onto the subspace orthogonal to the subspace spanned by thesignatures uiuiui. The algorithm to obtain this operator is explained in Algorithm2.

By applying this operator onto each pixel we obtain:

P⊥rrr = α0P⊥ttt+

n∑i=1

αiP⊥uiuiui + P⊥nnn (2)

= α0P⊥ttt+ P⊥nnn (3)

Since P⊥ projects every uiuiui to a subspace orthogonal to all of the uiuiui, thoseterms are eliminated to leave only the target signature and the noise term.

Algorithm 2 Orthogonal Subspace Projection Algorithm

I is the image data with each pixel having p spectral components, Iij is apixel.1. Using some method of endmember selection or a dictionary, select k end-members with p spectral components.2. If neither are provided, select k pixels from hyperspectral image as end-members.3. Form a pxk matrix U with the endmembers as columns.4. If U is not full column rank, column reduce the matrix U until it is fullcolumn rank5. Find orthogonal subspace operator P⊥ = I − U(UTU)−1UT

6. Apply P⊥ to all pixels Iij , obtaining projected hyperspectral image dataJ , with Jij = P⊥Iij

From there, there are two options for classification. The first is to applyan adapted matched filter to maximize the signal to noise ratio of the resultingprojected filter. The resulting map would give abundance values for the plumeat each pixel, while minimizing the noise in the image. The second method isto cluster the orthogonally projected data. In theory, if one were to successfullyremove the non-target signatures from the image, then only the target and somenoise should be able to appear clearly in the image. An example of the effectof orthogonal subspace projection can be seen in Figure 4. In the unprojectedimage, the plume is classified together with the surrounding ground pixels, andis itself difficult to see. However, the plume appears in its own cluster clearlyin the projected image, without any part of the background being visible.

4

Figure 4: The first frame of fuzzy clustering on r134a6-Victory. One is clusteredwithout using OSP (above) and one is clustered using OSP (below).

However, some problems arise when applying orthogonal subspace projec-tion. Because of the way the projection operator is constructed, the number ofendmembers we use for the projection matrix is limited by the dimension of thespace that each pixel belongs to, i.e. number of spectral bands per pixel. In thecase of hyperspectral images, this would not be a significant problem, as thereare typically many spectral bands. However, the more endmembers are used,the more noisy the resulting data becomes. Consequently, applying clusteringmethods to the results and classifying the gas plume becomes more difficultwith additional noise. In addition, every endmember used in to construct theprojection matrix results in a loss of information from the original image. Inparticular, certain endmembers may be similar enough to the target signatureso that the target signature is weakened or even eliminated from the image whenprojecting that endmember. This effect is shown in Figure 5. Notice that eachincrease in the number of endmembers results in an increase in noise and a de-crease in the amount of the plume that is visible. In order to form a successfulprojection matrix, one must choose the endmembers carefully to avoid blottingout the target signature.

5

Figure 5: the first frame of fuzzy clustering of r134a7-Victory using orthogonalsubspace projection with varying numbers of endmembers

3.1 Endmember Selection

In order to eliminate interfering background signatures without also eliminatingthe target signature, selecting endmembers is a vital part of orthogonal subspaceprojection. Ideally, one would have endmembers that represent the exact back-ground signatures to project from an image, resulting in a projection matrixthat removes every element of the background while leaving the target signa-ture unaffected. Though there are dictionaries with background signatures thatmay be readily available, there is still the task of selecting the most significantendmembers to adhere to the limits of the projection matrix.

Some endmember selection methods tested were principle component anal-ysis, non-negative matrix factorization, and automatic target generation pro-cess. The eigenvectors of principle component analysis could themselves be in-terpreted as endmembers of the image, as they correspond to a certain presencein an image. While non-negative factorization may also be used for unmixing,it produces both abundance values, which represent the unmixed image, and es-timations for endmembers that correspond those abundance values, which maybe used in the construction of the projection matrix.

Automatic target generating process is an algorithm for endmember se-lection that is predicated on the idea of choosing a distinct set of endmembers.For the purposes of orthogonal subspace projection, this translates to retrievingas much of the background information as possible to project from the image.The algorithm is described more in depth in Algorithm 3. The candidate pixels

6

are projected onto the subspace orthogonal to the subspace of already selectedpixels. The pixel with the greatest magnitude amongst the candidate pixels areselected for being most “distinct” from the already selected pixels.

Algorithm 3 Automatic Target Generation Process Algorithm

I is the image data with each pixel having p spectral components, Iij is a pixel.v0 is an initialized endmember vector, also with p spectral components.1. If not provided, randomly select a pixel v0 from I as an initial endmembervector2. Form Ui−1 = [ v0 v1 . . . vi−1 ], the previously found distinct pixels, and xis a pixel in the image.3.Find pixels vi, such that

vi = arg maxx

‖(I − Ui−1(UTi−1Ui−1)−1UTi−1))x‖2

where x is a pixel in the image, i.e. the pixel with the maximum magnitudewhen projected onto the subspace that is orthogonal to Ui−14. Repeat steps 2-3 until desired number of endmembers are obtained

3.2 Results Using Orthogonal Subspace Projection

As expected, orthogonal subspace projection did yield results in which the in-fluence of the background of the image is reduced. In sf6-32-Victory, one of ourmost diffuse data sets, the plume was hardly detected at all, and when it was,it was often classified together with some part of the background. Using or-thogonal subspace projection, the plume, though still faint, is the only distinctfeature of the image, as shown in Figure 6 and 7.

7

Figure 6: fuzzy clustering results for sf6-32-Victory. The background is promi-nent in these clusters, and more importantly, the plume is classified togetherwith these clusters.

Figure 7: fuzzy clustering results for sf6-32-Victory with OSP using endmembersselected from PCA. The plume is visible from the first cluster.

Certain enmember selection methods resulted in consistently better clus-tering results. In most cases, non-negative matrix factorization was the leasteffective endmember selection method, as much of the background was clearlyrepresented in each cluster. Most of the time, automatic target generation pro-cess would produce the best clustering results.

In even the most problematic data sets, where the plume was not evenremotely visible, orthogonal subspace projection allows the plume to becomevisible. In the r134a6-Romeo data set, there is a mountain that blocks the sig-nal of the plume, making it nearly impossible to see or segment from the image.However, through the use of orthogonal subspace projection, the influence ofthe mountain’s signatures is significantly reduced, and the plume is made vis-ible. This is evident in both fuzzy clustering (Figure 8 and 9) and in spectralclustering (Figure 10 and 11)

However, this effect is not always consistent, as certain frames do not

8

Figure 8: fuzzy clustering results for r134a6-Romeo. The background is promi-nent in these clusters, particularly the mountain signature on the right. Theplume is not visible in any frame.

Figure 9: fuzzy clustering results for r134a6-Romeo with OSP using endmembersselected from ATGP. The plume is visible from the first cluster. The mountainand other elements of the background are not classified under any cluster.

produce a part or all of a plume when expected. This may be due to thediffusivity of the plume; when pixels containing the plume are projected, boththe background and a part of the target signature are projected from the image,leaving an even weaker target signature. An example of this can be shown infigure 12 and 13. In this case, the plume signature is clustered into two separateparts.

9

Figure 10: spectral clustering results for r134a6-Romeo. The background isprominent in these clusters, particularly the mountain signature on the right.The plume is not visible in any frame.

Figure 11: spectral clustering results for r134a6-Romeo with OSP using end-members selected from PCA. The plume is circled in the clusters where it clearlyappears.

10

Figure 12: spectral clustering results for aa21-Victory. Notice that the top partof the plume, while not consistently classified as the same as the bottom half ofthe plume, manages to be included in severalframes.

Figure 13: spectral clustering results for aa21-Victory with OSP using endmem-bers selected from PCA. Notice that the top half of the plume does appear inthe clusters, but not with the same definition and frequency as the case withoutOSP.

11

4 Non-Negative Matrix Factorization

Non-Negative Matrix Factorization(NNMF) is a technique that approximatesthe endmembers and also performs unmixing for hyperspectral images. It breaksdown the image matrix into an approximate product of two non-negative matri-ces U and V such that U is the endmember matrix and V is the mixing matrixassociated with the original image matrix.

Algorithm 4 Non-Negative Matrix Factorization

Input: Data set D, termination criterion ε1. Initialize U and V such that D = UV .2. Update U as follows:

U ′ = U · AV

UV TV. (4)

3. Update V as follows:

V ′ = V · ATU ′

V (U ′)TU ′. (5)

4. U = U ′ and V = V ′.5. If ||A− UV T ||2 < ε, then stop. If not, go back to step 2.Output: U and V

We applied Non-Negative Matrix Factorization on an image from aa21-Victory and we obtained the following results in Figure ??. Each frame rep-resents the distribution of one of the ten endmembers. Thus, gas plume doesappear in some of the frames since endmember signature is strengthened orweakened after Non-Negative Matrix Factorization, which outlines the shape ofthe gas plume.

12

Figure 14: Result of Non-Negative Matrix Factorization on aa21-Victory

5 L1-Unmixing

L1-Unmixing is a technique that expresses each pixels as a linear combinationof endmembers and show the distributions of these endmembers on image. Forour case, we select pixels of the image as the endmembers. This technique notonly clusters the original data but also plays the role of image enhancement.

We apply L1-Unmixing on an image of the data set aa21-Victory and weobtained the following images in Figure 15. Each of the frame represents thedistribution of one of the ten endmembers. Again, gas plume appears in some ofthe frames due to the strengthening and weakening of the endmember signature.

13

Algorithm 5 L1-Unmixing Algorithm

Input: Data set D, Endmember matrix M , termination criterion ε, param-eters µ and λ1. Select endmembers from M .2. Initialize a0 = b0 = d0 = 0.3. Update the following:

bn+1 = bn + an − dn. (6)

an+1 = (MTM + λI)−1(MT fn − bn+1 + dn). (7)

dn+1 = max{an+1 + bn+1 − µ, 0} (8)

fn+1 = fn + f −Man+1 (9)

where f is the original pixel of D.4. If |fn+1 − fn| < ε, then stop. If not, go to step 3.Output: Data set D with new entries f .

Figure 15: Result of L1-Unmixing on aa21-Victory

14

Euclidean Distance raw data Computes the straight line distance betweenpoints

Cosine Distance raw data Also known as the spectral angle. Computes theangle formed between the point vectors. Invari-ant to scaling

Spectral GradientAngle (SGA)

raw data Computes the angle between the gradients oftwo points.

Spectral InformationDivergence (SID)

PDF Calculates relative entropy, the difference be-tween two components with a weight.

SID-Cosine mixedmeasure

both Weights the SID with the spectral angle, to forma more accurate metric.

Hausdorff Distance raw data Is used on spaces, tries to compute the best dis-tance between two spaces.

Modified Hausdorff raw data Uses a different distance calculation, to betterrepresent spaces.

Table 1: Table of different distance metrics

6 Distance Metrics

Due to the unknown geometry of hyperspectral data, different distance metricswere tested. There are two approaches to computing the distances in hyper-spectral data, utilize the raw data or use the normalized data such that the sumof a data point across the spectral band sums to one. The interpretation of thenormalization is that it forms a probability distribution function, PDF, for apoint, allowing for distribution metrics to be used. Table 1 gives an overview ofthe distance metrics we used.

Euclidean distance (deuc) is known best for being able to capture thestraight line distance from point to point. This is the standard metric used inalmost all distance computations, so we thought best to test it on the data.Due to the noisy nature of the data, with the occasional point behaving badlyconverting to emissivity, the Euclidean distance does not perform well. In mostsituations, it is able to distinguish parts of the gas plume from the surroundingbackground aspects, but the separation is not always clean.

Cosine distance (dcos) metric, also known as spectral angle, computesthe angle between the vectors formed from two points. This metric is invariantto scaling, which we believe might be an issue with the data. The results of thecosine metric have proven to be very reliable, resulting in clean separation ofthe data. An issue that might come up with future data sets is that data mightbe scalar multiples, due to the mixing of signatures, and the calculated distancewould be zero, although the signature would refer to completely different enti-ties.

15

Figure 16: Comparison between the cosine and modified cosine distance metric.

Modified cosine distance (dmcos), the arccosine function can be compu-tationally heavy and combined with the large number of data points, can reallyweigh down calculation time. So to combat this, we utilize approximation tothe cosine metric. Refer to Figure 16 for a comparison of the two metrics. Thefigure shows us that there is no difference between the two metrics, given thatwe only consider the k closest points.

Spectral gradient angle (dSGA) is an alternative to computing the anglebetween the vectors formed from the two points. It works by computing theangle formed from the gradient of the points. The gradient is formed from thedifference of two consecutive components of a pixel as shown below in Equation10. This metric works by not relying on the actual values of the pixel, allowingfor translation invariance.

X ′ = (x2 − x1, x3 − x2, ..., xn − xn−1) (10)

Spectral Information divergence(dSID) is a metric that utilizes thenormalized data, what was earlier called the probability distribution function.It is the difference two Kullback-Leibler information measure, or cross entropy.The metric finds the difference of two corresponding PDF values and weightsthese terms with a ratio of the terms.

Spectral Information Divergence - Spectral Angle mixed mea-sure (dSID SAM ) is a combination of the SID and cosine distance metrics. Itweights the SID by the value of the angle between the two formed vectors. It

16

hopes to incorporate the angle measure into the Spectral Information Diver-gence, making it more accurate.

Spectral Information Divergence - Spectral Angle mixed mea-sure (dSID MSAM ) modification to the cosine measure, allowing for faster com-putation.

Hausdorff distance metric works with neighbourhoods and tries to cap-ture the best distance between sets. An alternative is to compute the averageof the neighbourhood instead of using sets. There is also a modified Hausdorffthat uses a different function to compute the distance between spaces.

deuc(x, y) =

(n∑i=1

(xi − yi)2).5

dcos(x, y) = cos−1(< x, y >

||x|| ||y||

)dmcos(x, y) = 1− < x, y >

||x|| ||y||dSGA(x, y) = dcos(x

′, y′)

dSID(x, y) =

n∑i=1

(xix− yiy

)(log

xix− log

yiy

)

dSID SAM (x, y) =

n∑i=1

(xix− yiy

)(log

xix− log

yiy

)sin

(cos−1

(< x, y >

||x|| ||y||

))

dSID MSAM (x, y) =

n∑i=1

(xix− yiy

)(log

xix− log

yiy

)(< x, y >

||x|| ||y||

)

Hausdorff Distance

d(a,B) = minb∈B||a− b||

d5(A,B) = maxa∈A

d(a,B)

f(d5(A,B), d5(B,A)) = max(d(A,B), d(B,A))

17

Modified Hausdorf Distance

d(a,B) = minb∈B||a− b||

d6(A,B) =1

Na

∑a∈A

d(a,B)

f(d6(A,B), d6(B,A)) = max(d(A,B), d(B,A))

7 Clustering

In this section, we present various unsupervised clustering techniques that aresuccessful in detecting gas plume pixels without having to know the gas plumespectral signature. We used various distance metrics as part of these techniquesto determine the most appropriate one for identifying gas plume. The distancemetric acts as a similarity measure between pixels and thus helps group similarpixels together into one cluster. The clustering techniques used to identify gasplume pixels are K-means, Entropy-Based Clustering, Fuzzy K-means, FuzzyC-means, Spectral Clustering, and Nystrom Extension.

7.1 K-Means Algorithm

The first clustering technique we performed onto the dataset is K-means becauseit is the most basic and most widely used clustering technique. This algorithmselects k centroids and groups the pixels to the nearest centroid as one cluster.Thus, this leads to a result with k clusters in the image. This algorithm issummarized in Algorithm 1.

Being the most basic technique, K-means has the problem of capturingthe gas plume every time it is implemented because during the implementation,initial centroids are randomly selected from among the pixels of the image, sothere exists a likely possibility that none of the centroids selected are not partof the gas plume. Furthermore, with the centroids being randomly selected, theresult may differ from one implementation to another. Another problem is thenumber of clusters initialized. The number of clusters initialized may be moreor less than the actual number of clusters present in the image. Consequently,the algorithm would form more clusters or fuse clusters together, thus creatingan inaccurate result. Nevertheless, we performed K-means on various distancemetrics to examine the results and the problems we encountered from themmotivated us to use other clustering techniques.

To examine the result, we reshape the membership matrix into the appro-priate matrix size and display it as an image. In the image, pixels are clusteredtogether by being displayed as one color. Thus, for example, the mountain maybe displayed as one color, whereas the foreground may be displayed as a differentcolor.

18

Algorithm 6 K-means Algorithm

Input: Data set D, Number of clusters k, termination criterion ε1. Randomly pick k data points and initialize them as centroids Vi.2. Calculate the distance between each pixel to each centroid.3. Assign each pixel to the closet centroid.4. Compute new centroids by calculating the mean of the pixels in eachcluster.5. If maxij = |V ′i − Vi| < ε, then stop. If not, go back to step 2.Output: Membership matrix U (n× 1).

Figure 17: Results of K-Means on a7-Victory. Top left: Cosine. Top right:Spectral Gradiant Angle. Bottom left: Spectral Information Divergence. Bot-tom right: SID-Cosine.

In Figure 17 are the image results of k-means. We applied four differentdistance metrics: cosine, Spectral Gradient Angle, Spectral Information Diver-gence, and SID-Cosine. The distance metrics have produced different results.Althought the plume was captured clearly by the four distance metrics, cosineand Spectral Gradient Angle produced more detailed results. However, in Spec-tral Gradient Angle, the gas plume is being clustered with the mountain. Thismay be attributed to the noise of the data. Consequently, cosine has producedthe best result for k-means.

7.2 Entropy-Based Clustering

Because of the problems present in K-means, we implemented entropy-basedclustering. The algorithm is explained in Algorithm 2. This unsupervised clus-tering technique automatically determines the number of clusters based on eachpixel’s entropy value, and the result does not differ from one implementation toanother.

The entropy value of the pixel measures the total similarity between itand all the other pixels of the image. Before computing the entropy value, we

19

normalize values of each dimesion, or band, in the hyperspectral image to val-ues within the interval [0, 1]. Normalization allows a more accurate result whencomputing the entropy values. The entropy value for each pixel is computed asfollows:

Ei = −j 6=i∑j∈D

((Sij log2Sij) + (1− Sij)log2(1− Sij)), (11)

where Sij is the similarity value between pixels xi and xj such that i 6= j. Thesimilarity value is calculated as follows:

Sij = e−α·d(xi,xj) (12)

where α is a numerical constant and d(xi, xj) is the distance between pixels xiand xj . We determine α by computing the following:

α = − ln(β)

d(13)

where β is the similarity threshold and d is the mean pairwise distance betweenpixels. It should be noted that both Sij and β are values of the interval [0, 1].

According to Yao, the data point that has the minimum entropy value isselected as a centroid. However, because the lower the entropy value, the moresimilar the pixel is to all the other pixels, this centroid will most likely be simi-lar to both gas plume pixels and non-gas plume pixels, so consequently, the gasplume pixels and the non-gas plume pixels will be associated into one cluster.Therefore, in order to capture the gas plume pixels altogether into one cluster,it is preferable that we select the pixel with the maximum entropy value as thecentroid since it is the most dissimilar pixel to all the others, thus forming abetter clustering result.

Algorithm 7 Entropy-Based Clustering

Input: Data set D, similarity threshold β1. Place all of the xi ∈ D into the hyperspace T .2. Compute α according to equation 3.3. For each pixel, calculate the entropy value according to equation 1.4. Determine maximum Ei and select xi,max as a centroid.5. If the similarity values between xi,max and the data points are greater thanβ, put them in a cluster and remove them from T .6. If T is empty, then stop. If not, go to step 4.Output: Membership matrix U(n× 1).

To examine the image result, we do the exact same method as discussedin the previous section.

We perform entropy-based clustering on a7-Victory using cosine and Spec-tral Gradient Angle and we produced results shown in Figure 18. For cosine,

20

Figure 18: Results of Entropy-Based Clustering on a7-Victory. Left: Cosine.Right: Spectral Gradient Angle

Figure 19: Results of Entropy-Based Clustering on aa12-Victory. Left: Cosine.Right: Spectral Gradient Angle

the gas plume is not captured in its entirety perhaps because of the attributingnoise. For Spectral Gradient Angle, the gas plume is captured in better detail.

However, we perform the technique again on aa12-Victory to verify if Spec-tral Gradient Angle is better than cosine. The results are shown in Figure 19.For Spectral Gradient Angle, the plume is not captured at all, whereas for co-sine, most of the plume is captured but not in its entirety. Again, noise mayhave interfered with the results of entropy-based clustering. Thus, we concludethat we need to investigate entropy-based clustering further in order to improvefuture results.

7.3 Fuzzy K-means/C-means Clustering

We perform fuzzy clustering techniques, specifically fuzzy k-means and fuzzy c-means, onto our datasets because they might be able to capture gas plume pixelsthat may not be captured by the hard clustering techniques presented before.Fuzzy clustering associates a pixel to at least one cluster by a probability value.The higher the probability value associated with the cluster, the higher theassociation between the pixel and the cluster. Thus, a pixel that has morebackground signal than the gas plume signal may appear as part of the gasplume cluster in the fuzzy clustering result because of its slight association toit.

The output of fuzzy clustering is a k× n matrix U where k is the numberof clusters and n is the number of pixels in dataset D. The membership matrixsatisfies the following properties:

1. uij ∈ [0, 1],

21

2.

k∑i=1

uij = 1,

3.

n∑j=1

uij > 0,

where uij is an entry of U such that 1 ≤ i ≤ k and 1 ≤ j ≤ n.In regards to fuzzy k-means and fuzzy c-means, although one may find the

algorithms and their results similar, they actually do differ, especially how themembership value uij is computed. For fuzzy k-means, the membership valueis computed as follows:

uij =[d(xj , Vi)]

− 2q−1

k∑l=1

[d(xj , Vi)]− 2

q−1

, (14)

whereas for fuzzy c-means, the membership value is expressed as follows:

uij =

[k∑l=1

(d(xj , Vi)

d(xj , Vl)

) 2q−1

]−1, (15)

where q is the fuzziness parameter such that q > 1. The following algorithmsummarizes how fuzzy k-means and fuzzy c-means are implemented.

Algorithm 8 Fuzzy K-Means/C-Means Clustering

Input: Data set D, Number of clusters k, fuzziness parameter q, terminationcriterion ε1. Randomly pick k data points and initialize them as centroids.2. Compute the membership matrix according to Equation 4 for fuzzy k-means or Equation 5 for fuzzy c-means.3. Compute new centroids as follows:

V ′i =

n∑j=1

uqijxj

n∑j=1

uqij

. (16)

4. Compute membership matrix (u′ij) again with new centroids.5. If maxij |uij − u′ij | < ε, then stop. If not, go back to step 3.Output: Membership matrix U(k × n)

To examine the result, we take the row vector of the membership matrixand reshape it into the appropriate matrix size. Afterward, we display it as an

22

image. Unlike the image results created from hard clustering, we see the imageresult to be most similar to the original image. However, we see from the result,for our case, colors that vary from blue to green to red. Because the row vectorrepresents one of the clusters created by fuzzy clustering, pixels that are coloredblue are indicated to be less associated with the cluster selected, whereas pix-els that are colored red are indicated to be more associated with it. Thus, forexample, if one of the clusters were to be represented as the gas plume pixels,displaying its respective row vector as an image will show the gas plume pixelsas green to red. Nevertheless, if we display a cluster not representative of thegas plume pixels as an image, we may be able to see the gas plume by its slightlydifferent shade of blue compared to the surrounding pixels. For our results,we were able to pre-initialize one of the centroids to be the pixel that has thehighest probability of containing the gas plume. The pixel was found throughAMSD. Hence, for the following results, we present images whose centroids werepre-initialized. Furthermore, we need to normalize the values of the images tovalues of the interval [0, 1], especially for Spectral Information Divergence andSID-Cosine.

In Figure 20, we performed fuzzy k-means on a7-Victory applying thefour different distance metrics: cosine, Spectral Gradient Angle, Spectral In-formation Divergence, and SID-Cosine. From the images, we see that SpectralInformation Divergence and SID-Cosine produced the most detailed cluster ofthe gas plume. Because the results between the two distance metrics are ex-tremely similar, we cannot conclude the best of the metric between these twofor this dataset.

In Figure 21, we performed fuzzy c-means on the same data set with thesame distance metrics. We can see that Spectral Gradient Angle is the worstmetric to use for fuzzy c-means. However, for the other three metrics, we seea more detailed cluster of the gas plume compared to fuzzy k-means. Further-more, we see that the results for the three distane metrics are extremely similar.Therefore, we find that fuzzy c-means is a better clustering algorithm than fuzzyk-means.

23

Figure 20: Results of Fuzzy K-Means on a7-Victory. Top left: Cosine. Topright: Spectral Gradiant Angle. Bottom left: Spectral Information Divergence.Bottom right: SID-Cosine.

Figure 21: Results of Fuzzy C-Means on a7-Victory. Top left: Cosine. Topright: Spectral Gradiant Angle. Bottom left: Spectral Information Divergence.Bottom right: SID-Cosine.

7.4 Spectral Clustering

Spectral Clustering[25] is a soft clustering method that tries to separate thedata with the minimum number of cuts. It works by utilizing the eigenvectorsand eigenvalues of the graph Laplacian. The algorithm is given below.

The results of spectral clustering are the eigenvalues and correspondingeigenvectors. Each eigenvalue determines the importance of the correspondingeigenvector, starting from 1 and descending to 0. The each eigenvector displaysthe separation that is formed from performing the minimum number of cutsneeded to split the data. In the ideal case, the data is binary, 1 and -1, each

24

forming a clustering.

Algorithm 9 Spectral Clustering

Input I is image data with n spectral components, Iij is a pixel in the image,Iij = (I(ij,1), I(ij,2), ..., I(ij,n))Form D, the matrix of distances between each pixelForm S, the matrix of similarities between each pixelCalculate d, the matrix of row sums. dii =

∑nj=1 Sij

Form N , the normalize similarity matrix, d−0.5Sd−0.5

Compute the largest eigenvalues and corresponding eigenvectors of matrix N

There are many variations of the spectral clustering, all of which tweak orcan adjust the results. Our method required the usage of only computing the knearest neighbors of each point, because the entire similarity matrix could notbe computed. Aside from that, we also tested the self tuning σ, allowing fora different sigma for each pixel, allowing for a better representation of a pixelneighborhood.

Figures 22 to 25 show the results of the eigenvectors that corresponds to thegas plume, after 10 iterations of manifold denoising. What we can gather fromthese results is that manifold denoising is an excellent technique for enhancingthe graph Laplacian to better allow separation of the gas plume. Figure 25 isdifferent in that the first 6 eigenvectors do not correspond to the gas plume, asthe gas plume eigenvector was not properly segmented.

Figure 26 show the first 16 of the eigenvectors of the graph Laplacian forthe aa21 data set. What is to be noticed here is that there aren’t any connectionsbetween the top and bottom half of the gas plume. This is a problem within thedataset cased by the sharp difference of temperature between the mountain andforeground. After many applications of k-means, we are unable to connect thetop and bottom portions of the gas plume. Figure 27 show all of the eigenvectorsof the graph Laplacian after 10 iterations of manifold denoising. It appears toconnect more of the top and bottom portions, allowing for the resulting k-meansshown in Figure 28. We can conclude that manifold denoising allows for a betterrepresentation of the gas plume, even with a very strong signal disrupting theimage.

7.5 Nystrom Extension

The Nystrom Method [3] is a fast approximation to the eigenvectors and eigen-values of the graph Laplacian. It works by taking a sample of the data toapproximate the diagonalization of the graph Laplacian. The algorithm is givenin Equation 10.

The fast approximation of Nystrom Extension allows for computationsof huge similarity matrices that would otherwise be impossible to compute.An example of this is to put multiple frames and compute the approximationof the eigenvectors and eigenvalues of the graph Laplacian. This allows for

25

Figure 22: Results of Spectral Clustering with Manifold Denoising on aa12

Figure 23: Results of Spectral Clustering with Manifold Denoising on aa21

26

Figure 24: Results of Spectral Clustering with Manifold Denoising on r134a7

Figure 25: Results of Spectral Clustering with Manifold Denoising on sf6-32

27

Figure 26: Spectral Clustering results of aa21, all eigenvectors

Figure 27: Spectral Clustering results of aa21, all eigenvectors after 10 iterationsof manifold denoising

Figure 28: K-mean on Eigenvectors of the graph Laplacian of aa21 after denois-ing

28

Algorithm 10 Nystrom Extension

Input: Image data I, kRandomly select k data points to form A, rest of data forms BCompute distances amongst data in A, DA

Compute distances between data in A and B, DB

Approximate the distances amongst data in B, DC , with DC = D′BD−1A DB

Compute the row sum of the matrix d =

[DA DB

D′B DC

], where di is the ith

element of dNormalize Each element of DA and DB , DA and DB , where DA(i, j) = DA(i,j)

didj

Q = DA +DA−.5 ∗DB ∗DB

′ ∗DA−.5

Find the singular value decomposition of Q, Q = USV ′

Compute V =

[DA

DB′

]DA−.5

UL−.5

Compute the eigenvector approximation Eig = Vi

V1i(1−Lii).5

the circumvention of the computation of the eigenvalues and eigenvectors of a300000×300000 matrix. The results can be seen in figure 29. The results showsus how the gas plume is clearly grouped together across multiple frames.

Figure 29: Results of Nystrom Extension applied across multiple frames

8 Summary

In this paper, we have shown that endmember selection methods were sucessfulin unmixing the game plume detection. Furthermore, we were able to reduce thebackground signatures using Orthogonal Subspace Projection. We also testedmanifold denoising, which allows better clustering results. In regards to cluster-ing, we have shown that k-means and spectral clustering to be robust, but we

29

encountered some problems with k-means, which motivated us to attempt fuzzyclustering and entropy-based clustering. Entropy-based clustering improves theresults of k-means by forming more natural clusters without having to initial-ize the number of clusters. Fuzzy clustering, on the other hand, requires thenumber of clusters to be initialized but it was able to detect more diffuse gasplume pixels due to its probabalistic property. In addition, we examined otherdistance metrics tailored for hyperspectral imaging. Although they have shownpromising results, we need to further investigate them to decide the best dis-tance metric. However, thus far, cosine still shows to be the best metric fordetecting gas plume pixels.

9 Problems and Future Work

Despite some of our successful results, we encounter various problems in ourhyperspectral images. Noise, a typical problem in image processing, interfereswith the signals of the gas plume pixels, thus making our clustering result in-accurate. Furthermore, background can also interfere with the detection. Forexample, for the dataset from the Romeo camera, detection of the gas plumepixels is extremely challenging since the mountain in the background emits astrong signal that overwhelms the gas plume signal. In aa21-Victory, the atmo-sphere also poses a problem since it divides the gas plume cluster into two.

In order to solve these problems, we need to investigate other denoisingtechniques and background subtraction techniques to further improve our re-sults.

References

[1] Muna Al-Razgan and Carlotta Domeniconi. Clustering ensembles withactive constraints, 2009.

[2] Jon Atli Benediktsson, Xavier Ceamanos Garcia, Bjorn Waske, JocelynChanussot, Johannes R. Sveinsson, and Mathieu Fauvel. Ensemble methodsfor classification of hyperspectral data. Geoscience and Remote SensingSymposium, 2008. IGARSS 2008. IEEE International, pages I–62 – I–65,2008.

[3] Andrea L. Bertozzi and Arjuna Flenner. Diffuse interface models on graphsfor classification of high dimensional data. Multiscale Modeling and Simu-lation, 10(3):1090–1118, 2012.

[4] Michael Boghese, Alex Honda, Samuel Lim, Daniel Waltrip, and MonicaYoo. Video tracking of airborne gases. Technical report, UCLA Departmentof Mathematics, 2010.

30

[5] J. B. Broadwater, D. Limsui, and A. K. Carr. A primer for chemicalplume detection using lwir sensors. Technical report, National SecurityTechnology Department, 2011.

[6] Antoni Buades, Bartomeu Coll, and Jean-Michael Morel. A non-local al-gorithm for image denoising. IEEE, 2:60–65, 2005. Proceedings of the2005 IEEE Computer Society Conference on Computer Vision and PatternRecognition.

[7] Nidal Zeidat Christoph F. Eick and Zhenghong Zhao. Supervised clusteringalgorithms and benefits. Technical report, Department of Computer ScienceUniversity of HoustonHouston.TX, 77204.USA, 2005.

[8] Julie Delon. Midway image equalization. Journal of Mathematical Imagingand Vision, 21:119–134, 2004.

[9] Carlotta Domeniconi, Dimitrios Gunopulos, Sheng Ma, Dimitris Pa-padopoulos, and Bojun Yan. Locally adaptive metrics for clustering highdimensional data. Technical report, 2006.

[10] Guojun Gan, Chaoqun Ma, and Jianhong Wu. Data Clustering: Theory,Algorithms, and Applications. SIAM, 2007.

[11] Matthias Hein and Markus Maier. Manifold denoising. In Advances inNeural Information Processing Systems (NIPS) 19. MIT Press. 5, 2006.

[12] Sanghoon Lee and Melba M. Crawford. Unsupervised multistage imageclassification using hierarchical clustering with a bayesian similarity mea-sure. IEEE Transactions on Image Processing, 14(3):312–320, 2005.

[13] Dimitris Manolakis. Standoff hyperspectral chemical agent detection: Phe-nomenology, sensors, data, and algorithms. 2010.

[14] Michael Moller, Ernie Esser, Stanley Osher, Guillermo Sapiro, and JackXin. A convex model for matrix factorization and dimensionality reduc-tion on physical space and its application to blind hyperspectral unmixing.IEEE Transactions on Image Processing, 21:3239–3252, 2012.

[15] Jean-Philippe Morin, Christian Desrosiers, and Luc Duong. Image segmen-tation using random-walks on the histogram, 2012.

[16] Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. On spectral clustering:Analysis and an algorithm. Advances in Neural Information ProcessingSystems, 14, 2002.

[17] Oleg Okun and Giorgio Valentini. Supervised and Unsupervised EnsembleMethods and their Applications. Springer-Verlag, 2008.

[18] Lior Rokach. Pattern Classification using Ensemble Methods. World Sci-entific, 2010.

31

[19] Leszek Rutkowski. Data clustering methods. Computational Intelligence:Methods and Techniques, pages 349–369, 2008.

[20] Jonathon Shlens. A tutorial on principal component analysis. Technicalreport, Center for Neural Science, New York UniversityNew York City, NY 10003-6603 andSystems Neurobiology Laboratory, Salk Institute for Biological StudiesLa Jolla, CA 92037, 2009.

[21] SPIE. L1 Unmixing and Its Application to Hyperspectral Image Enchance-ment, volume 7334. SPIE, 2009.

[22] Yuliya Tarabalka, Jon Atli Benediktsson, and Jocelyn Chanussot. Spectral-spatial classification of hyperspectral imagery based on partitional cluster-ing techniques. IEEE Transactions on Geoscience and Remote Sensing,47(8):2973–2987, 2009.

[23] Silvia Valero, Philippe Salembier, Jocelyn Chanussot, and Charles M.Cuadras. Improved binary partition tree construction for hyperspectralimages: Application to object detection. IEEE international, pages 2515–2518, 2011. conference publication from Geoscience and Remote SensingSymposium (IGARSS).

[24] Veronica Vilaplana, Ferran Marques, and Philippe Salembier. Binary par-tition trees for object detection. Image Processing, IEEE Transactions on,17(11):2201–2216, 2008.

[25] Ulrike von Luxburg. A tutorial on spectral clustering. Statistics and Com-puting, 17:1–32, 2007.

[26] Li Xiaohui and Zhao Chunhui. Based on the clustering of the backgroundfor hyperspectral imaging anomaly detection. 2011 International Confer-ence on Electronics, Communications, and Control, pages 1345 – 1348,2011.

[27] Rui Xu and II Donald C. Wunsch. Clustering. John Wiley Sons, Inc., 2009.

[28] Ping Lin Inge SkjEllaaen Xue-Cheng Tai, Oddvar Christiansen. Imagesegmentation using some piecewise constant level set methods with mbotype of projection. International Journal of Computer Vision, 73(1):61–76, 2007.

[29] Lihi Zelnik-manor and Pietro Perona. Self-tuning spectral clustering. InAdvances in Neural Information Processing Systems 17, pages 1601–1608.MIT Press, 2004.

32