automated underground pipe condition assessment

8/13/2019 Automated underground pipe condition assessment

1/14

AUTOMATED

UNDERGROUND PIPECONDITION ASSESSMENT

BY IMAGE ANALYSIS OF

THE STATE-OF-THE-ART

SEWER SCANNER AND

EVALUATION

TECHNOLOGY SURVEYS

By:

Sunil K. Sinha

Doctoral Candidate, Civil Engineering

University of Waterloo, Ontario, Canada

Paul W. Fieguth

Assistant Professor, Systems Design


Maria Anna Polak

Associate Professor, Civil Engineering


Robert A. McKim

Associate Professor, Civil Engineering

Louisiana Tech. University, Ruston, U.S.A.

Presented at:No Dig 1999 Conference

North American Society for Trenchless TechnologyOrlanda, Florida May 23-26, 1999


2/14

2

AUTOMATED UNDERGROUND PIPE CONDITION ASSESSMENT BY

IMAGE ANALYSIS OF THE STATE-OF-THE-ART SEWER SCANNER

AND EVALUATION TECHNOLOGY SURVEYS

ABSTRACT

Closed Circuit Television (CCTV)surveys are used widely in North America to

assess the structural integrity of sewagepipes. The video images are examined

visually, and classified into grades

according to degrees of damage. The humaneye is extremely effective at recognition and

classification, but it is not suitable for

assessing pipe defects in thousands of milesof pipeline images due to fatigue and cost.

In addition, manual inspection for surfacedefects in the pipeline has a number of

drawbacks, including subjectivity, varyingstandards, and consumption of time. These

concerns have motivated the Centre for

Advancement of Trenchless Technology(CATT) to conduct a research project for the

development of an automated underground

sewer pipe inspection system, based on thestate-of-the-art Sewer Scanner and

Evaluation Technology (SSET) survey for

major cities in North America.

The paper presents a system for the

application of computer vision techniques to

the automatic assessment of the structuralcondition of sewers from SSET scanned

images. Automatic recognition of various

pipe defects is of considerable interest sinceit solves problems of fatigue, subjectivity,

and ambiguity, leading to economic benefits.

The proposed system can provide accuracy,

efficiency, and economy of sewer pipeexamination. The main efforts of the paper

are placed on investigating algorithms and

techniques for image processing, pipejoint/lateral discrimination and crack

detection. For development of an automated

pipe inspection system the issues exploredare: various signals and image processing

concepts, nonlinear filtering, feature

extraction, pattern recognition, and

artificial intelligence. In this paper, anattempt has also been made to develop the

framework for cost model based on detected

defect population. The proposed systemcould overcome many of the limitations of

the current CCTV surveys, and can provide

a more accurate assessment of underground

sewage pipe conditions.

1. Introduction

The National Science Foundation (NSF)has estimated the total U.S. investment incivil infrastructure systems at $US 20trillion. The investments in sewer pipelinecollection systems represent a majorcomponent of this overall investment.Recent studies have shown that, to upgradeor restore municipal infrastructure in Canadato an acceptable level the cost will be inexcess of $Can 45 billion. Municipalinfrastructure systems are eroding due to

aging, excessive demand, misuse,mismanagement, and neglect, as shown inFigure 1. Due to its lack of visibility,rehabilitation of underground infrastructureis frequently neglected until a catastrophicfailure occurs, resulting in difficult andcostly rehabilitation.

While CCTV technology has advanced,the basic process and results have remainedunchanged [1]. The technician has to

identify pipe defects in a TV monitorlocated in a control vehicle at the job site.The potential weak link in this process is thejudgement that the field technician mustexercise when identifying and classifyingpipe defects in the field. Many factors, suchas experience, mental awareness, equipmentcapability, field distractions andinterruptions, impact the judgment. An


3/14

3

Figure 1: Underground pipe image showing a largehole, scanned by PSET (Core Corp.) in the city ofAlbuquerque.

automatic pipe inspection system isrequired, based on CCTV surveys, whichcan extract and assess conditions to ensureaccuracy, efficiency, and economy of pipe

examination. With recent developments inthe hardware and software for computervision technology, an automated sewer-pipeinspection system is possible andachievable.

In this study, a computer vision systemis established for the automatic assessmentof the structural condition of undergroundsewer pipes. The primary objective of thisresearch is to develop an automated

underground pipe inspection system, basedon the state-of-the-art PSET survey for 10major cities in North America. The maineffort is concentrated on the investigation ofimage processing algorithms for detection ofdefects, and artificial intelligence techniquesfor classification of these defects. Anattempt has also been made to set out theframework for cost and deterioration modelbased on the detected defect population. Thepaper is organized as follows. Section 2

gives a short review of repair andrehabilitation methods, followed bydescription of recent development in sewerscanning technology in Section 3. Section 4presents image processing and segmentationtechniques for detection of defects. Section5 shows some methods for extraction ofdefect features. Fuzzy neural networkclassification methodology is described in

Figure 2: CCTV video image showing joint andlongitudinal crack.

Section 6. Section 7 presents the proposedsystem evaluation and Section 8 gives shortdescription of the proposed cost anddeterioration model. Finally, conclusion isgiven in the end.

2. Sewer Pipes Inspection

Interior defects generally present thefirst warnings of problems that could occurwithin sewer lines. CCTV surveys areconducted using remotely controlledvehicles carrying a television camerathrough an underground pipe. The dataacquired from this process consist ofvideotape, photographs of specific defects,and a record produced by the technician.The technician attempts to classify eachdefect (i.e., cracks, holes, root intrusion,misalign joints, defective laterals, etc.)according to the various levels ofdeterioration and risk. Thus, diagnosis ofdefects depends on experience, capability,and concentration of the operator, makingthe detection of defect error prone. Thetypical CCTV video image is shown inFigure 2.

3. Sewer Scanner and Evaluation

Technology Survey (SSET)

Sewer scanner and evaluationtechnology (SSET) is an innovativetechnology for obtaining images of theinterior of sewer pipe [2]. SSET isdeveloped by TOA Grout, CORE Corp.,


4/14

4

Figure 3: SSET probe for obtaining an unfoldedimage of sewer pipe surface.

California, and the Tokyo MetropolitanGovernment's Sewer Services (TGS)Company. SSET is a system that offers anew inspection method minimizing some of

the shortcomings of the traditionalinspection equipment that relies on a CCTVinspection. This is accomplished by utilizingscanning and gyroscopic technology. TheSSET probe that obtains an unfolded imageof sewer pipe surface is shown in Figure 3.The major benefit of the SSET system overthe current CCTV technology is that theengineer will have higher quality image datato make critical rehabilitation decisions.Figure 4 shows the typical image obtained

from SSET survey. It can be observed thatthe pipe joint in this image is a dark narrowstraight line across the scanned image,which provide good objects for imageprocessing. In addition to the fact that SSETscanner provides an unfolded image of thesewer pipe, it also has other advantages overCCTV image, like:

1. The image has uniform illuminationirrespective of the pipe background.

2. Geometric localization of defects ismore precise.

3. Salient features can be easily extracted.4. Analysis of laterals and joints is easier

and precise due to the unfolded qualityof image that gives a head-on view ofthe defects as compared to the obliqueview obtained from CCTV.

Figure 4: Pipe image showing a joint and multiplecracks, most severely above the joint

4. Image Processing and

Segmentation

Digital image processing hasexperienced explosive growth over the pasttwo-decade [3]. The term digital imageprocessing generally refers to processing ofa two-dimensional picture by a digitalcomputer. In a broader context, it impliesdigital processing of any two-dimensionaldata. Figure 5 shows the fundamental stepsin digital image processing and patternrecognition.

After obtaining digital image, the nextstep deals with preprocessing that image.The key function of preprocessing is toimprove the image in ways that increase thechances for success of the other processes.Preprocessing typically deals withtechniques for enhancing contrast, removingnoise, and isolating regions whose featureindicate a likelihood of alphanumericinformation. Image segmentation is the mostimportant stage in any image processingoperation. It partitions an input image intoits constituent parts or objects. The output ofthe segmentation stage is usually raw pixeldata, constituting either the boundary of aregion or all the points in the region itself. Ineither case, converting the data to a formsuitable for computer processing isnecessary.


5/14

5

Figure 5: Fundamental steps in digital imageprocessing and pattern recognition.

4.1 Color Image Processing and

Enhancement

An obstacle to automatic detection and

classification of underground pipe defects isthat pipe images are usually obtained underdifferent pipe conditions and non-uniformdistributed lighting. To effectively recognizedefect patterns on the pipe's inner surface, itis necessary to first convert all sourceimages to gray scale and to a standardizedbackground lighting condition.

Many image-processing problemsrequire transformation of color images into

gray scale images. After analysis ofunderground sewer pipe images, it has beenobserved that taking the average of threeprimary color bands (red, blue, and green)and converting them into gray scale is noteffective for the detection of cracks in sewerpipe. Signal to noise ratio in the variouscolor bands seems to be an obvious functionof underground pipe background color. Insuch a case the task is to develop a model toobtain gray scale by linear weighting of

color bands. In this study, the gray scaleimages are obtained by using Fisher's lineardiscriminant analysis [4].

To extract the defect information withfidelity from the original image, it isnecessary to convert the gray scale image toa standardized background lightingcondition. The intensity matrix from an

Figure 6: Enhanced underground pipe image byinterpolated multiplier method.

image with a pipe distress contains threetypes of variations: (1) non-uniformbackground illumination, a very low-frequency and fairly high-amplitude signal;

(2) pipe distress or non-distress irregularities(stains or dark materials on surface), a high-amplitude and strictly negative signal havinghigh-frequency components on the edges;(3) noise caused by heterogeneous materialsand granularity, a random, high frequency,and a low-to-medium amplitude signal. Thepixel intensity of image I(p), p={x,y} mayhave the following composition:

I(p)=Ib(p)+In(p)+Id(p)where Ib(p) is the background illumination

signal;Id(p)is pipe distress signal; andIn(p)represents the noise. The algorithm thatremoves the background illuminationconsiders the frame divided into rectangularwindows and starts the process bycalculating the average light intensity for allpixels in each of the windows. Once thismatrix is purged of every distress intrusion,it is converted into a matrix of multiplierscapable of pushing all intensity values to acommon base intensity B. This operation

will have the effect of converting thevariable background intensity Ib(p) to theconstant B. Each window multiplier M(p) iscalculated according to:

M(p) = B/I(p)The result from the enhancement of theoriginal image by the interpolated multipliermethod is shown in Figure 6.

INPUT IMAGE

IMAGE PREPROCESSING

IMAGE SEGMENTATION


6/14

6

Figure 7: Diagram showing the different steps fordetection of linear structures.

4.2 Detection of Linear Structures

In the past 20 years, many approacheshave been developed to deal with the

detection of linear features on optic or radarimages [5-6]. Most of them combine twocriteria: a local criterion evaluating theintensity on some small neighborhoodsurrounding a target pixel to discriminatelines from background and a global criterionintroducing some large scale knowledgeabout the structures to be detected.Concerning the local criteria, most of thetechniques used for road detection in visiblerange images are based either on

conventional edge or line detectors [7]. Theyfail in processing all kinds of imagesbecause they often rely on the assumptionthat the noise is white additive andGaussian, this is not verified in scannedimage. These methods, therefore, roughlyspeaking, evaluate difference of averages,implying noisy results and variable false-alarm rate. The Bayesian framework, whichis well adapted for taking some contextualknowledge into account, has been widely

used [8]. Edge detection and line findingtechniques, of course, have been studiedsince the early days of this field and aredescribed in textbooks [9,10,11]. However,in spite of the large amount of previousresearch in this area, and a number ofcomparative studies [12-13], the choice ofalgorithms suitable for detection of defectsin the underground pipe images is not clear.

Figure 8: Thresholded responses of the line detectorsfor multiple cracks in the underground pipe.

To develop suitable algorithm fordetection of cracks in underground sewerpipes it is important to define their statisticalproperties. In the present study, cracks have

been defined to have the followingcharacteristics:

They are usually not in a straight line Their width lies within a particular range Cracks usually are seen to lie below

certain grey-level intensity.

The approach taken in this study fordetection of linear structures is based on thefusion of the results from two line detectors,both taking the statistical properties ofimage into account. Line detector D1 isbased on the ratio edge detector [14], widelyused in coherent imagery. An in-depthstatistical study of its behavior is given inLopes et. al. [15]. The brightness of pixel s

is A s , so the empirical mean i of a given

region i having in pixels is:

= sis

i

i An

1 . The response of the

edge detector between region i and j is

defined as ijr :

=i

j

j

iij 1r

,min

Detector D2 uses the normalized centeredcorrelation between two populations of pixel[16]. The cross-correlation coefficients

ij can be shown to be:

GREY SCALE IMAGE

D1 DETECTOR D2 DETECTOR

FUSION

CLEANING & LINKING OPERATION

SEGMENTED IMAGE


7/14

7

Figure 9: An opening morphological operation withcircular structuring element.

( )( )2ijji

2

jjij

2

ii

ji

2

ij

1cnn

ncnnn1

1

+++

=

*

where in is the pixel number in region i,

j

iijc

= is the empirical contrast between

region i and j, and i is the variationcoefficient. Both responses from D1 and D2are merged to obtain a unique response aswell as associated direction in each pixel.The detection results are post-processed toprovide candidate segments. Figure 7 showsthe diagram for different steps of the

proposed method. Threshold response of theline detectors after fusing and linkingoperations is shown in Figure 8. The crackfeatures detected in the image will be laterused for feature extraction and classification.

4.3 Morphological Image Processing

Mathematical morphology provides anapproach to the processing of digital imagesthat is based on shape. Appropriately used,

mathematical morphological operations tendto simplify image data preserving theiressential shape characteristics andeliminating irrelevancies. Morphologicaloperations are derived from the branch ofmathematical analysis called Minkowskialgebra [17]. Morphological imageprocessing is useful for shape analysis,feature extraction and nonlinear filtering.

Figure 10: Pipe image showing a joint, two laterals,and minor defects.

Basically, morphological image processingis the analysis of the geometricalrelationship between the image and asmaller image called a structuring element.

Here, some background knowledge ofmathematical morphology is provided. First,some basic binary mathematicalmorphological operations are described, asin reference [18]. LetXbe the set of a binaryimage andBis the set of a structure element(or template), which is also a subset of theuniversal spaceE. Then:

Binary Erosion

{ }BbXbxxBX +=

Binary Dilation

{ }BbXxbxBX += ,:

Binary Opening

( ) BBXBX =o

Binary Closing

( ) BBXBX =

From the above it is observed thaterosion has the effect of shrinking the imageX, while dilation expands it. Openingremoves narrow isthmuses, capes andislands, while closing fills gulfs, and smallholes. An opening with a circular structuringelement of radius ris performed (Figure 9)by removing a strip of object with the width

r

InputImage

DilatedImage

ErodedImage

StructuringElementOpening Operations


8/14

8

Figure 11: Analysis of joint by mathematicalmorphology.

r along the object boundary (erosion) andthen adding a strip of object with the samewidth back on to whatever portion remainsof the object (dilation). Small features will

be removed and can not be recovered,whereas large features will shrink and thenexpand to their original size. With differentsizes of structuring elements, performing aseries of opening operations can thenidentify features of various sizes.Mathematically speaking, the sizemeasurement at a point of object area isequal to the radius of the largest inscribeddisk that contains the point and lies entirelywithin the object area. A unique feature of

the generalized size analysis is that it doesnot require a predefined single object,therefore, it is especially suitable forcharacterizing an irregular andinterconnected region.

In the present work, grey scalemorphology is applied to an image (Figure10) for analysis of joints and laterals. Themain objective of this approach is tosegment out the background and other

objects from the image and hence the jointsand laterals can be easily obtained by asubsequent global thresholding technique.Since the joint is straight and horizontal inthe scanned images, the horizontalstructuring element is used for analysis. Forextraction of lateral the square shapedstructuring element is used. Figure 11 showsthe capability of opening operators for

Figure 12: Analysis of lateral by mathematicalmorphology.

analysis of joint, and Figure 12 shows theextraction of laterals. To define defectivejoints and laterals the information obtainedfrom these images will be used.

5. Image Feature Extraction

In a system designed to distinguishobjects of different types, it is important todecide which characteristics of the objectsshould be measured to produce descriptiveparameters. Proper selection of the featuresis important, since only these will be used toidentify the objects. Figure 13 shows thefundamental steps in image featureextraction and classification.

Many features can be used to describe anobject. The most basic of all image featuresis the measures of image amplitude in termsof spectral value, luminance, tristimulusvalue, or other units. The most commonobject measurements made are those thatdescribe shape. Image transforms providethe frequency domain information on thedata. Transform coefficient featureextraction has proved practical in severalapplications in which the transform domainfeatures are used as inputs to a patternclassification system [19]. The texturalfeatures of an object can often be used todiscriminate between the surface finish of asmooth or coarsely textured object [20]. Thegray level histogram of an image oftencontains sufficient information to allow


9/14

9

Figure 13: Fundamental steps in image featureextraction and classification.

analysis of the image content and, inparticular, to discriminate between objects.The second-order histogram features arebased on the definition of the joint

probability distribution of pairs of pixel[21]. Various measures have been suggested[22] to specify the energy spread about thediagonal ofjoint distribution.

Ideally, the features will be able toclearly distinguish between classes ofdefects. Since this is rarely the case, andsince some feature will be more importantthan others, a minimal set of features thatadequately distinguish between the classes

must be found. Reducing the number offeatures will also be computationallyadvantageous. In this study, a set of 25features is extracted that describe thegeometric attributes of the cracks in theunderground pipe images. Various featureslike mean, standard deviation, second ordermoment, co-occurrence matrix, andmorphological features are calculated. Fromthis large set of features five optimalfeatures based on gray level concurrence

matrix have been kept for classification.

The features extracted will be used forclassification by the fuzzy neural network.The features will be classified according tothe nature and severity of the undergroundsewer pipe defect.

6. Fuzzy Neural Network Classifier

Underground pipe defects appear in theform of random shaped cracks. Thesedefects have been classified, but the decisionto put a defect in a particular class can varyfrom person to person or from time to time.For such a complicated decision ruleproblem, the solution is based on the use ofa neural network that can mimic humanreasoning. The benefits of the neuralnetwork is the generalization ability aboutthe untrained samples due to the massivelyparallel interconnections and the ease ofimplementation simply by training withsamples for any complicated rule ormapping problem. The utility of fuzzy sets[23] lies in their ability to model theuncertain or ambiguous data so oftenencountered in real life. Therefore, to enablea system to take care of real life situations ina manner more like humans, the concept offuzzy sets into the neural network has beenincorporated.

6.1 Backpropagation Neural Network

An artificial neural network (ANN)attempts to mimic, in a very simplified way,the human mental neural structure andfunctions [24]. It can be characterized as amassively parallel interconnection of simpleneurons that function as a collective system.The ability of a neural network to processinformation is obtained through a learningprocess, which is the adaptation of linkweights so that the network can produce anappropriate output. In general, the learningprocess of an ANN will reward a correctresponse of the system to input by

increasing the strength of the current matrixof nodal weights. Therefore, the likelihoodof producing similar output when the sameinputs are entered in the near future willincrease. The learning process may besupervised or unsupervised based on theavailability of target output. In supervisedlearning, inputs proceed through the networkand produce an output. The difference

Classified Image

Feature Extraction

Defect

ClassDefect

Severity


10/14

10

between this output and the target outputrepresents an error that is then propagatedback through the network to 'train' it. Inunsupervised learning, the networkautomatically detects important features andorganizes the input data into classes basedon these features.

For supervised pattern recognition, themost commonly used ANN is the feed-forward network trained using the back-propagation algorithm [25] that is adopted inthe present study. The backpropagationalgorithm can be described in threeequations. First, weight connections arechanged in each learning step (k) with

][

)1(

]1[][][

)()( s

kij

s

j

s

pj

s

kijwmxtw

+=

Second, for output nodes it holds that]['][ ()( sjjjj

opj Ifod =

and third, for the remaining nodes it holdsthat

]1[]1[]['][ )( ++= sjkk

spk

sjj

spj wIf

where

][s

jx = actual output of node jin layers; ][sijw = weight of the connection between

node Iat layer (s-1) and node jat layer (s);][s

pj =measure for the actual error of node j;][s

jI = weighted sum of the inputs of node j

in layer s; )(t = time-dependent learningrate;f( ) = transfer function; m = momentum

factor (between 0 and 1); and jd , jo =

desired and actual activity of node j (for

output nodes only).

6.2 Neuro-Fuzzy Algorithm

Human reasoning is somewhat fuzzy innature. The combination of neural networksand fuzzy logic into neuro-fuzzy system canhelp to enhance the performance of thesystem by using special learning algorithms.

In general, there are two kinds ofcombinations between neural networks andfuzzy systems. In the first approach neuralnetwork and fuzzy system workindependently of each other. Thecombination lies in the determination ofcertain parameters of a fuzzy system by aneural network, or a neural network learningalgorithm. This can be done offline, oronline during the use of the fuzzy system.The second kind of combination defines ahomogenous architecture, usually similar tothe structure of a neural network. This canbe done by interpreting a fuzzy system as aspecial kind of neural network, or byimplementing a fuzzy system using neuralnetwork. Besides these models there areapproaches in which a neural network is

used as a preprocessor or as a postprocessorto a fuzzy system. Such combinations do notoptimize a fuzzy system, but only aim toimprove the performance of the combinedsystem. Learning takes place in the neuralnetwork only; the fuzzy remains unchanged[26]. In this case the improvement of neuralnetwork learning is the main intention.

To increase the classification rate, aneuro-fuzzy algorithm is employed that

combines neural networks and the fuzzyconcepts. Neural networks have learningcapability and the fuzzy concepts can absorb

variability in feature values. In this study,fuzzy concept is applied simply inconverting feature values into fuzzified data,which are inputs to the backpropagationneural network algorithm. Sometimes,variation of feature values is large, then it isdifficult to classify features correctly basedon these feature values. To solve this

problem, each defect feature value isconverted into three fuzzy data, thenlearning is performed with these fuzzifieddata using the backpropagation algorithm.Finally, defects are classified using thebackpropagation algorithm.

To convert five normalized features into15 fuzzy data, the MAXand MINvalues are


11/14

11

Figure 14: Neuro-Fuzzy network with trapezoidalmembership function.

determined that are the maximum andminimum feature values for entire data set,respectively. As shown in Figure 14 threemembership functions are generated denoted

by S (small), M (medium), and L(large). Note that these membershipfunctions are specified by Min and Max, asshown in Figure 14. Then three fuzzy data iscomputed for each feature values and usesthese data as the input data to neural

networks. In Figure 14 )( iS x , )( iM x , and

)( iL x are three fuzzy data of an input

feature value )( ix , corresponding to

linguistic variables of S, M, and L,

respectively. The trapezoidal membershipfunction, as shown in Figure 14, locates atthe average value of features of the sameimage, and has a maximum value of 1 overthe limited range that is specified by thestandard deviation of the feature value. Togenerate a linguistic variable the averageand standard deviation of the feature valuesare computed. Then the interval betweenMIN and MAX is uniformly divided intoseveral subintervals, where MIN and MAX

represents the minimum and maximum ofaverage values of the specific feature,respectively. The membership function ofeach image is centered at the average valueof the features of the image. Variation offeature values for the same image is allowedby employing the trapezoidal membershipfunction, i.e. the width at the top of thetrapezoidal membership function is set to

Table 1. Classification rates by conventionalbackpropagation and neurofuzzy algorithm.

i , where i denotes the standard deviationof the ith feature value. Note that for inputdata greater (smaller) than MAX (MIN) weclip the membership value to 1 (0).

6.3 Experimental Verification

To test the proposed approach 256x256pixels color images of sewer pipes are used.Fifty samples are used for training andtwenty samples for testing. All values usedfor training and testing are the normalizedvalues. For performance comparison withthe proposed neuro-fuzzy algorithm, theconventional backpropagation algorithm is

simulated.

In the backpropoagation algorithm toreduce the computational complexity and toavoid local minima problem, the learningrate and momentum factor are variedadaptively. In the input layer, there exist fivenodes for five features, and one neuron inthe output layer for defect classification. Thenumber of nodes in the hidden layer isdetermined experimentally: in this

simulation the number is selected whichgives the best classification results, i.e. fiveneurons are used. In the neuro-fuzzyalgorithm 15 fuzzy data are used in the inputlayer and in the hidden and output layer thesame number of neurons are used as in theconventional backpropagation algorithm.Table 1 shows the classification results bythe backpropagation algorithm and proposed

Pipe Defect Pipe Defect

Recognition Method Recognition

Rate

(%)

Backpropagation Algorithm - 88.51

Neuro-Fuzzy Algorithm

Triangular Membership Function - 91.74

Trapezoidal Membership Function - 93.26

(x)

Feature Value x

x MaxMin

S M L

Output


12/14


13/14

13

sewer pipe conditions. This model will thenbe used to predict the deterioration rate andcost based on the current condition of thepipe, cost of repair, repair method, and theaggregate opinion of the experts in this field.

9. ConclusionA practical use of computer vision for

automatic assessment of the structuralcondition of sewers from SSET scannedimages has been suggested andimplemented. Two main obstacles are thepoor background illumination, and thehighly patterned surface of sewer pipes,which causes problem for detection ofdefects. The defect detection techniquesused in this study, especially the generalizededge detection, are robust for detection ofcracks, but for root extraction, thetechniques need to be improved further. Forjoint and lateral analysis, a technique basedon mathematical morphology has beenproposed and investigated. The technique isquite effective if the original size of the jointand lateral is available or estimatedaccurately.

A fully automated sewer pipe inspectionsystem is envisaged, in which successiveimage frames are analyzed until a pipedefect is identified. The defective framewould be selected, and analyzed using thetechniques discussed in this study. Such ascheme would extract valuable informationfrom SSET surveys. This goal has not yetbeen achieved, and although there is still alot of work to be done, the principles havebeen developed and demonstrated in thispaper. The proposed automated system has

the potential to overcome the limitations ofthe current CCTV inspection, and canprovide a more accurate assessment ofunderground sewer pipe conditions.

10. Acknowledgement

This research is supported by NaturalScience and Engineering Research Council

(NSERC) of Canada, and Liqui-forceServices Ltd., Ontario, Canada. The authorswould like to thank CORE Corp., California,for providing the scanned images needed forthis study, and the municipalities in NorthAmerica for their valuable input.

References

1. Iseley, T., Abraham, D., M., andGokhale, S., (1997). 'Intelligent SewerCondition Evaluation Technologies,'Intl. NO-DIG Conf., p. 254-265.

2. Wirahadikusumah, R., Abraham, D. M.,Iseley, T., and Prasanth, R. K., (1998).'Assessment Technologies for SewerSystem Rehabilitation,' J. of Automationin Const., vol. 7(4), p. 259-270.

3. Heijden, F. V. D. (1994). Image BasedMeasurement Systems, John Wiley andSons, Inc., New York, N. Y.

4. Schalkoff, R. J., (1992). PatternRecognition: Statistical, Structural, andNeural Approaches, John Wiley andSons, Inc., New York, N. Y.

5. Geman, D., and Jedynak, B., (1996). AnActive Testing Model for TrackingRoads in Satellite Images, IEEE Trans.Pattern Anal. Machine Intell. , Vol. 18,p. 1-14.

6. Hellwich, O., Mayer, H., and Winkler,G., (1996). 'Detection of Lines inSynthetic Aperature Radar (SAR)Scenes,' in Proc. Intl. ArchivesPhotogrammetry Remote Sensing, vol.31, Austria.

7. Canny, J., (1986). A ComputationalApproach to Edge Detection, IEEETrans. Pattern Anal. Machine Intell.,Vol. 8, p. 679-698.

8. Samadani, R., and Vesecky, J. F.,(1990). Finding Curvilinear Features inSpeckled Images, IEEE Trans. Geosci.Remote Sensing, Vol. 28, p. 669-673.

9. Castleman, K. R., (1996). Digital ImageProcessing, Prentice-Hall, Inc., NewJersey, N. J.


14/14

14

10.Gonzalez, R. C., and Woods, R. E.,(1993). Digital Image Processing,Addison-Wesley, Inc., New York, N. Y.

11.Pratt, W. K., (1991). Digital ImageProcessing,John Wiley and Sons, Inc.,New York, N. Y.

12.Fram, J. R., and Deutsch, E. S., (1975).On the Quantitative Evaluation of EdgeDetection Schemes and TheirComparison with Human Performance,IEEE Trans. Comput.,C-24. p. 616-628.

13.Fua, P., and Leclerc, Y. G., (1990).Model Driven Edge Detection,Machine Vision Application, Vol. 3, p.45-56.

14.Touzi, R., Lopes, A., and Bousquet, P.,(1988). A Statistical and GeometricalEdge Detector for SAR Images, IEEE

Trans. Geosci. Remote Sens., Vol. 26,p.764-773.

15.Lopes, A., Nezry, E., Touzi, R., andLaur, H., (1993). Structure Detection,and Statistical Adaptive Filtering in SARImages,Int. Journal of Remote Sensing,Vol. 14(9), p. 1735-1758.

16.Tupin, F., Maitre, H., Mangin, J.,Nicolas, J., and Pechersky, E., (1998).Detection of Linear Features in SARImages: Application to Road Network

Extraction, IEEE Trans. Geosci.Remote Sensing, Vol. 36(2), p. 434-452.

17.Serra, J., (1986). Introduction toMathematical Morphology, Comput.Vis. Graphics Image Process., 35, p.283-305.

18.Vogt, R., (1989). Automatic Generationof Morphological Set Recognition

Algorithms, Springer-VerlagPublication, Munich, Germany.

19.Jain, A. K., (1989). Fundamentals of

Digital Image Processing, PrenticeHall, Inc., New Jersey, N. J.

20.Haralick, R. M., Shanmugam, K., andDinstein, I., (1973). Textrual Featuresfor Image Classification, IEEE Trans.Syst., Man, Cybern., Vol. SMC-3, p.610-621.

21.Reed, T. R., and duBuf, J. M. H., (1993).A Review of Recent Texture

Segmentation and feature ExtractionTechniques, Comput. Vis. GraphicsImage Process., 57(3), pp. 359-372.

22.Haralick, R. M., (1979). Statistical andStandard Approaches to Texture, Proc.,IEEE Conf. on Pattern Recognition,67,p. 786-804.

23.Zadeh, L. A. (1965). Fuzzy Sets,Information Control, 8, p. 338-353.

24.Rumelhart, D. E., and McClelland, J.,(1986). Parallel DistributedProcessing, Vol. 1, MIT Press,Cambridge, Massachusetts.

25.Schmidt, W. F., (1993). Initialization,backpropagation, and Generalization ofFeedforward Classifier,IEEE Int. Conf.on Neural Networks, p. 598-604.

26.Bezdek, J. C., and Pal, S. K., (1992).

Fuzzy Models for Pattern Recognition,IEEE Press, New York, N.Y.

27.Ross, S. M. (1996). 'StochasticProcesses', John Wiley & Sons, NewYork, N.Y.

automated underground pipe condition assessment

Documents