A NEURAL NETWORK MODEL FOR ANALYSIS AND OPTIMIZATION OF PROCESSES

J. SAVKOVrC-STEVANOVIC

Derartment of Chemical Enqineerinq, Faculty of Technologyand Metallurgy University of Be l qrade , Karnegijeva 4,

P.O.B. 494 Yugoslavia

ABSTRACT

Artificial neural networks based on a feedforward architecture and trained bvthe backprapagation technique were applied to analysis and improvement of aseparation process. Various neural network topologies have been tested andcompared. The Powell method was used to train the network by minimising thesum of squares of residuals as well as GDR algorithm. The performance of thenetwork have been analysed. The obtained results show th8a~~liqj\)i1i:tv · . .the neural networks structure with and without hidden layer.' These resultsilustrate the feasibility of us i nq a neural network as a data analyser and asan optimisation tool,

KEYWORDS

Decision support; neural network, learning in process engineering, backpropa­gation algorithm, feedforward neural network.

INTRODUCTION

Neural networks are a promising tool for a large variety of data analysisprocedures. A neural net consists of a number of connected r.rocessinq unitsor neurodes, each of which nerforms a nonlinear transformation on its inouts.Often the neurodes are organised into distinct layers, inout, hidden or .\associative, and output layers. When the layers are connected in a feedforwardway (so that signals are passed from input to hidden to output layers withoutfeedback or communication within a layer). the net is called a multilayeredpercep ti on.

Various architectures of neural networks have been s unces ted in literature(Kern, 1990; Wan~ and Wu ·, , 991). . ,

In this paper neural network architectures of three distinct layers (f nnut ,hidden and output) and without hidden layers are investiaated. The layers areconnected by feedforward, wei(1ted connections . The function of the inout layeris to distribute values of input oarameters into the net. .

In the traininq phase a number of example that representative of the problemto be learned are provi ded to the net. By adapti n9 its we ights, the netler.l7"ns - the desired input-output relation. Neural networks tvpically learn toperform tasks by itertive error-minimization. The Powell method and algorithmof the Generalized Delta Rule (GDR) were used to train the network minimizin9the sum of squares of residuals.

As a case study is considered the process of separation of butylacetate ­butylalcohol - water system in interlinked industrial distillation columns.

8411

S412 European Symposium on Computer Aided Process Engineering-2

NEURAL NETWORK MODELS

Although there has been some research on the desi<J1of optimal neural network- NN structure it is still largely an art to determine the number of hiddenlayers and ",umber of units in each hidden layer. Neural network structure isdenoted as I x H x O. whe re I. Hand 0 represent number of input units , hi ddenunits. and output units. respestively.

Input-Output Mapeing with Neural Network. Most neural networks use sigmoidalactivation functf ons which make it possible for the neural network to performa complicated f nput-output mapp lnq trough the back propagation procedure. Inessence. a neural network model is equivalent to a set of algebraic equationsarranged in a hierarchical order to form a input-output mapping. Chainging thestructure of a neural network is nothing more than changing the hierarchicalorder of algebraic equations. Training a neural network is just another way tosay estimating the parameters in the complex inout-output transformation func­tion (Plshwi ck, 1989) formed by activation functions.

It is difficult to study the innut-output transformation function of a neuralnetwork with hidden layer(s). Whitout a hidden layer the neural network outputis a function of a linear combination of the input variables.

TRAINING A FEEDFORWARD NEURAL NETWORK

The backpropagation training algorithm has been used successfully in trainingthe neural networks for wide applications. The backpropagation algorithmadjusts the weights in a feedforward neural network consisting of severallayers. and an output layer. The goal is to teach the network to associatespecific output states. call target states. to each of several input states.Having learned the fundamental relationships between inputs and outputs. theneura1 network can produce the correct output for a new previ ously unseeninput. The back propagation learning algorithm does not need too much comnu­tation time to obtain the correct weights if the training data is small size.In other words if the relationship between inputs and outputs is simple. a setof weights is easy to obtain. ~1ost often for comnlex problems the relation­ship between inputs and outputs cannot be completely represented with smallamount of training data set. Therefore the computation time required by back­propagation is very large.

A popular configuration of neural networks for back propagation is a totallyfeedforward net (Epping and Ni t.te rs , 1990; Venkatasubraumantan. et.al. 1990;Savkovi c-Stevanovi C. 1991.). In the feedforward nets inputs feed up th roughhidden layers to an output layer. Each neuron forms a weighted sum of the in­puts from previous layers to which it is connected. adds a threshold value andproduces a nonlinear function of this sum as its output value (Fig.1.). Theoutput value serves as input to the next layer to which the neuron is connec­ted. and the process is repeated until output values are obtained for theneurons in the output 1ayer. Thus. each neuron performs

p pY.=f(LW..X.-e.) (1)J i lJ J J

where Wij is the weight from neuron i to neuron j. Wij can be a positive ornegati ve real number. and e j is the threshol d of the Jth neuron, P means thepth pattern. The f(x) is a nonlinear function of activation that is oftenchosen to be of a sigmoidal form:

f(x) = (1/2)(1 + tan h(x)) (2)

is used in this analYpis where tanh is the hyperbolic tangent. If dj are thedesired outputs and Yi are the outputs obtained from the output layer for thepth pattern. Neural nets are trained by minimizing the error function

European Symposium on Computer Aided Process Engineering-2

P P 2E = P~1 i~1 (di - Yi)

S413

( 3)

Fig.1. The neural network NN(8x2x2)

Product 1

Hiddenlayer

tnoutlayer

where i indexes the number of neurons in the output layer, and p means thep-th input pattern of the training set is presented on the input layer.

Minimising the sum of squares of errorsis not alw~ys the best way of traininga neural network but for some applica­tions it suffices (Bulsari and Saxen,1991.). Backpropaqation by the genera­lised delta rule, a kind of gradientdescent method is one popular method(Wang and Wu, 1991., Korn, 1990. Jonesand Hoskrins , 1987, Venkatasubraumantanet. a1. 1990.).

The gradient descents was describedmethod by the following eqns(4)-(9).

The commonly used steepest descentsprocedure in minimizinq E ;s to chanqeWij and 8j by ~Wij and ~8j' where

aE~w' . = - -- - 11, J 9W' •

1J(4)

aEt.e· = - -n.J aej

where 11 is learning rate.

After simpl ification llwi j and M,j can be exoressed as

p P~Wij = 11 ~o /i

( 5)

(6 )

~ej = 11 E 6~P J

(7)

where

(8)o~= (d~ -l)l(l-y.)"J J J J 1

if the j-th neuron is in the output 1ayer, and

o~ = y~(1 -l) E oPk w'k (9)J , J k J

if the j-th neuron is the hidden layer and k is overall neurons in the layerabove neuron j.

DESCRIPTION OF THE CASE STUDYIn order to analyze the learning of neural networks for orocess classifier andoptimization the industrial separation plant was investigated.

The process units are shown in Fig.2. The number of input neurodes Ni is equalthe-nuritJerof input parameters N· = 8 as shown in Fig.1. x(l}, x(2) and x(3) arefraction of butylalcohol(1}, butyl-acetate(2) and water(3) in the feed fromwaste streams of oenicillin production. t.P1 and t.PZ are pressure drops in thefirst-102 and second column-103 resnectively. R1 and RZ internal reflux ratiofor the first and the second column, respectively. Re 1S external reflux ratio.

The number of hidden neurodes Nn = 2. The output layer consists of two neurodesNo = 2.

S414 European Symposium on ComputerAided Process Engineering-2

...

?

bL.CClAT<ou. J

Fig.2. Scheme of industrial plant

THE TRAINING ALGORITHMSNetwork training aims at achieving the least of errors , the errors measured asthe difference between the calculated output and the desired .output. In thisinvestigation the Powell method (Powell, 1965; Savkovit-Stevanovit, 1988), wasused to calculate the weights in the neural networks which minimised the SIl1lof squares of errors (SSQ) as well as backpropagation algorithm by the genera­lised delta rule - GDR method (Fig.3).

Step 1.

Step 2.

Step 3.

assign all neuron offsets (thresholds) to smallrandom values: 8j

assign all weights to small random values: wi'.J

repeatfor p=l to TP (TP is total number of trainingpatterns)for j=l to "2 (n2 is the mmber of neurons inthe hidden layer)calculate neuron outputs in the hidden layer: Yiendforfor k=1 to n3 (n3 is the number of neurons in theoutput layer)calculate n~uron outputs in the output layer: Vicaleul ate o~

(Sk) = 8~P- ) +ll8~p)

endforfor j=1 to n2 and k=1 to n3W}k) = W~k-1) + 6W3k)endforfor j=1 to "2ca1cu1ate 0 ~

J

European Symposium on Computer Aided Process Engineering-2

e\P) = e\r- 1) + ~e(p)J J J

endforfor i=1 to n1 and j=1 to n2

(n1 is the number of neurons in theinput 1ayer)

w{~) = w{~-1) + ~w{~)lJ lJ lJ

endforendforuntil W<E;, (E;, is the converoence criterion)

Fi g. 3.Tm e back.pronaga ti on GDR algorithm

S4iS

NN structure

RESULTS

In this investigations two neural network structures are compared: 1) withhidden layer and 2) without hidden layer. The training parameters are sumari­zed in Table 1.

Table 1. Effect of hidden l~er and training method

Training error

8x2x2

8x2

0.108 0.287

0.338 0.406

The initially set to be the same for different neural newtork structures, thatis learninq rate n = 0.5.

The neural net calculate on its own on optimal combination ·of input values. Insuch a case the net has to operate in reverse: Given a desired higher produc­tion level, what are the corresponding input values? This reverse computationtries to arrive at an 8-0 new input vector based on a desired 2-D output vec­tor. This problem has an infinite number of solutions. Therefore, the solutionspace has to be reduced by adding constraints: a) The values of the new inputsshoul d be in the range of those of the training set, b) The di fference betweencurrent and new inputs should be as small as possible, e) Some of the in~ut

parameters should be fixed.

After about 1000 iilterations t the adapti ve weights and hence the new inputshave changed in such a way that the network output has converged to the desi­red output.The neural net advice for ontimization of process separation is summarized inTable 2.

Table 2. The outcome of the run with minimum-error level

Input parameter Current value New value

1. X(1) vol.% 58.43 59.00

2. X(2) vo1. % 31.40 30 .00

3. X( 3) vo1.% 10. 17 11.00

4. t.P1' Pa 0.0785 105 0.0790 105

5. ~P2' Pa 0.196 105 0.070 105

6. R1 0.10 4.75

7. R2 0.25 1

8. Re 0.44 3.75

S416 European Symposium on Computer Aided Process Engineering-2

CONCLUSIONIn the present investigation is evaluated the applicability of neural networksfor the industrial data analysis. The training results obtained by GDR algo­rithm are compared with those obtained by Powell method for minimising the sumof squares of errors. The obtained results are close for different neural net­work architectures. These results show that the applicability the neural net­work structures with and without hidden layer. The optimal operation condi­tions in separation plant were experimental verified.

REFERENCESBulsari A.B. and H. Saxen (1991) A Chemical reactor selectionexpert system implementation in an artifical network. Computer- Oriented Process Engineering (edited by L. Puigjaner and A.Espuna), Elsevier Science Publicher, B.V. Amesterdam pp. 23-28(Proc. of COPE'91, 14-16 Oct.) Barcelona, Spain.Epping W.J.M. and G. Nitters (1990) A neural network for analy­sis and improvement of gas well production, Proc . of SummerComputer Simulation Conf., pp. 329-334, July 16-18, Calgary,Canada.Korn G.A. (1990) Interacti ve simulati on of backpropagation andcreeping - random search learning in neural networks, Simula­tion, October 214-219.Jones, W.P. and J. Hoskins (1987) Backp~ooagation. A generali­zed delta learning rule, Byte, October, 155-162.Powell, M.J.D. (1965) Method for minimizing a sum of squares ofnonlinear functions without calculating derivates, Computer J.,2, 303-307.Savkovic-Stevanovic J., (1988). Software packaqe for processoptimization: Documentation and users manual. Dep. of. Chem.Eng., Faculty of Technology and Metallurgy, Beograd, YugoslaviaSavkovi c-Stevanovi c J., (1991). Neural networks models forprocesses analysis, Proc . of the 5th Sem. and Symp. Informationand Expert Systems in the Process Industries, IES '91, pp(I-29),3-4 Oct., Beograd, Yugoslavia.Venkatasubraumantan, V. Vaidyanathan and Y. Yamanoto (1990),Process fault detection and diagnosis using neural networks-ISteady-State processes, Computers Chern. Engng., .!i, 699-712 .Wang C.J. and C.H.Wu (1991), Parallel simulation of neuralnetworks, Simulation, April, 223-232.