8/6/2019 T3-R-00-0763

1/6

A Recurrent Neural Architecture Applied in Dynamic Programming andGraph Optimization

Ivan Nunes da Silva

Department of Electrical Engineering

University of So Paulo USP/SEL/EESC

So Carlos SP, CP 359, CEP 13566-590, Brazil

Phone: +55 (16) 3373-9367, Fax: +55 (16) 3373-9372, E-mail: insilva@sel.eesc.usp.br

Abstract - This paper presents an architecture of recurrent

artificial neural networks that can be used to solve bipartite

graph optimization problems and dynamic programming

problems. More specifically, a modified Hopfield network is

used and its internal parameters are explicitly computed

using the valid-subspace technique. These parameters

guarantee the convergence of the network to the equilibrium

points, which represent a solution of the consideredoptimization problem. Simulation results are presented to

validate the proposed approach.

Keywords: artificial neural networks, graph optimization,

dynamic programming, Hopfield networks.

I. INTRODUCTION

Artificial neural networks have been applied to several

classes of optimization problems and have shown promise

for solving such problems efficiently. Most of the neural

architectures proposed in literature solve specific types of

optimization problems [DIL, 02][TAK, 99][KAK, 00].Differently of these neural models, the network used here

is able to treat several kinds of problems using unique

network architecture.

The approach described in this paper uses a modified

Hopfield network with equilibrium points representing the

solution of the optimization problems. The internal

parameters of the network have been computed using the

valid-subspace technique [AIY, 90]. This technique allows

to define a subspace, which contains only those solutions

that represent feasible solution to the problem analyzed. It

has also been demonstrated that with appropriately setparameters, the network confines its output to this

subspace, thus ensuring convergence to a valid solution.

Differently of other approaches, the mapping of

optimization problems using the modified Hopfield

network always consists of determining just two energy

functions, which are defined byEconfandE

op. The function

Econf is a confinement term that groups all structural

constraints associated with the problems, and Eop is an

optimization term that leads the network output to the

equilibrium points corresponding to optimal solutions.

More specifically, the approach proposed in this paper has

been based on the model presented in [SIL, 00], whichwas used a modified Hopfield network for solving

combinatorial optimization. We have extended this

architecture in order to solve bipartite graph optimization

problems and dynamic programming problems. The main

advantages of using the modified Hopfield network

proposed in this paper are the following: i) the internal

parameters of the network are explicitly obtained by the

valid-subspace technique of solutions, ii) the valid-subspace technique groups all feasible solutions

associated with the problem, iii) lack of need for

adjustment of weighting constants for initialization, iv) for

all classes of optimization problems, a same methodology

is used to derive the internal parameters of the network.

The organization of the present paper is as follows. In

Section II, the modified Hopfield network is presented,

and the valid-subspace technique used to design the

network parameters is described. In Section III, the

mapping of optimization problems using the modified

Hopfield network is formulated. In Section IV, simulation

results are given to validate the developed approach. InSection V, the key issues raised in the paper are

summarized and conclusion drawn.

II. THE MODIFIED HOPFIELD NETWORK

Hopfield networks are single-layer networks with

feedback connections between nodes. In the standard

case, the nodes are fully connected, i.e., every node is

connected to all others nodes, including itself [HOP, 84].

The node equation for the continuous-time network with

Nneurons is given by:

=

++=N

j

bijijii itvTtutu

1

)(.)(.)(

))(()( tugtv ii =

where: ui(t) is the current state of the i-th neuron, vi(t) is

the output of the i-th neuron, bii is the offset bias of the i-

th neuron, .ui(t) is a passive decay term, Tij is the weightconnecting thej-th neuron to i-th neuron.

In (2), g(ui(t)) is a monotonically increasing thresholdfunction that limits the output of each neuron to ensure

that network output always lies in or within a hypercube.

(1)

(2)

8/6/2019 T3-R-00-0763

2/6

It is shown in [HOP, 84] that ifTis symmetric and =0,the equilibrium points of the network correspond to values

v(t) for which the energy function (3) associated with the

network is minimized:

bivvTv = TT ttttE )()()(2

1)(

Therefore, the mapping of optimization problems using

the Hopfield network consists of determining the weight

matrix T and the bias vector ib to compute equilibrium

points. In this paper, we have used a modified energy

functionEm(t), composed by two energy terms, defined as

follows:

Em(t) =E

conf(t) +E

op(t)

where Econf

(t) is a confinement term that groups the

structural constraints associated with the respective

optimization problem, and E

op

(t) is an optimization termthat conducts the network output to the equilibrium points

corresponding to a cost constraint. Thus, the minimization

ofEm(t) of the modified Hopfield network is conducted in

two stages:

I. Minimization of the Term Econf(t):

confTconfTconfttttE ivvTv .)()(..)(

2

1)( =

where v(t) is the network output, Tconf

is a weight matrix

and iconf

is a bias vector belonging toEconf

. This corresponds

to confine v(t) into a valid subspace generated from thestructural constraints imposed by the problem.

II. Minimization of the Term Eop

(t):

opTopTopttttE ivvTv .)()(..)(

2

1)( =

where Top

is weight matrix and iop

is bias vector belonging

to Eop

. This corresponds to move v(t) towards an optimal

solution (the equilibrium points). Thus, the operation of

the modified Hopfield network consists of three main

steps, as shown in Fig. 1:

Fig. 1. The modified Hopfield network.

Step ((I)): Minimization of Econf

, corresponding to the

projection ofv(t) in the valid subspace defined by:

vTval.v +s

where: Tval

is a projection matrix (Tval

.Tval

= Tval

) and the

vector s is orthogonal to the subspace (T

val

.s = 0). Thisoperation corresponds to an indirect minimization of

Econf

(t), i.e., Tconf

= Tval

and iconf

=s. An analysis of the valid-

subspace technique is presented in [AIY, 90].

Step ((II)): Application of a nonlinear piecewise

activation function constraining v(t) in a hypercube:

=

1if,0

10if,

1>if,1

)(

8/6/2019 T3-R-00-0763

3/6

problems and dynamic programming problems, is as

follows.

The vector p n represents the solution set of anoptimization problem consisted of n nodes (neurons).

Thus, the elements belonging to p have integer elements

defined by:

pi {1,...,n} where i {1..n}

The vector p can be represented by a vector v, composed

of ones and zeros, which represents the output of the

network. In the notation using Kronecher products [GRA,

81], we have:

is a matrix (nxn) defined by:

=

ji

jiij

if0,

=if,1

(k) n is a column vector corresponding to k-thcolumn of .

v(p) is an n.m dimensional vector representing the formof the final network output vector v, which corresponds to

the problem solution denoted by p. The vector v(p) is

defined by:

=

)(

)(

)(

)(2

1

np

p

p

pv

vec(U) is a function which maps the mxn matrix Uto then.m-element vector v. This function is defined by:

v = vec(U) = [U11

U21...U

m1U

12U

22...U

m2 U

1nU

2n...U

mn]

T

V(p) is an nxn dimensional matrix defined by:

=T

n

T

T

p

p

p

)(

)(

)(

)(2

1

pV

where [V(p)]ij

= [(pi)]

j.

PQdenotes the Kronecher product of two matrices. IfP is an nxn matrix, and Q is an mxm matrix, then (PQ) isa (n.m)x(n.m) matrix given by:

=QQQ

QQQ

QQQ

QP

nnnn

n

n

PPP

PP

PPP

P

21

22221

11211

whdenotes the Kronecher product of two vectors. Ifwis an n-element vector and h an m-element vector, then

(wh) is an n.m-element vector given by:

=

h

h

h

hw

.

.

.

2

1

nw

w

w

The properties of the Kronecher products [GRA, 81]

utilized are the following ones:

(wh) = (wh)

(wh)T(xg) = (wTx)(hTg)

(PQ)(wh) = (PwQh)

(PQ)(EF) = (PEQF)

vec(Q.V.PT) = (PQ).vec(V)

on and On are respectively the n-element vector and thenxn matrix of ones, that is:

}{1..,for1][

1][nji

ij

i

=

=

O

o

Rn

is an nxn projection matrix (i.e., R

n

.R

n

= R

n

)defined by:

nnn

nOIR

1=

The sum of the elements of each row of a matrix is

transformed to zero by post-multiplication withRn, while

pre-multiplication by Rn

has the effect of setting the sum

of the elements of each column to zero.

B. Formulation of Graph Optimization Problems

The graph optimization problem considered in this paper

is the matching problem in bipartite graphs. However,

several other types of graph optimization problems, such

as the graph-coloring problem and the max-clique

problem, can be solved by the proposed neural approach.

A graph G is a pair G = (V,E), where Vis a finite set of

nodes or vertices and E has as elements subsets of V of

cardinality two called edges [PAP, 82]. A matchingMof

a graph G = (V,E) is a subset of the edges with the

property that no two edges ofMshare the same node. The

graph G = (V,E) is called bipartite if the set of vertices V

can be partitioned into two sets, Uand W, and each edge

inEhas one vertex in Uand one vertex in W.

(10)

(11)

(12)

(13)

(14)

(15)

(16)

(17)

(18)

(19)

(20)

(21)

(22)

(23)

8/6/2019 T3-R-00-0763

4/6

For each edge [ui, w

j] Eis given a numberp

ij 0 called

the connection weight of [ui, w

j]. The goal of the matching

problem in bipartite graphs is to find a matching ofG with

the minimum total sum of weights. As an example, for a

bipartite graph with four nodes, the sets V,E, Uand Ware

given by:

V= {u1, u

2, w

1, w

2}

E= {[u1,w

1], [u

1,w

2], [u

2,w

1], [u

2,w

2]}

U= {u1, u

2}

W= {w1, w

2}

The equations ofTconf

and iconf

are developed to force the

validity of the structural constraints. These constraints

mean that each edge inEhas just one activated node in U

and one activated node in W. Thus, the matrix V(p) isdefined by:

=

=n

j

ij

1

1)]([ pV ; [V(p)]ij {1,0}

A valid subspace for matching problem in bipartite graphs

can be represented by:

Iconf

= V=n

1on.o

nT

It is now necessary to guarantee that the sum of the

elements of each line of the matrix Vtakes value equal to

1. Using the properties of the matrixRn, we have:

V.Rn

= Tconf

.V

In.V.R

n= T

conf.V

Using (25) and (27) in equation of the valid

subspace (V= Tconf

.V+Iconf

),

V=In

.V.Rn

+ n1on

.onT

Applying operator vec(.) given by (21) in (28),

vec(V) = vec(In.V.R

n) +

n

1vec(o

n.1.o

nT

)

vec(V) = (InRn).vec(V) +

n

1 (onon)

Changing vec(V) byv in equation (29), we have:

v= (InR

n).v +

n

1 (ono

n)

Thus, the parameters Tconf

and iconfare given by:

Tconf

= (InRn)

iconf =

n

1 (onon)

Equations (31) and (32) satisfy the properties of the valid

subspace, i.e., Tconf

.Tcon

= Tconf

and Tconf

.iconf

= 0.

The parameters Top

and iop

are obtained from the

corresponding cost constraint given by:

Eop

= trace(V(p)T.P)

Using the properties of Kronecher product in (33), we

have:

Eop

= vec(V(p)T

).vec(P) = v(p)T

.vec(P)

Comparing (34) and (6), the parameters Top

and iop

are

given by:

Top

= 0

iop

= vec(P)

To illustrate the performance of the proposed neural

network, some simulation results are presented in Section

IV.

C. Formulation of Dynamic Programming Problems

A typical dynamic programming problem can be modeled

as a set of source and destination nodes with n

intermediate stages, m states in each stage, and metric data

dxi,(x+1)j

, where x is the index of the stages, and i andj are

the indices of the states in each stage. The goal of the

dynamic programming problem considered in this paper is

to find a valid path which starts at the source node, visits

one and only one state node in each stage, reaches the

destination node, and has a minimum total length (cost)

among all possible paths.

The equations ofTconf

and iconfare developed to force the

validity of the structural constraints. These constraints, for

dynamic programming problems, mean that one and only

one state in each stage can be actived. Thus, the matrix

V(p) is defined by:

=

=m

j

ij

1

1)]([ pV ; [V(p)]ij {1,0}

A valid subspace (V=Tval

.V + Iconf

) for the dynamic

programming problem can be represented by:

(24)

(25)

(26)

(27)

(28)

(29)

(30)

(31)

(32)

(33)

(34)

(35)

(36)

(37)

8/6/2019 T3-R-00-0763

5/6

Iconf

= V=m

1 on.o

mT

Equation (38) guarantees that the sum of the elements of

each line of the matrix V takes values equal to 1.

Therefore, the term Tconf

.V must also guarantee that the

sum of the elements of each line of the matrix V takesvalue equal to zero. Using the properties of the matrixR

n,

we have:

V.Rm

= Tconf

.V

In.V.R

m= T

conf.V

Using (38) and (39) in equation of the valid

subspace (V= Tconf

.V+Iconf

),

V=In.V.R

m+

m

1 on.o

mT

Applying operator vec(.) given by (21) in (40),

vec(V) = vec(In.V.R

m) +

m

1 vec(on.1.o

mT

)

vec(V) = (InRm).vec(V) +

m

1 (onom)

Changing vec(V) byvin equation (41), we have:

v= (I

n

Rm

).v + m

1

(o

n

om

)

Thus, the parameters Tconf

and iconf

are given by:

Tconf

= (InRm)

Iconf

=m

1 (onom )

Equations (43) and (44) satisfy the properties of the valid

subspace, i.e., Tconf

.Tconf

= Tconf

and Tconf

.iconf

= 0.

The energy functionEop

of the modified Hopfield networkfor the dynamic programming problem is projected to find

a minimum path among all possible paths. When Eop

is

minimized, the optimal solution corresponds to the

minimum energy state of the network. The energy

functionEop

is defined by:

]..[+

]....[41

n

n=x 1 Term.4

,

1

1=x 1 Term.3

,

2 1 1 Term.2

)1(,)1(

1

1 1 1 Term.1

)1()1(,

==

= = =

= = =++

+

++=

m

i th

xindestinatioxi

m

i rd

xixisource

n

x

m

i

m

j nd

jxxixijx

n

x

m

i

m

j st

jxxijxxiop

vdvd

vvdvvdE

In this equation, the first term defines the weight (metriccost) of the connection linking the i

thneuron of stage x to

the jth

neuron of the following stage (x+1). The second

term defines the weight of the connection linking the ith

neuron of stagex to thejth

neuron of previous stage (x1).

The third term provides the weight of the connection

linking the source node to all others nodes of the first

stage, while the fourth term provides the weight of the

connection linking the destination to all other nodes of thelast stage. Therefore, optimization ofE

opcorresponds to

minimize each term given by (45) in relation to vxi. From

(45), the matrix Top

and vector iop

can be given by:

+=

==

+ yxyxxy

yjxiyjxixyyjxipq

opd

)1()1(

,,,

][

2

1][].[][][

Q

PQPT

]d

0000dd[

,ndestinatio2,ndestinatio1,

2)-.(

source,1msource,1211,

m

ndestinationmnn

nmm

sourceop

dd

d=i

where:

Top

nmxnm andiopnm

p = m.(x 1) + i

q = m.(y1) +j

x,y {2..n 1}

i,j {1..m}

To illustrate the performance of the proposed neuralnetwork, some simulation results are presented in the next

section.

IV. SIMULATION RESULTS

In this section, some simulation results are presented to

illustrate the application of the neural network approach

presented in the previous sections for solving matching

problems in bipartite graphs and dynamic programming

problems.

I. Bipartite Graph Optimization Problems

The modified Hopfield network has been used in the

solution of the matching problem proposed in [PAP, 82],

with matrixP given by:

=

84748

22497

81383

55969

49127

P

A graphical representation of this problem is illustrated in

Fig. 2. The modified Hopfield network has converged

after 50 iterations.

(38)

(39)

(40)

(41)

(42)

(43)

(44)

(45)

(46)

(47)

8/6/2019 T3-R-00-0763

6/6

Fig. 2. The matching problem in bipartite graphs (ten nodes).

After convergence process, the edges set, representing the

optimal solution, is given by {[1,3], [2,5], [3,1], [4,4],

[5,2]}. The vectors p and v(p), and the matrix V(p)

representing the obtained solution is provided by:

pT = [3 5 1 4 2]

]0001001000000011000000100[)( =Tpv

=

00010

01000

00001

10000

00100

)( pV

The minimization of the energy term E

op

guarantees theminimum total sum (E

op= 15) among all edges.

II. Dynamic Programming Problems

Figure 3 shows a dynamic programming problem to be

solved by the modified Hopfield network. In this problem,

the goal is to find a minimum path (among all possible

paths) which starts at the source node and reaches the

destination node, and that pass by only one state node in

each stage. The values of the weights dxi,(x+1)j

, which link

the i-th neuron of stage x to the j-th neuron of the

following stage (x+1), are also indicated in Fig. 3.

Fig. 3. Dynamic programming problem.

The optimal solution for the problem is given by theshaded states, i.e., state 2 in stage 1, state 1 in stage 2, and

state 2 in stage 3. The modified Hopfield network applied

in this problem has always converged after three

iterations. The vectors p and v(p), and the matrix V(p)

representing the obtained solution are given by:

pT

= [2 1 2] ]100110[)( =Tpv

=

10

01

10

)(pV

The minimization of the energy term Eop

guarantees that

obtained solution represents the minimum path (Eop= 21)

among all possible paths

V. CONCLUSION

This paper presents an approach for solving optimizationproblems using neural networks. Specifically, a modified

Hopfield network is used and its internal parameters are

explicitly computed using the valid-subspace technique.

The optimization problems treated in this paper are the

matching problem in bipartite graphs and the dynamic

programming problem. An energy function Eop was

designed to conduct the network output to the equilibrium

points corresponding to a cost constraint. All structural

constraints associated with the optimization problems can

be grouped inEconf.

The simulation results demonstrate that the network is an

alternative method to solve these problems efficiently. All

simulation results show that the proposed network is

completely stable and globally convergent to the solutions

of the optimization problems considered in this paper.

REFERENCES

[AIY, 90] S. V. B. Aiyer, M. Niranjan and F. Fallside , A theoretical

investigation into the performance of the Hopfield network, IEEE

Trans. on Neural Networks, vol.1, 1990, pp. 204-215.

[DIL, 02] J. D. Dillon, and M. J. OMalley, Lagrangian augmented

Hopfield network for mixed integer non-linear programming problems,

Neurocomputing, vol. 42, 2002, pp. 323-330.

[GRA, 81] A. Graham, Kronecher Products and Matrix Calculus,

Chichester: Ellis Horwood Ltd., 1981.

[HOP, 84] J. J. Hopfield, Neurons with a graded response have

collective computational properties like those of two-state neurons,

Proc. of the Nat. Academy of Science, vol. 81, 1984, pp. 3088-3092.

[KAK, 00] H. Kakeya, and Y. Okabe, Fast combinatorial optimization

with parallel digital computers, IEEE Trans. on Neural Networks, vol.

11, 2000, pp. 1323-1331.

[PAP, 82] C. H. Papadimitriou and K. Steiglitz, Combinatorial

Optimization - Algorithms and Complexity, Englewood Cliffs: Prentice-

Hall, 1982.

[SIL, 00] I. N. da Silva, A. N. de Souza, and M. E. Bordon, A

Modified Hopfield Model for Solving the N-Queens Problem, Proc. of

IEEE International Joint Conf. on Neural Networks, 2000, pp. 509-514.[TAK, 99] Y. Takahashi, Solving dynamic optimization problems with

adaptive networks, Neurocomputing, vol. 25, 1999, pp. 18-38.

(p1)

T (p2)

T (p3)

T (p4)

T (p5)

T

1

2

1

2

1

2

source destination

stage 1 stage 2 stage 3

u1

u2

u3

u4

u5

w1

w2

w3

w4

w5

p11

p12 p

21

(p1)

T (p2)

T (p3)

T