Download - T3-R-00-0763
-
8/6/2019 T3-R-00-0763
1/6
A Recurrent Neural Architecture Applied in Dynamic Programming andGraph Optimization
Ivan Nunes da Silva
Department of Electrical Engineering
University of So Paulo USP/SEL/EESC
So Carlos SP, CP 359, CEP 13566-590, Brazil
Phone: +55 (16) 3373-9367, Fax: +55 (16) 3373-9372, E-mail: [email protected]
Abstract - This paper presents an architecture of recurrent
artificial neural networks that can be used to solve bipartite
graph optimization problems and dynamic programming
problems. More specifically, a modified Hopfield network is
used and its internal parameters are explicitly computed
using the valid-subspace technique. These parameters
guarantee the convergence of the network to the equilibrium
points, which represent a solution of the consideredoptimization problem. Simulation results are presented to
validate the proposed approach.
Keywords: artificial neural networks, graph optimization,
dynamic programming, Hopfield networks.
I. INTRODUCTION
Artificial neural networks have been applied to several
classes of optimization problems and have shown promise
for solving such problems efficiently. Most of the neural
architectures proposed in literature solve specific types of
optimization problems [DIL, 02][TAK, 99][KAK, 00].Differently of these neural models, the network used here
is able to treat several kinds of problems using unique
network architecture.
The approach described in this paper uses a modified
Hopfield network with equilibrium points representing the
solution of the optimization problems. The internal
parameters of the network have been computed using the
valid-subspace technique [AIY, 90]. This technique allows
to define a subspace, which contains only those solutions
that represent feasible solution to the problem analyzed. It
has also been demonstrated that with appropriately setparameters, the network confines its output to this
subspace, thus ensuring convergence to a valid solution.
Differently of other approaches, the mapping of
optimization problems using the modified Hopfield
network always consists of determining just two energy
functions, which are defined byEconfandE
op. The function
Econf is a confinement term that groups all structural
constraints associated with the problems, and Eop is an
optimization term that leads the network output to the
equilibrium points corresponding to optimal solutions.
More specifically, the approach proposed in this paper has
been based on the model presented in [SIL, 00], whichwas used a modified Hopfield network for solving
combinatorial optimization. We have extended this
architecture in order to solve bipartite graph optimization
problems and dynamic programming problems. The main
advantages of using the modified Hopfield network
proposed in this paper are the following: i) the internal
parameters of the network are explicitly obtained by the
valid-subspace technique of solutions, ii) the valid-subspace technique groups all feasible solutions
associated with the problem, iii) lack of need for
adjustment of weighting constants for initialization, iv) for
all classes of optimization problems, a same methodology
is used to derive the internal parameters of the network.
The organization of the present paper is as follows. In
Section II, the modified Hopfield network is presented,
and the valid-subspace technique used to design the
network parameters is described. In Section III, the
mapping of optimization problems using the modified
Hopfield network is formulated. In Section IV, simulation
results are given to validate the developed approach. InSection V, the key issues raised in the paper are
summarized and conclusion drawn.
II. THE MODIFIED HOPFIELD NETWORK
Hopfield networks are single-layer networks with
feedback connections between nodes. In the standard
case, the nodes are fully connected, i.e., every node is
connected to all others nodes, including itself [HOP, 84].
The node equation for the continuous-time network with
Nneurons is given by:
=
++=N
j
bijijii itvTtutu
1
)(.)(.)(
))(()( tugtv ii =
where: ui(t) is the current state of the i-th neuron, vi(t) is
the output of the i-th neuron, bii is the offset bias of the i-
th neuron, .ui(t) is a passive decay term, Tij is the weightconnecting thej-th neuron to i-th neuron.
In (2), g(ui(t)) is a monotonically increasing thresholdfunction that limits the output of each neuron to ensure
that network output always lies in or within a hypercube.
(1)
(2)
-
8/6/2019 T3-R-00-0763
2/6
It is shown in [HOP, 84] that ifTis symmetric and =0,the equilibrium points of the network correspond to values
v(t) for which the energy function (3) associated with the
network is minimized:
bivvTv = TT ttttE )()()(2
1)(
Therefore, the mapping of optimization problems using
the Hopfield network consists of determining the weight
matrix T and the bias vector ib to compute equilibrium
points. In this paper, we have used a modified energy
functionEm(t), composed by two energy terms, defined as
follows:
Em(t) =E
conf(t) +E
op(t)
where Econf
(t) is a confinement term that groups the
structural constraints associated with the respective
optimization problem, and E
op
(t) is an optimization termthat conducts the network output to the equilibrium points
corresponding to a cost constraint. Thus, the minimization
ofEm(t) of the modified Hopfield network is conducted in
two stages:
I. Minimization of the Term Econf(t):
confTconfTconfttttE ivvTv .)()(..)(
2
1)( =
where v(t) is the network output, Tconf
is a weight matrix
and iconf
is a bias vector belonging toEconf
. This corresponds
to confine v(t) into a valid subspace generated from thestructural constraints imposed by the problem.
II. Minimization of the Term Eop
(t):
opTopTopttttE ivvTv .)()(..)(
2
1)( =
where Top
is weight matrix and iop
is bias vector belonging
to Eop
. This corresponds to move v(t) towards an optimal
solution (the equilibrium points). Thus, the operation of
the modified Hopfield network consists of three main
steps, as shown in Fig. 1:
Fig. 1. The modified Hopfield network.
Step ((I)): Minimization of Econf
, corresponding to the
projection ofv(t) in the valid subspace defined by:
vTval.v +s
where: Tval
is a projection matrix (Tval
.Tval
= Tval
) and the
vector s is orthogonal to the subspace (T
val
.s = 0). Thisoperation corresponds to an indirect minimization of
Econf
(t), i.e., Tconf
= Tval
and iconf
=s. An analysis of the valid-
subspace technique is presented in [AIY, 90].
Step ((II)): Application of a nonlinear piecewise
activation function constraining v(t) in a hypercube:
=
1if,0
10if,
1>if,1
)(
-
8/6/2019 T3-R-00-0763
3/6
problems and dynamic programming problems, is as
follows.
The vector p n represents the solution set of anoptimization problem consisted of n nodes (neurons).
Thus, the elements belonging to p have integer elements
defined by:
pi {1,...,n} where i {1..n}
The vector p can be represented by a vector v, composed
of ones and zeros, which represents the output of the
network. In the notation using Kronecher products [GRA,
81], we have:
is a matrix (nxn) defined by:
=
ji
jiij
if0,
=if,1
(k) n is a column vector corresponding to k-thcolumn of .
v(p) is an n.m dimensional vector representing the formof the final network output vector v, which corresponds to
the problem solution denoted by p. The vector v(p) is
defined by:
=
)(
)(
)(
)(2
1
np
p
p
pv
vec(U) is a function which maps the mxn matrix Uto then.m-element vector v. This function is defined by:
v = vec(U) = [U11
U21...U
m1U
12U
22...U
m2 U
1nU
2n...U
mn]
T
V(p) is an nxn dimensional matrix defined by:
=T
n
T
T
p
p
p
)(
)(
)(
)(2
1
pV
where [V(p)]ij
= [(pi)]
j.
PQdenotes the Kronecher product of two matrices. IfP is an nxn matrix, and Q is an mxm matrix, then (PQ) isa (n.m)x(n.m) matrix given by:
=QQQ
QQQ
QQQ
QP
nnnn
n
n
PPP
PP
PPP
P
21
22221
11211
whdenotes the Kronecher product of two vectors. Ifwis an n-element vector and h an m-element vector, then
(wh) is an n.m-element vector given by:
=
h
h
h
hw
.
.
.
2
1
nw
w
w
The properties of the Kronecher products [GRA, 81]
utilized are the following ones:
(wh) = (wh)
(wh)T(xg) = (wTx)(hTg)
(PQ)(wh) = (PwQh)
(PQ)(EF) = (PEQF)
vec(Q.V.PT) = (PQ).vec(V)
on and On are respectively the n-element vector and thenxn matrix of ones, that is:
}{1..,for1][
1][nji
ij
i
=
=
O
o
Rn
is an nxn projection matrix (i.e., R
n
.R
n
= R
n
)defined by:
nnn
nOIR
1=
The sum of the elements of each row of a matrix is
transformed to zero by post-multiplication withRn, while
pre-multiplication by Rn
has the effect of setting the sum
of the elements of each column to zero.
B. Formulation of Graph Optimization Problems
The graph optimization problem considered in this paper
is the matching problem in bipartite graphs. However,
several other types of graph optimization problems, such
as the graph-coloring problem and the max-clique
problem, can be solved by the proposed neural approach.
A graph G is a pair G = (V,E), where Vis a finite set of
nodes or vertices and E has as elements subsets of V of
cardinality two called edges [PAP, 82]. A matchingMof
a graph G = (V,E) is a subset of the edges with the
property that no two edges ofMshare the same node. The
graph G = (V,E) is called bipartite if the set of vertices V
can be partitioned into two sets, Uand W, and each edge
inEhas one vertex in Uand one vertex in W.
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
-
8/6/2019 T3-R-00-0763
4/6
For each edge [ui, w
j] Eis given a numberp
ij 0 called
the connection weight of [ui, w
j]. The goal of the matching
problem in bipartite graphs is to find a matching ofG with
the minimum total sum of weights. As an example, for a
bipartite graph with four nodes, the sets V,E, Uand Ware
given by:
V= {u1, u
2, w
1, w
2}
E= {[u1,w
1], [u
1,w
2], [u
2,w
1], [u
2,w
2]}
U= {u1, u
2}
W= {w1, w
2}
The equations ofTconf
and iconf
are developed to force the
validity of the structural constraints. These constraints
mean that each edge inEhas just one activated node in U
and one activated node in W. Thus, the matrix V(p) isdefined by:
=
=n
j
ij
1
1)]([ pV ; [V(p)]ij {1,0}
A valid subspace for matching problem in bipartite graphs
can be represented by:
Iconf
= V=n
1on.o
nT
It is now necessary to guarantee that the sum of the
elements of each line of the matrix Vtakes value equal to
1. Using the properties of the matrixRn, we have:
V.Rn
= Tconf
.V
In.V.R
n= T
conf.V
Using (25) and (27) in equation of the valid
subspace (V= Tconf
.V+Iconf
),
V=In
.V.Rn
+ n1on
.onT
Applying operator vec(.) given by (21) in (28),
vec(V) = vec(In.V.R
n) +
n
1vec(o
n.1.o
nT
)
vec(V) = (InRn).vec(V) +
n
1 (onon)
Changing vec(V) byv in equation (29), we have:
v= (InR
n).v +
n
1 (ono
n)
Thus, the parameters Tconf
and iconfare given by:
Tconf
= (InRn)
iconf =
n
1 (onon)
Equations (31) and (32) satisfy the properties of the valid
subspace, i.e., Tconf
.Tcon
= Tconf
and Tconf
.iconf
= 0.
The parameters Top
and iop
are obtained from the
corresponding cost constraint given by:
Eop
= trace(V(p)T.P)
Using the properties of Kronecher product in (33), we
have:
Eop
= vec(V(p)T
).vec(P) = v(p)T
.vec(P)
Comparing (34) and (6), the parameters Top
and iop
are
given by:
Top
= 0
iop
= vec(P)
To illustrate the performance of the proposed neural
network, some simulation results are presented in Section
IV.
C. Formulation of Dynamic Programming Problems
A typical dynamic programming problem can be modeled
as a set of source and destination nodes with n
intermediate stages, m states in each stage, and metric data
dxi,(x+1)j
, where x is the index of the stages, and i andj are
the indices of the states in each stage. The goal of the
dynamic programming problem considered in this paper is
to find a valid path which starts at the source node, visits
one and only one state node in each stage, reaches the
destination node, and has a minimum total length (cost)
among all possible paths.
The equations ofTconf
and iconfare developed to force the
validity of the structural constraints. These constraints, for
dynamic programming problems, mean that one and only
one state in each stage can be actived. Thus, the matrix
V(p) is defined by:
=
=m
j
ij
1
1)]([ pV ; [V(p)]ij {1,0}
A valid subspace (V=Tval
.V + Iconf
) for the dynamic
programming problem can be represented by:
(24)
(25)
(26)
(27)
(28)
(29)
(30)
(31)
(32)
(33)
(34)
(35)
(36)
(37)
-
8/6/2019 T3-R-00-0763
5/6
Iconf
= V=m
1 on.o
mT
Equation (38) guarantees that the sum of the elements of
each line of the matrix V takes values equal to 1.
Therefore, the term Tconf
.V must also guarantee that the
sum of the elements of each line of the matrix V takesvalue equal to zero. Using the properties of the matrixR
n,
we have:
V.Rm
= Tconf
.V
In.V.R
m= T
conf.V
Using (38) and (39) in equation of the valid
subspace (V= Tconf
.V+Iconf
),
V=In.V.R
m+
m
1 on.o
mT
Applying operator vec(.) given by (21) in (40),
vec(V) = vec(In.V.R
m) +
m
1 vec(on.1.o
mT
)
vec(V) = (InRm).vec(V) +
m
1 (onom)
Changing vec(V) byvin equation (41), we have:
v= (I
n
Rm
).v + m
1
(o
n
om
)
Thus, the parameters Tconf
and iconf
are given by:
Tconf
= (InRm)
Iconf
=m
1 (onom )
Equations (43) and (44) satisfy the properties of the valid
subspace, i.e., Tconf
.Tconf
= Tconf
and Tconf
.iconf
= 0.
The energy functionEop
of the modified Hopfield networkfor the dynamic programming problem is projected to find
a minimum path among all possible paths. When Eop
is
minimized, the optimal solution corresponds to the
minimum energy state of the network. The energy
functionEop
is defined by:
]..[+
]....[41
n
n=x 1 Term.4
,
1
1=x 1 Term.3
,
2 1 1 Term.2
)1(,)1(
1
1 1 1 Term.1
)1()1(,
==
= = =
= = =++
+
++=
m
i th
xindestinatioxi
m
i rd
xixisource
n
x
m
i
m
j nd
jxxixijx
n
x
m
i
m
j st
jxxijxxiop
vdvd
vvdvvdE
In this equation, the first term defines the weight (metriccost) of the connection linking the i
thneuron of stage x to
the jth
neuron of the following stage (x+1). The second
term defines the weight of the connection linking the ith
neuron of stagex to thejth
neuron of previous stage (x1).
The third term provides the weight of the connection
linking the source node to all others nodes of the first
stage, while the fourth term provides the weight of the
connection linking the destination to all other nodes of thelast stage. Therefore, optimization ofE
opcorresponds to
minimize each term given by (45) in relation to vxi. From
(45), the matrix Top
and vector iop
can be given by:
+=
==
+ yxyxxy
yjxiyjxixyyjxipq
opd
)1()1(
,,,
][
2
1][].[][][
Q
PQPT
]d
0000dd[
,ndestinatio2,ndestinatio1,
2)-.(
source,1msource,1211,
m
ndestinationmnn
nmm
sourceop
dd
d=i
where:
Top
nmxnm andiopnm
p = m.(x 1) + i
q = m.(y1) +j
x,y {2..n 1}
i,j {1..m}
To illustrate the performance of the proposed neuralnetwork, some simulation results are presented in the next
section.
IV. SIMULATION RESULTS
In this section, some simulation results are presented to
illustrate the application of the neural network approach
presented in the previous sections for solving matching
problems in bipartite graphs and dynamic programming
problems.
I. Bipartite Graph Optimization Problems
The modified Hopfield network has been used in the
solution of the matching problem proposed in [PAP, 82],
with matrixP given by:
=
84748
22497
81383
55969
49127
P
A graphical representation of this problem is illustrated in
Fig. 2. The modified Hopfield network has converged
after 50 iterations.
(38)
(39)
(40)
(41)
(42)
(43)
(44)
(45)
(46)
(47)
-
8/6/2019 T3-R-00-0763
6/6
Fig. 2. The matching problem in bipartite graphs (ten nodes).
After convergence process, the edges set, representing the
optimal solution, is given by {[1,3], [2,5], [3,1], [4,4],
[5,2]}. The vectors p and v(p), and the matrix V(p)
representing the obtained solution is provided by:
pT = [3 5 1 4 2]
]0001001000000011000000100[)( =Tpv
=
00010
01000
00001
10000
00100
)( pV
The minimization of the energy term E
op
guarantees theminimum total sum (E
op= 15) among all edges.
II. Dynamic Programming Problems
Figure 3 shows a dynamic programming problem to be
solved by the modified Hopfield network. In this problem,
the goal is to find a minimum path (among all possible
paths) which starts at the source node and reaches the
destination node, and that pass by only one state node in
each stage. The values of the weights dxi,(x+1)j
, which link
the i-th neuron of stage x to the j-th neuron of the
following stage (x+1), are also indicated in Fig. 3.
Fig. 3. Dynamic programming problem.
The optimal solution for the problem is given by theshaded states, i.e., state 2 in stage 1, state 1 in stage 2, and
state 2 in stage 3. The modified Hopfield network applied
in this problem has always converged after three
iterations. The vectors p and v(p), and the matrix V(p)
representing the obtained solution are given by:
pT
= [2 1 2] ]100110[)( =Tpv
=
10
01
10
)(pV
The minimization of the energy term Eop
guarantees that
obtained solution represents the minimum path (Eop= 21)
among all possible paths
V. CONCLUSION
This paper presents an approach for solving optimizationproblems using neural networks. Specifically, a modified
Hopfield network is used and its internal parameters are
explicitly computed using the valid-subspace technique.
The optimization problems treated in this paper are the
matching problem in bipartite graphs and the dynamic
programming problem. An energy function Eop was
designed to conduct the network output to the equilibrium
points corresponding to a cost constraint. All structural
constraints associated with the optimization problems can
be grouped inEconf.
The simulation results demonstrate that the network is an
alternative method to solve these problems efficiently. All
simulation results show that the proposed network is
completely stable and globally convergent to the solutions
of the optimization problems considered in this paper.
REFERENCES
[AIY, 90] S. V. B. Aiyer, M. Niranjan and F. Fallside , A theoretical
investigation into the performance of the Hopfield network, IEEE
Trans. on Neural Networks, vol.1, 1990, pp. 204-215.
[DIL, 02] J. D. Dillon, and M. J. OMalley, Lagrangian augmented
Hopfield network for mixed integer non-linear programming problems,
Neurocomputing, vol. 42, 2002, pp. 323-330.
[GRA, 81] A. Graham, Kronecher Products and Matrix Calculus,
Chichester: Ellis Horwood Ltd., 1981.
[HOP, 84] J. J. Hopfield, Neurons with a graded response have
collective computational properties like those of two-state neurons,
Proc. of the Nat. Academy of Science, vol. 81, 1984, pp. 3088-3092.
[KAK, 00] H. Kakeya, and Y. Okabe, Fast combinatorial optimization
with parallel digital computers, IEEE Trans. on Neural Networks, vol.
11, 2000, pp. 1323-1331.
[PAP, 82] C. H. Papadimitriou and K. Steiglitz, Combinatorial
Optimization - Algorithms and Complexity, Englewood Cliffs: Prentice-
Hall, 1982.
[SIL, 00] I. N. da Silva, A. N. de Souza, and M. E. Bordon, A
Modified Hopfield Model for Solving the N-Queens Problem, Proc. of
IEEE International Joint Conf. on Neural Networks, 2000, pp. 509-514.[TAK, 99] Y. Takahashi, Solving dynamic optimization problems with
adaptive networks, Neurocomputing, vol. 25, 1999, pp. 18-38.
(p1)
T (p2)
T (p3)
T (p4)
T (p5)
T
1
2
1
2
1
2
source destination
stage 1 stage 2 stage 3
u1
u2
u3
u4
u5
w1
w2
w3
w4
w5
p11
p12 p
21
(p1)
T (p2)
T (p3)
T