inteligenta computationala
TRANSCRIPT
-
7/28/2019 Inteligenta computationala
1/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
2/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
3/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
4/148
Chonbuk National University 2008
n
iii
n
iijij
n
iii
xwy
xwy
xwy
122
1
111
..
jiji yxw ~
1y
2y
-
7/28/2019 Inteligenta computationala
5/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
6/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
7/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
8/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
9/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
10/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
11/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
12/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
13/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
14/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
15/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
16/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
17/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
18/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
19/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
20/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
21/148
Chonbuk National University 2008
kk
m
k
kk owoy
0
-
7/28/2019 Inteligenta computationala
22/148
Chonbuk National University 2008
xo
x o
xxx mj ,...,,...1
-
7/28/2019 Inteligenta computationala
23/148
Chonbuk National University 2008
jj xxn
i
ijij xw1
tanhx
R
d
dRBF
ddRBFjj
exp)(:exemplude
;),( wxx
)1(1 jxtrix iijji x
altfel,0
;daca,1 xx
xtrinq
1numberquantanq
12,..1 nqj
-
7/28/2019 Inteligenta computationala
24/148
Chonbuk National University 2008
2exp,, kk
defk
K xxxxxx
-
7/28/2019 Inteligenta computationala
25/148
Chonbuk National University 2008
n
k
kkk Kdby
1
,xx
2
2
2exp),(
r
drdg
2
2exp,
kkK xxxx
n
i
jiij
e cxd1
2
2cx
kj xc2/1
2r
-
7/28/2019 Inteligenta computationala
26/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
27/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
28/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
29/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
30/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
31/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
32/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
33/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
34/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
35/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
36/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
37/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
38/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
39/148
Chonbuk National University 2008
n
i
jiij
mcxd
11
cx
n
i
jiij
ecxd
1
2
2cx
else1
if0),(
rk
drkd
rdm
2k
2
2
2exp),(
r
drd
g
-
7/28/2019 Inteligenta computationala
40/148
Chonbuk National University 2008
else1
if0),(
r
drd
rdm
-
7/28/2019 Inteligenta computationala
41/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
42/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
43/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
44/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
45/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
46/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
47/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
48/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
49/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
50/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
51/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
52/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
53/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
54/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
55/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
56/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
57/148
Chonbuk National University 2008
5.
-
7/28/2019 Inteligenta computationala
58/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
59/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
60/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
61/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
62/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
63/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
64/148
Chonbuk National University 2008
2
1
22
1
2
)(m
jjjo
m
jjh
wodydeJ
eJ
jj
j
jj
o
jj
h
o
ye
e
o
o
ye
e
J
ee
J
22
2
2
1
22
1
2
)(m
jjjo
m
jjh
wodydeJ
eJ
jj
j ewo
yee
-
7/28/2019 Inteligenta computationala
65/148
Chonbuk National University 2008
K
k
m
jkjkjk
K
kko wodeJ
1
2
11
2
K
kkjk
K
k kjkj weo
yee
11
-
7/28/2019 Inteligenta computationala
66/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
67/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
68/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
69/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
70/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
71/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
72/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
73/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
74/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
75/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
76/148
Chonbuk National University 2008
rule 1: IF temperature IS cool AND pressure IS weak
-
7/28/2019 Inteligenta computationala
77/148
Chonbuk National University 2008
rule 1: IF temperature IS cool AND pressure IS weak,THEN throttle is P3.
rule 2: IF temperature IS cool AND pressure IS low,THEN throttle is P2.
1,0xA
-
7/28/2019 Inteligenta computationala
78/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
79/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
80/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
81/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
82/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
83/148
Chonbuk National University 2008
xA
-
7/28/2019 Inteligenta computationala
84/148
Chonbuk National University 2008
xA
-
7/28/2019 Inteligenta computationala
85/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
86/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
87/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
88/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
89/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
90/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
91/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
92/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
93/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
94/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
95/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
96/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
97/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
98/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
99/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
100/148
Chonbuk National University 2008
Classic solution:
Alternative:
Compact Image Compression for Mobile Equipment
-
7/28/2019 Inteligenta computationala
101/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
102/148
Chonbuk National University 2008
71x71 micro meters
per cell in a 2 um techn.
-
7/28/2019 Inteligenta computationala
103/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
104/148
Chonbuk National University 2008
xxg )( )( 0123 xzzzzxf
3 absolute value nestsThe absolute value nest
-
7/28/2019 Inteligenta computationala
105/148
Chonbuk National University 2008
0 00
zf
-
7/28/2019 Inteligenta computationala
106/148
Chonbuk National University 2008
0
0
zf1k
k
k fzf
mk ,...,1
m=0
m=1
m=2
0f
)(011 fzf
)(122
fzf
0
0
0 z
10
1
0 zz
10
1
1 zz
2
1
0
2
0
z
zz
2
1
0
2
0
z
z
z2
1
0
2
0
z
z
z
2
1
0
2
0
z
z
z
3
1i
iibu
expander.nonlinearaasacting
input whensignalabecanitor
Adalineanofoutputthebe
maywhichuescalar vala-
,nests""ofnumberthe-
exponent,positiveais-
,parametersbiasare,..,0
m
zz m
1k
k
kfgzf
-
7/28/2019 Inteligenta computationala
107/148
Chonbuk National University 2008
)(xgRTD
A
g
pV VV
-
7/28/2019 Inteligenta computationala
108/148
Chonbuk National University 2008
IP
IVVV VPPVP
x
g
g
g
V
2gg
P
P
V
Ig
Pv
PV
VV
IIg
projection
-
7/28/2019 Inteligenta computationala
109/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
110/148
Chonbuk National University 2008
n
i
iiub1
msfy
The architecture of the multi-nested cell
Example:
PARITY-8: 1,2,4,0,1,1,1,1,1,1,1,1,1,1 zb s ,
-
7/28/2019 Inteligenta computationala
111/148
Chonbuk National University 2008
Digital MLP
Multi-nested PWL Cell
The M-nested Cell Equation
-
7/28/2019 Inteligenta computationala
112/148
Chonbuk National University 2008
Any Boolean function with n inputs has an associatedgene G. The main problem is to find (learn) the gene.
-
7/28/2019 Inteligenta computationala
113/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
114/148
Chonbuk National University 2008
0pZ LZ 216384214
L
-
7/28/2019 Inteligenta computationala
115/148
Chonbuk National University 2008
tDiscID
n
tanh2
2
0z
pZ
BpB b
zLZ 2
bLB
16384214
L
Found IDs 6 03 22 5 91 11 61228 61174 60074 58124 60190 524 54 46388
b 1 4 6 7 8 10 75 75 0 1500
z1 16 16 16 16 16 20 0 2 00 0 4 000
z
b
After 106
iterations
-
7/28/2019 Inteligenta computationala
116/148
Chonbuk National University 2008
30000
99995.0
-
7/28/2019 Inteligenta computationala
117/148
Chonbuk National University 2008
trials
-
7/28/2019 Inteligenta computationala
118/148
Chonbuk National University 2008
The MNEST cell f or classif icat ion problems
-
7/28/2019 Inteligenta computationala
119/148
Chonbuk National University 2008
B1=[-0.2689 0.7069 -0.8141 0.3913]Z1=[0.2501]
B2=[0.9122 0.2739 -2.6800 -1.6449]Z2=[-0.1138 0.6395 -2.0810 0.6826 -1.0501 -0.3774]
B3=[-0.7208 -0.1217 0.8324 1.1242]Z3=[-0.2136 0.1033 -0.8857 0.8326 -0.4945 -0.4378 -0.1670]
0%
1.33%
1.33%
MNEST
cells trainedto solve theIRIS
problem
The IRISproblem
Cell2
Cell1
Cell3
Missclassification error on test data
Using a digital multiplexer
16 to 1
Multiplexer
More than 120 CMOS transistors
are needed for each CNN cell
-
7/28/2019 Inteligenta computationala
120/148
Chonbuk National University 2008
p
Using an RTD-CNN cell
Iref0
Iref1
Irefm
-
7/28/2019 Inteligenta computationala
121/148
Chonbuk National University 2008
IIN2 IINm+1
IR1
I1
In
IRTD1
I1
I0
IIN1
Legend
IRTD2
-
7/28/2019 Inteligenta computationala
122/148
Chonbuk National University 2008
)( xx UsynI
inout IIII 12
-
7/28/2019 Inteligenta computationala
123/148
Chonbuk National University 2008
n
i
biibin UsynuUsynI1
0 )()(
inout 12
-
7/28/2019 Inteligenta computationala
124/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
125/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
126/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
127/148
Chonbuk National University 2008
uj
uj
-
7/28/2019 Inteligenta computationala
128/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
129/148
Chonbuk National University 2008
n
i
iinn vbzzzzsignc1
121 ...
lii icc ll
-
7/28/2019 Inteligenta computationala
130/148
Chonbuk National University 2008
A filtering problem given by training samples
Test samples used to evaluate the quality
The DRAM content (512 bits)
The result of the simplicial filtering
The simplicial filter with continuous
coefficients; error evolution during training
-
7/28/2019 Inteligenta computationala
131/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
132/148
Chonbuk National University 2008 1 bit quantization
-
7/28/2019 Inteligenta computationala
133/148
Chonbuk National University 2008
The binary gene
can be realized with
a simple m-nest cell instead
of the RAM ----->
323221432 ,,,,,, uuuuuuuuuu1u432 ,,, uuuu1u
-
7/28/2019 Inteligenta computationala
134/148
Chonbuk National University 2008
4241323221432 ,,,,,,,, uuuuuuuuuuuuuu1u
C1 C2 C3 N C1Q C2Q C3QClass 1: 19.2% 20.13% 19.95% 17% 29.35% 22.61% 20.91%
5432 ,,,, uuuuu1u
514131215432 ,,,,,,,, uuuuuuuuuuuuu1u
32514131215432 ,,,,,,,,, uuuuuuuuuuuuuuu1u
-
7/28/2019 Inteligenta computationala
135/148
Chonbuk National University 2008
,1211kk
uu mk ,...,2
-
7/28/2019 Inteligenta computationala
136/148
Chonbuk National University 2008
Objective: A c om pac t , VLSI f r ien dly neuralarchitecture for classification tasks
Precursors: Thesimplicial neural net (S-net)
-
7/28/2019 Inteligenta computationala
137/148
Chonbuk National University 2008
Problems with the S-nets : Requires hugememory O(2n) for n inputs, impractical forn>20
Main feature of the SORT net : Requires much
less memory, O(n2
) at most for n inputs, it isbased on the observation that sorting of the
input vector can provide a very effectivemean to generate a partition of the inputspace. The result is a kernel-based neural net
-
7/28/2019 Inteligenta computationala
138/148
Chonbuk National University 2008
-
7/28/2019 Inteligenta computationala
139/148
Chonbuk National University 2008
M
k
jkcy
1
For a given number of inputs provides an
expanded vector to the sorting processor. The
effect is equivalent to having amuch finer
subdivisionof the inputspace(morebasis)suchth t h d li bl b l d
-
7/28/2019 Inteligenta computationala
140/148
Chonbuk National University 2008
subdivisionof the input space (more basis) suchthat hard nonlinear problems can be solved.
ii uu 1,iml ,2FOR
121 1,, lili uu
END
How it w ork s (2 inpu t s c ase / z=1)
No expander
[1 1]Expander
[1 2]
1u
2u
1U
2U
1u
2u
1U
2U
3U
Sorter outputs: sequence of coefficientsjSorter outputs: sequence of coefficientsj
0 2 0 4 8
-
7/28/2019 Inteligenta computationala
141/148
Chonbuk National University 2008
Sorter outputs:
1 0 1 2
2 0 2 1
3 1 2 0
4 1 0 2
5 2 0 16 2 1 0
1u1
1
sequence of coefficientskj
0 5 10
0 6 9
1 6 8
1 4 10
2 4 92 5 8
Sorter outputs:
1 0 1
2 1 0
sequence of coefficientskj
0 3
1 2
21 UU
12 UU
942)5( cccy u
2u
e.t.c
0 11 -12 -1
3 X1
1 Class +2 1 Cl
If the memory stores:
kjkcj
uu y
Fast learning: all coefficients selected
when u is in 3 and 6 take value 1
-
7/28/2019 Inteligenta computationala
142/148
Chonbuk National University 2008
3 X4 15 -16 -17 X
8 -19 110 1
1 Class +2 1 Class +
3 -3 Class -4 1 Class +
5 1 Class +
6 -3 Class -
Using larger size expanders more
complicated class separation boundaries
can be learned.
Controlling the generalization performanceThe z parameter
Sorter outputs: sequence of coefficientskj5
0 4 8Sorter outputs: sequence of coefficients
kj5
0 0 4Z = 1 Z=2
-
7/28/2019 Inteligenta computationala
143/148
Chonbuk National University 2008
1 0 1 2
2 0 2 1
3 1 2 0
4 1 0 2
5 2 0 16 2 1 0
j0 5 10
0 6 9
1 6 8
1 4 10
2 4 92 5 8
1 0 1 2
2 0 2 1
3 1 2 0
4 1 0 2
5 2 0 16 2 1 0
j0 5 6
0 2 5
1 2 4
1 0 6
2 0 52 1 4
No more
than one
coefficientoverlap
Thissegment
is a
separationfrontier
3 overlapping
Coefficients
- the same
output for
either 3 or 6
This segment
is notanymore a
separation
frontier
Low numbernof inputs
M=[1 ,1 , 1 , 1 , 1 ] M=[1 ,2 , 1 , 1 , 1 ] M=[3,1,3,3,1] M=[3,1,3,3,2]
Z = 1 27% (25) 23% (36) 18% (120) 18% (143)
Problem: Phonem e 5 input s, hard (best k now n resul t : 14.2%)
-
7/28/2019 Inteligenta computationala
144/148
Chonbuk National University 2008
( )
Z= 2 29% (15) 23% (18) 20% (65) 24% (72)
Z=3 28% (10) 27% (12) 27% (33) 24.5% (36)
Z= 4 31% (5) 31% (6) 31% (22) 28.5% (24)
For low dimensional inputs, the best performance (lowestpercentage of incorrect classified patterns) is usuallyoptimized by trying various expansion schemes (M) for z=1
(no truncation in the bits representing the sequence)
Large number n of inputs
Problem: OPTD64 64 input s, 10 c lasse s (best know n resul t : 2.3%)
High memorization Less accurate separation frontier
Z= 1 2 3 4 5 6 7
-
7/28/2019 Inteligenta computationala
145/148
Chonbuk National University 2008
In general no expansion is needed or when is needed it
should be applied up to 3 times for a few of the inputs.
The best performance (lowest percentage of incorrect
classified patterns) is now optimized by trying varioustruncation values z>1.
The number in parenthesis indicates the number of
coefficients to be stored
Misclerror
%
9.69
(26330)
8.86
(13760)
7.75
(7280)
8.08
(3980)
9.58
(2240)
15.6
(1250)
90
(640)
-
7/28/2019 Inteligenta computationala
146/148
Chonbuk National University 2008
The main advantage is implementation simplicity for good
performance, compared with more sophisticated classifiers
-
7/28/2019 Inteligenta computationala
147/148
Chonbuk National University 2008
performance, compared with more sophisticated classifiers
Using standard RAMs to store weights is very convenient and
can lead to very compact designs in the absence of
multipliers. The most critical component is the sorter but
some good VLSI implementations were already reported.
Problem: Choosing the right expansion vector M (GA or other
optimization techniques may be used) -> may take time,
cannot be determined using a simple method.
-
7/28/2019 Inteligenta computationala
148/148
Chonbuk National University 2008