Com

plex

Pro

blem

Sol

ving

With

Neu

ral

Net

wor

ks: L

earn

ing

Che

ss

Mr.

Jack

Sig

anD

r. A

leks

ande

r Mal

inow

ski,

Adv

isor

Dep

t. of

Ele

ctric

al a

nd C

ompu

ter E

ngin

eerin

g

Apr

il 19

, 200

5

Out

line

•In

trodu

ctio

n to

pro

ject

and

neu

ral n

etw

orks

•N

eura

l net

wor

ks a

nd c

hess

•N

eura

l net

wor

k sy

stem

des

ign

•To

ols

and

proc

edur

es–

Pre

proc

essi

ng–

Trai

ning

–In

terfa

ce•

Res

ults

and

con

clus

ions

–In

itial

resu

lts–

Cre

atio

n of

the

eval

uatio

n fu

nctio

n–

Fina

l con

clus

ions

Sys

tem

Blo

ck D

iagr

am

AN

N“S

yste

m”

Inte

rfac

e (G

ener

ates

Bo

ard

D

escr

ipti

on)

Pla

yer

Mo

ve

AN

N M

ove

(s)

AN

N“S

yste

m”

Dat

a P

rep

roce

ssin

g

Bo

ard

Po

siti

on

Ch

ose

n M

ove

s

Pla

yin

g o

r A

dvi

sory

Mo

de

Lea

rnin

g M

od

e

1792

Fee

dfo

rwar

d n

etw

ork

s:2

laye

rs o

f 128

no

des

Gam

e D

ataB

ase

Neu

ral N

etw

orks

W1

W2

Wn

Thre

shol

d (T

)

. . .

X 1 X 2 X n

∑ =n ii

iWX

0)

(O

ut (Y

)

∑ =

−=

n ii

iT

WX

Tanh

Y0

))

((

Figu

re 1

A: N

ode

stru

ctur

e w

ithhy

perb

olic

tang

ent a

ctiv

atio

nfu

nctio

n

Figu

re 1

B: S

impl

e pa

rtial

lyco

nnec

ted

neur

al n

etw

ork

stru

ctur

e fro

m S

tuttg

art

Neu

ral N

etw

ork

Sim

ulat

or(S

NN

S)

Che

ss a

nd N

eura

l Net

wor

ks?

•D

emon

stra

tes

com

plex

dec

isio

n m

akin

g•

Hig

hly

nonl

inea

r pro

blem

•S

chem

as•

Wid

ely

stud

ied

•M

assi

ve a

mou

nts

of a

vaila

ble

data

•S

ucce

ss w

ith c

heck

ers

•M

ixed

resu

lts w

ith c

hess

in th

e pa

st

Sta

ndar

ds a

nd re

sear

ch•

Num

erou

s ap

plic

able

“sta

ndar

ds”

–C

hess

“law

s”•

FID

E (F

édér

atio

n In

tern

atio

nale

des

Éch

ecs)

•ht

tp://

ww

w.fi

de.c

om–

PG

N fi

le s

tand

ard

•re

c.ga

mes

.che

ss, 1

994

–E

PD

file

sta

ndar

d•

Form

at s

uppl

ied

by “E

PD

_Pos

ition

.exe

” (st

anda

rd?)

–C

hess

eng

ine

stan

dard

com

man

d re

fere

nce

•M

ost i

nflu

entia

l res

earc

h–

K. C

hella

pilla

and

D.B

. Fog

el–

C. P

osth

off,

S. S

chaw

elsk

i and

M. S

chlo

sser

Par

alle

l Net

wor

k D

esig

ns

•D

ecre

ase

lear

ning

cyc

le ti

me

•Le

ss d

estru

ctiv

e le

arni

ng p

roce

ss•

Mul

tiple

“sug

gest

ed” m

oves

may

be

retu

rned

•Si

mpl

er n

etw

ork

arch

itect

ures

•2

para

digm

s fo

r dec

onst

ruct

ing

ches

s–

Geo

grap

hica

l “m

ove

base

d”–

Func

tiona

l “pi

ece

base

d”•

Exte

rnal

logi

c is

app

lied

to b

oth

para

digm

s to

filte

r out

ille

gal m

oves

or i

mpo

ssib

le m

oves

Par

alle

l Net

wor

k D

esig

ns

Figu

re 3

: Map

ping

the

lega

l mov

es in

che

ss, u

sing

ove

rlay

cons

istin

g of

que

en +

kni

ght m

oves

Exc

ludi

ng c

astli

ng, t

here

are

185

6 m

oves

pos

sibl

e—

igno

ring

the

piec

e ty

pe w

hich

is m

oved

Exa

mpl

e: M

ovin

g fro

m d

3 to

d4

is c

onsi

dere

d O

NE

pos

sibl

em

ove,

whe

ther

the

piec

e is

a p

awn,

que

en, k

ing,

etc

.

I

I

Par

alle

l Net

wor

k D

esig

ns•

star

ting

posi

tion

i•

final

pos

ition

f•

mif=

f(bi)

repr

esen

ts a

ll le

gal m

oves

for a

boa

rd p

ositi

on b

i•

gam

e g

of n

mov

es m

ay b

e ex

pres

sed

as a

set

of b

oard

posi

tions

bi,

biЄ

g, w

here

i is

the

posi

tion

num

ber 0

to n

.

∑ =

)(

=n i

ibf

L0 ∑∞ =

)=

0

(i

iLM

•In th

is d

esig

n, it

is re

quire

d to

cre

ate

an in

divi

dual

AN

N s

truct

ure

for a

ll m

oves

t, tЄ

M. M

=185

6, ig

norin

g ca

stlin

g

Lega

l mov

es fo

r a g

ame

of n

pos

ition

s

Lega

l mov

es fo

r che

ss (a

ll ga

mes

)

App

roac

h A

: Geo

grap

hica

l

AN

NPo

sitio

n A

8-A7

AN

NPo

sitio

n A

8-B8

AN

NPo

sitio

nH

2-H1

RU

LE L

OG

ICB

inar

y O

utpu

t‘1

’=Le

gal

Mov

e

Dec

isio

n Fi

naliz

atio

n(P

ick

The

Stro

nges

t Out

put F

rom

The

Out

put S

et)

Save

AN

N R

esul

t And

Le

galit

y Fo

r Ada

ptiv

e R

eson

ance

Tra

inin

g . .

.

‘1’ W

ill C

lose

“S

witc

h”‘0

’ Dis

able

s M

ove

Out

put

(Mov

e)

Figu

re 4

: Des

ign

with

“mov

e sp

ecifi

c” n

eura

l net

wor

ks—

the

geog

raph

ical

app

roac

h

Par

alle

l Net

wor

k D

esig

ns•

star

ting

posi

tion

i•

final

pos

ition

f•

piec

e p

•m

if=f(p

,bi)

repr

esen

ts a

ll le

gal m

oves

for a

pie

ce in

bi

•ga

me

g of

n m

oves

may

be

expr

esse

d as

a s

et o

f boa

rdpo

sitio

ns b

i, biЄ

g, w

here

i is

the

mov

e nu

mbe

r 0 to

n.

∑ =

)(

=n i

ibp

fL

0

,

∑∞ =

)=

0

(i

iLM

•In th

is d

esig

n, it

is re

quire

d to

cre

ate

an in

divi

dual

AN

N s

truct

ure

for a

ll pi

eces

p, p

=1 to

16

Lega

l mov

es fo

r a g

ame

of n

mov

es

Lega

l mov

es fo

r a p

iece

(all

gam

es)

App

roac

h B

: Fun

ctio

nal

Bla

ck P

awn

1B

lack

Paw

n 2

Bla

ck K

ing

RU

LE L

OG

ICB

inar

y O

utpu

t‘1

’=Le

gal

Mov

e

Dec

isio

n Fi

naliz

atio

n (P

ick

The

Stro

nges

t Out

put F

rom

The

O

utpu

t Set

)

Sav

e E

ach

AN

N R

esul

t A

nd L

egal

ity F

or U

se

In A

dapt

ive

Res

onan

ce T

rain

ing

. .

.

‘1’ W

ill C

lose

“S

witc

h”‘0

’ Dis

able

s M

ove

Out

put

(Mov

e)

Switc

h fo

r EA

CH

O

utpu

tFi

gure

5: D

esig

n w

ith “p

iece

sp

ecifi

c” n

eura

l net

wor

ks—

the

func

tiona

l app

roac

h

Fina

l AN

N S

yste

m

AN

NPo

sitio

n A

8-A7

AN

NPo

sitio

n A

8-B8

AN

NPo

sitio

nH

2-H1

Dec

isio

n Fi

naliz

atio

n(P

ick

The

Stro

nges

t Out

put F

rom

The

Out

put S

et)

. .

.

‘1’ W

ill C

lose

“S

witc

h”‘0

’ Dis

able

s M

ove

Out

put

(Mov

e)

Eval

uatio

n Fu

nctio

n an

d R

ule

Logi

c

Cur

rent

B

oard

Po

sitio

n

Des

ign

with

“mov

e sp

ecifi

c” n

eura

l net

wor

ks—

the

geog

raph

ical

app

roac

h

Dat

a R

epre

sent

atio

ns

1. e4 d6 2. d4 Nf6 3. Nc3 g6 4. Nf3 Bg7 5. Be2 O-O 6. O-O Bg4

7. Be3 Nc6 8. Qd2 e5 9. d5 Ne7 10. Rad1 Bd7 11. Ne1 Ng4 12. Bxg4 Bxg4

13. f3 Bd7 14. f4Bg4 15. Rb1 c6 16. fxe5 dxe5 17. Bc5 cxd5 18. Qg5 dxe4

19. Bxe7 Qd4+ 20. Kh1f5 21. Bxf8 Rxf8 22. h3 Bf6 23. Qh6 Bh5 24. Rxf5 gxf5

25. Qxh5 Qf2 26. Rd1e3 27. Nd5 Bd8 28. Nd3 Qg3 29. Qf3 Qxf3 30. gxf3 e4

31. Rg1+ Kh8 32. fxe4 fxe4 33. N3f4 Bh4 34. Rg4 Bf2 35. Kg2 Rf5 36. Ne7 1-0

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - pm d4;

rnbqkbnr/pppppppp/8/8/3P4/8/PPP1PPPP/RNBQKBNR b KQkq d3 pm Nf6;

rnbqkb1r/pppppppp/5n2/8/3P4/8/PPP1PPPP/RNBQKBNR w KQkq - pm Nf3;

rnbqkb1r/pppppppp/5n2/8/3P4/5N2/PPP1PPPP/RNBQKB1R b KQkq - pm b6;

Figu

re 6

: Exa

mpl

e of

the

PG

N (a

lgeb

raic

) sta

ndar

d

Figu

re 7

: Exa

mpl

e of

the

EP

D (s

tring

) for

mat

Inpu

t Vec

tor C

reat

ion

0.1,

-0.1

p,P

Paw

n

0.3,

-0.3

b.B

Bis

hop

0.4,

-0.4

n,N

Knig

ht

0.5,

-0.5

r,RR

ook

0.9,

-0.9

q,Q

Que

en

1.0,

-1.0

k,K

Kin

g

Wei

ght

Bla

ck, w

hite

EPD

Cha

ract

erB

lack

, whi

tePi

ece

“Sta

ndar

d” v

alue

s fo

r pie

ces

used

in th

e flo

atin

g po

int

inpu

t vec

tor c

reat

ion

Floa

ting

poin

t inp

ut v

ecto

r for

the

initi

al b

oard

pos

ition

Pre

proc

essi

ng

PGN

File

EPD

File

Floa

ting

Poin

tVe

ctor

Dat

abas

eSe

lect

ions

Star

tEn

d

The

gam

e da

ta p

repr

oces

sing

pro

cedu

re

•Boa

rd p

ositi

onal

dat

a, p

lus

a ‘y

es’ o

r ‘no

’ dec

isio

n in

rega

rds

to m

akin

gth

e sp

ecifi

c m

ove

mus

t be

in th

e flo

atin

g po

int v

ecto

r.

•A m

ix o

f ‘ye

s’ a

nd ‘n

o’ s

ampl

es w

ill b

e us

ed in

all

train

ing

sets

•Tra

inin

g se

ts a

re ra

ndom

ly c

hose

n

•Do

not d

iffer

entia

te b

etw

een

piec

es to

be

mov

ed, o

nly

the

initi

al a

nd fi

nal

posi

tions

are

impo

rtant

—ru

le lo

gic

is s

epar

ate

Trai

ning

Set

up

. .

. .

PC1

PC2

PCn

GD

AN

SK S

ERV

ER Networ

k file

Train

ing

data

Trai

ning

Pro

cess

Sta

rt

Ob

tain

netw

ork

n

am

e f

rom

co

nfi

gu

rati

on

file

Ob

tain

req

uir

ed

raw

d

ata

files

(do

wn

load)

Gen

era

te t

rain

ing

d

ata

set

fro

m r

aw

d

ata

Init

ialize

a n

ew

n

etw

ork

Ch

eck f

or

up

date

s

an

d p

rocess t

hem

Sta

rt S

NN

S, lo

ad

n

etw

ork

an

d d

ata

Tra

in 5

0 e

po

ch

s

Sen

d t

rain

ed

n

etw

ork

to

serv

er

(FT

P)

Mo

re

netw

ork

s t

o

train

?

Gen

era

te s

cri

pts

an

d b

atc

h f

iles—

Sta

rt b

atc

h

execu

tio

n

Setu

p(S

hell

P

rog

ram

)

Data

P

rocessin

gT

rain

ing

Sta

rt c

on

tro

l scri

pt

Exit

Bu

tto

n(K

ill all

pro

cesses)

En

dY

es

Trai

ning

Pro

blem

s•

50 e

poch

s of

abo

ut 5

thou

sand

pat

tern

s•

1792

net

wor

ks w

ith 1

8 m

inut

es p

erne

twor

k•

SN

NS

is N

OT

the

mos

t ele

gant

sol

utio

n!–

Java

ver

sion

lack

s sc

riptin

g ab

ility

–Sl

ow tr

aini

ng–

Varia

ble

exec

utio

n tim

e•

Pro

blem

s w

ith .N

ET

/ lab

mac

hine

s

Inte

rface

Mod

ule

Initi

al F

indi

ngs

and

Res

ults

•B

ackp

rop

does

not

wor

k•

Usi

ng re

silie

nt b

ackp

rop

inst

ead

•N

o si

gnifi

cant

diff

eren

ce b

etw

een

2 or

mor

e hi

dden

laye

rs (i

n tra

inin

g sp

eed)

•C

lose

out

put p

roxi

mity

—ho

w to

dec

ide?

•A

n ev

alua

tion

func

tion

is re

quire

d

Eva

luat

ion

Func

tion

Eva

luat

ion

Func

tion

1

•O

nly

top

NN

out

puts

will

be

cons

ider

ed•

Rat

e th

e m

oves

, ign

ore

NN

out

put

•C

onsi

der

–M

ater

ial (

?M)

–Th

reat

s (?

T)–

Mob

ility

(?O

)–

Vul

nera

bilit

ies

(?V

)•

Scor

e =

a*?

M +

b*?

T +

c*?

O +

d*?

V•

a,b,

c,d

are

wei

ghts

(exp

erim

enta

lly d

eter

min

ed)

Eva

luat

ion

Func

tion

2•

Onl

y to

p N

N o

utpu

ts w

ill b

e co

nsid

ered

•R

ate

the

mov

es w

ith N

N o

utpu

t as

a fa

ctor

•C

onsi

der

–M

ater

ial (

?M)

–Th

reat

s (?

T)–

Mob

ility

(?O

)–

Vul

nera

bilit

ies

(?V

)–

NN

out

put (

Y)

•Sc

ore

= a*

?M

+ b

*?T

+ c*

?O

+ d

*?V

+ Y

•a,

b,c,

d ar

e w

eigh

ts (e

xper

imen

tally

det

erm

ined

)

Fina

l Eva

luat

ion

Func

tion

•A

che

ss e

ngin

e is

util

ized

–Pr

ovid

es a

n ex

perim

enta

l “ba

selin

e”–

Prov

ides

a b

oard

eva

luat

ion

“E” o

nly

•W

hite

sid

e ch

oose

s m

ove

base

d on

“E”

•B

lack

use

s E

+a*Y

•Y

is th

e ne

ural

net

wor

k ou

tput

•Th

e “a

” coe

ffici

ent i

s fo

und

expe

rimen

tally

Find

ings

and

Res

ults

•N

eura

l net

wor

k ou

tput

can

con

tribu

te to

the

eval

uatio

n of

a m

ove

•Pe

rform

ance

of t

he N

N s

yste

m is

sig

nific

antly

bette

r tha

n th

e ch

ess

engi

ne a

lone

•Th

e sy

stem

per

form

s ve

ry p

oorly

in e

nd-g

ame

scen

ario

s, p

ossi

bly

indi

catin

g a

need

for p

iece

-sp

ecifi

c kn

owle

dge

•M

odifi

ed tr

aini

ng m

ay le

ad to

impr

ovem

ents

inpe

rform

ance

…

Find

ings

and

Res

ults

Avera

ge Sc

ore vs

Ply

-4-3-2-1012345

05

1015

2025

3035

4045

Ply

Average Score (Gaviota Engine)

Contr

olNN

x2

Trai

ning

Vec

tor C

odin

g•

May

nee

d to

eva

luat

e ot

her f

orm

ats

–B

inar

y re

pres

enta

tion

(use

d in

end

-gam

e re

sear

ch)

–M

ultip

le s

patia

l rel

atio

nshi

ps?

•Tr

aini

ng m

ust b

e st

arte

d to

kno

w!

Figu

re 1

1: P

ossi

ble

spat

ial r

elat

ions

hips

Add

ition

al q

uest

ions

and

com

men

ts a

rein

vite

d. P

leas

e co

ntac

t:Ja

ck S

igan

, jsi

gan@

brad

ley.

edu

Pro

ject

Web

site

:ht

tp://

cegt

201.

brad

ley.

edu/

proj

ects

/pro

j200

5/nn

ches

s/

Que

stio

ns?