Multi-Valued Neuron with Sigmoid Activation

Function

Shin-Fu Wu 2013/5/10

Outlines

Multi-Valued Neuron (MVN)

MVN with Sigmoid Activation Function (MVN-sig)

Motivation and Expectation

Multi-Valued Sigmoid Activation Function

Learning using Back-propagation Rule

Simulation Results

Benchmark simulations

Problems and Limitations

Movement of Weighted Sum

Future Works

MVN

MVN-sig

Motivation and Expectation

Basic idea: approach the functionality of MVN using sigmoid function

Differentiable

Multi-Valued Logic

Expectation:

Better performance and tolerance

More execution time

MVN-sig

arg(z)

...

2π 0

1

2

k-1

...

τ 1 τ 2 τ 3 τ k-1 ...

MVN-sig

MVN-sig

MVN-sig

MVN-sig

MVN-sig

Simulation Results

Benchmark Simulations

Wine dataset (5-fold CV, 100% trained)

MVN

Epoch

Sec. acc.

109.2 0.6581

90.17

115.6 0.7072

90.17

91.6 0.5966

90.17

100.4 0.6506

89.69

121.2 0.6927

90.17

107.6 0.6610

90.074

MVN-sig (C=1)

Epoch Sec. Acc.

1228.6 5.9988

70.58

705.2 5.0383

71.53

656 4.2992

70.58

1425.6 6.8748

71.53

1089 8.0087

70.10

1020.88

6.0440

70.86

MVN-sig (C=5)

Epoch Sec. Acc.

1048.6 3.3985

93.50

2022 3.4830

94.02

410.4 2.6888

93.50

3691 9.7389

93.02

299.2 2.0722

92.51

1494.24

4.2763

93.31

Simulation Results

Glass identification dataset (5-fold CV, 100% trained)

MVN

Epoch

Sec. acc.

108.6 0.6565

91.12

123.8 0.6907

89.21

131.6 0.7178

89.69

103 0.6129

90.60

110 0.6461

90.17

115.4 0.6648

90.158

MVN-sig (C=1)

Epoch

Sec. Acc.

1143.4

4.7687

72.48

1032.6

7.4362

70.62

458.4 2.3950

72.01

388.4 2.0841

71.06

899.8 4.4695

72.05

784.52

4.2307

71.644

MVN-sig (C=5)

Epoch

Sec. Acc.

281.4 3.4416

90.21

207.2 1.9953

93.98

1040.4

5.8584

90.64

1939.2

7.9271

90.60

348.8 3.2267

93.54

763.4 4.4898

91.784

Simulation Results

Simulation Results

Future Works