9 9 9 8 () 2 4kazu/ecei-experimentd/2019/s… · april, 2019 0 0 0 d [kazuyuki tanaka part 5] 12...

26
April, 2019 999D [Kazuyuki Tanaka Part 5] 1 999 8 () 24 501 0119 (22/4441,#/)/)12.(.+3#%*/7+#63 8 "&$/#'& (22/4441,#/)/)12.(.+3#%*/7+#63! 5/&0),&-2

Upload: others

Post on 23-Sep-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019��9��9��9�����D

[Kazuyuki Tanaka Part 5] 1

��9��9��9������� ����8���

() 2 4

5 01 01 1 9

(22/���444�1,#/)/�)1�2.(.+3�#%�*/�7+#63�

����8"&$/#'&�(22/���444�1,#/)/�)1�2.(.+3�#%�*/�7+#63� � !� 5/&0),&-2�������

Page 2: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019U3�Q��U��#� ��WD

[Kazuyuki Tanaka Part 5] 2

-PF_�G(7

:���J: =8}q�^`a;��9$L�R, /��6, 2006.:���J: zdkb�upr��g_DN<&O_'9, h�t>, 2009.:���EJ: H*��'9?�SGC�dx��Y=8<#��9\DN�� ---0X]by��o\[_o~�r�b�Z, ide�l>�2006.�:�2, 5�V�:���J ( %�C): ---CVIMo~�r�b�j��m--- h�w~�nvk��+�Afds3(�,!��)KITE), B6@��M1=8�\=8<;��9_4�\�S�pp.137-179, bsh{�|qcb."�>, December 2010. Kazuyuki Tanaka: Statistical-mechanical approach to image processing (Topical Review), Journal of Physics A: Mathematical and General, vol.35, no.37, pp.R81-R150, 2002.C. M. Bishop: Pattern Recognition and Machine Intelligence, Springer, 2007.M. Opper and D. Saad: Advanced Mean Field Method, MIT Press, 2001.H. Nishimori: Statistical Physics of Spin Glasses and Information Processing, ---An Introduction---, Oxford University Press, 2001.M. J. Wainwright and M. Jordan: Graphical Models, Exponential Families, and Variational Inference (Foundations and Trends® in Machine Learning), Now Publishers, 2008.M. Mezard and A. Montanari: Information, Physics, and Computation, Oxford University Press, 2009.

Page 3: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

Contents

1. ��?��� ���#5).'<3/2;>+

2. ��$����3. �������$��

---69-4� 7<0*9:�#�����---4. �������#5).'<3/2;>+

---69-4���#�����---5. ����#5).'<3/2;>+---,84(*9719#�����---

6. %#&

April, 2019!�=��=!�= ��"D

[Kazuyuki Tanaka Part 5] 3

Page 4: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019(�W"�W(W����)D

[Kazuyuki Tanaka Part 5] 4

����������

��� ��RJU

ASO>@URJU

P?F8�

P?D=VMHK�����%��

401<8NXL/$#2+56&956,:Y

����;�-JXG7�.-:� CEIQ

���';��3:*��8!�=UBTFQ8���

Page 5: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019� �����������D

[Kazuyuki Tanaka Part 5] 5

������ �

{ } { }{ }

{ } { } { }AABBAABAAB

PrPr,PrPr,PrPr

ºA

B

�����������

��!������ }Pr{A

��A���B����� { } { }BABA Ç= Pr,Pr

Page 6: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019�������������D

[Kazuyuki Tanaka Part 5] 6

��� ����

{ } { }ååå=A C D

DCBAB ,,,PrPr

�� B �����

A B

C D

���

Page 7: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019�������������D

[Kazuyuki Tanaka Part 5] 7

������� �

{ } { } { }{ } { } { }{ } { } { }AABBC

AABBAC

BABACCBA

PrPrPr

PrPr,Pr

,Pr,Pr,,Pr

=

=

=

{ } { }{ }{ }

{ }å

å=

=

BA

BCBA

CBACCACA

,,,Pr

,,PrPr,PrPr

A

B

C

{ }ABPr

{ } { }BCBAC Pr,Pr =�������

Page 8: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019��:��:��:����D

[Kazuyuki Tanaka Part 5] 8

��$6.0-9423&�

�����%+!�*&'� "�)��<��� ���"�)��<�+#(1589/7;,������"�)��<

{ } { }? TTTT WSWR ==<>

== AAAA PrPr

Page 9: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019 ���� �������D

[Kazuyuki Tanaka Part 5] 9

�������������

{ }{ } { }{ } { } { }{ } { } { } { }{ } { } { } { }CCSCRRSW

CCSSCRRSCW

SCSCRRSCW

RSCRSCW

WRSC

AAAAAAAA

AAAAAAAAAA

AAAAAAAAA

AAAAAAAAAAA

PrPrPr,Pr

PrPr,Pr,,Pr

,Pr,Pr,,Pr

,,Pr,,Pr,,,Pr

=

=

=

=

Page 10: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019 ���� �������D

[Kazuyuki Tanaka Part 5] 10

�������������

{ } { }{ }TPr

TT,PrTTPrW

WRWR =

=====

AAAAA

{ } { }{ }TPr

TT,PrTTPrW

WSWS =

=====

AAAAA

Page 11: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019 ���� �������D

[Kazuyuki Tanaka Part 5] 11

�������������

{ } { } 2781.0,,,Pr,Pr ====== å å= =FT, FT,

WRSCWSC R

TTTTA A

AAAAAA

{ } { } 6471.0T,,,PrTPrFT, FT, FT,

WRSCWC S R

==== å å å= = =A A A

AAAAA

{ } { } 4581.0,,,Pr,Pr ====== å å= =FT, FT,

WRSCWRC S

TTTTA A

AAAAAA

Page 12: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019��0��0��0����D

[Kazuyuki Tanaka Part 5] 12

�� .)*(/-+,"�

{ } { }{ } 7079.0

6471.04581.0

TPrTT,PrTTPr

W

WRWR ==

===

===A

AAAA

{ } { }{ } 4298.0

6471.02781.0

TPrTT,PrTTPr

W

WSWS ==

===

===AAAAA

� 2���!'��&"#�����������%'$�1

Page 13: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019!�G��G!G����"D

[Kazuyuki Tanaka Part 5] 13

35� .A8:7F>;<

?H=�--2/�����/���%��

Falseor True=ix

( )

( ) ( ) ( )

}}

} ;,,,

F){or Tfor(

F){or Tfor( F){or Tfor(

;0

2111

2

2

1

!

"

!

L

L

xxxPxPxPx

xxxP

+¬=

==¬

( )åå åº2 3

,,,,)( 32111x x x

LL

xxxxPxP !!

( )( )2ln1exp2 1 -=- LL

( )LeO

12 -L �DH@��/�*#+� /��(6@E9CB

&0@E9CB,1L=10�0?H=,1�$$6-')4L=20�,�17�IL=30�,�12�IL=40�,�34�$$6J

Page 14: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019(�K%�K(�K��� )D

[Kazuyuki Tanaka Part 5] 14

�����

1. '"=�/4+?HE�6��GBJ5�.1��4 �=�,;N2. '"=�0?HE�6��GBJ27$�>J@IAF34;N

O���6DLC6��27*;-'"=9?HE�6��GBJ2�����6�&=!�.M�:<;$� �3��4 �=�#.18;N

{ } { } { }{ } { }{ } { }{ } { }21

1324

25436

67658

PrPrPrPr

Pr,Pr

XPr,Pr Pr

XXXXXX

XXXXX

XXXX

´

´

´

=X

Page 15: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019� �� �� ����D

[Kazuyuki Tanaka Part 5] 15

��������������������

{ }

),(),(),( ),(),,(),,(x

,,,Pr

311342245225

7667543346865568

882211

xxWxxWxxWxxWxxxWxxW

xXxXxX

´=

=== !

1X

3X

2X

4X

6X 5X

13W

67W

24W

25W346W

568W

8X7X

�����

�����

Page 16: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019������������D

[Kazuyuki Tanaka Part 5] 16

������

{ }{ }åº

iX

ii

XXXXXXXXXP

\87654321 ,,,,,,,Pr

)(

X

{ }{ }åº

kji XXXkjiijk XXXXXXXXXXXP

,,\87654321 ,,,,,,,Pr),,(

X

{ }{ }åº

ji XX

jiij

XXXXXXXX

XXP

,\87654321 ,,,,,,,Pr

),(

X

1X

3X

2X

4X

6X 5X

13W

67W

24W

25W346W

568W

8X7X

Page 17: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019������������D

[Kazuyuki Tanaka Part 5] 17

������������

1X

3X

2X

4X

6X 5X

8X7X

346W

)( 3133 XM ¬ )( 4244 XM ¬

)( 6676 XM ¬

)( 65686 XM ¬

3X 4X

6X 5X

8X7X

)( 6676 XM ¬

)( 65686 XM ¬

)( 63466 XM ¬

( ) ( )åå=3 4

64334666 ,,x x

xxxPxP

( ) ( )

( ) ( )667665686

634666

661

xMxM

xMZ

xP

¬¬

¬

´

=( ) ( )

( ) ( )( ) ( )667665686

42443133

643346346

643346

,,1,,

xMxMxMxM

xxxWZ

xxxP

¬¬

¬¬

´´

=

Page 18: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019��������������D

[Kazuyuki Tanaka Part 5] 18

��������� �

( )

( )ååååå

¬¬

¬¬

¬ =

3 4 6

3 4

)()(),,(

)()(),,()(

42443133643346

42443133643346

63466

x x x

x x

xMxMxxxW

xMxMxxxWxM

����������

1A

3A

2A

4A

6A

)( 3133 XM ¬ )( 4244 XM ¬

)( 63466 XM ¬

1X

3X

2X

4X

6X 5X

13W

67W

24W

25W346W

568W

8X7X

Page 19: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019������������D

[Kazuyuki Tanaka Part 5] 19

Message������������

ååå ÷÷ø

öççè

æ´

÷÷ø

öççè

æ´

=

¬¬

¬¬

¬¬

¬¬

3 4 6)()()()(),,(

)()()()(),,(

),,(

656866676

42443133643346

656866676

42443133643346

643346

X X X XMXMXMXMXXXW

XMXMXMXMXXXW

XXXP

1X

3X

2X

4X

6X 5X

8X7X

346W

)( 3133 XM ¬ )( 4244 XM ¬

)( 6676 XM ¬ )( 65686 XM ¬

Page 20: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019������������D

[Kazuyuki Tanaka Part 5] 20

Message Passing Algorithm

( )

( )ååååå

¬¬

¬¬

¬ =

3 4 6

3 4

)()(),,(

)()(),,()(

42443133643346

42443133643346

63466

X X X

X X

XMXMXXXW

XMXMXXXWXM

Message Passing Algorithm

1X

3X

2X

4X

6X

)( 3133 XM ¬ )( 4244 XM ¬

)( 63466 XM ¬

1X

3X

2X

4X

6X 5X

13W

67W

24W

25W346W

568W

8X7X

3W 4W

5W

6W

Page 21: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019��+��+�+�� ��D

[Kazuyuki Tanaka Part 5] 21

������"���

������ ( )** MM!!!

Y=���

( )( )( )

!

"""

"""

"""

23

12

01

MM

MM

MM

�'�!��*��#�)( "#%',������$�����#�&)(-

0M1M

1M

0

xy =

)(xy Y=

y

x*M

Page 22: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019������������� D

[Kazuyuki Tanaka Part 5] 22

��

4393.0)0(5607.0)1( 88 == PP

4360.0)0(5640.0)1( 88 == PP

Belief Propagation

Exact

1X

3X

2X

4X

6X 5X

13W

67W

24W

25W346W

568W

8X7X

{ }åº

8\8765432188 ),,,,,,,()(

x

xxxxxxxxPxPx

3629.0)0,0( 0871.0)1,0(0764.0)0,1( 4736.0)1,1(

5858

5858

====

PPPP

3636.0)0,0( 0864.0)1,0(0724.0)0,1( 4776.0)1,1(

5858

5858

====

PPPP

{ }åº

85 ,\876543218558 ),,,,,,,(),(

xx

xxxxxxxxPxxPx

Page 23: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019��������������D

[Kazuyuki Tanaka Part 5] 23

����

{ }{ }

{ } 8261.04393.03629.0

PresentPrPresent,PresentPr

PresentPresentPr

Dyspnea

DyspneaBronchitis

DyspneaBronchitis

===

===

==

XXX

XX���

1X

3X

2X

4X

6X 5X

13W

67W

24W

25W346W

568W

8X7X

3W 4W

5W

6W

Page 24: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019� ,��,��,�����D

[Kazuyuki Tanaka Part 5] 24

�����

}|{ Ωx Î= ixi

!!!!!!!!!! "!!!!!!!!!! #$ij

PPnPnPnPjj xnxnjjj

i

Node at Field External to respect with Node at Averageof Deviation

)()()()( åå -=-ºDxx

xx ,,)( ~~)( dd

÷÷ø

öççè

æ D=

-

®i

ji

h

jiij

hnP

nPmPnmP

i

)(lim

)()(),()(

0

( )mxi ihP

ZP ,exp)(~

1)(~ dxx º

1X

3X

2X

4X

6X 5X

8X7X

-���&� %()�+��"!'��+�*�$#��+�*�$�#�*/.

Page 25: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019������������� D

[Kazuyuki Tanaka Part 5] 25

��

1X

3X

2X

4X

6X 5X

13W

67W

24W

25W346W

568W

8X7X

3W 4W

5W

6W

{ }{ }

{ }0187.0

4393.00082.0

Pr,Pr

Pr

==

=

===

==

PresentPresentPresent

PresentPresent

Dyspnea

DyspneaisTuberculos

DyspneaisTuberculos

XXX

XX

Page 26: 9 9 9 8 () 2 4kazu/ECEI-ExperimentD/2019/S… · April, 2019 0 0 0 D [Kazuyuki Tanaka Part 5] 12 .)*(/-+," { } { } { } 0.7079 0.6471 0.4581 Pr T Pr T, T Pr T T W R W R W = = A A A

April, 2019G)hD�hG�h�����JD

[Kazuyuki Tanaka Part 5] 26

5,�BRTOU

cX\Wgb^`O5,��*V.LN5,�BR&A;��8�-V.LNK(R�B[]_dSR1�

cX\Wgb^`R CR�

<$90, !"F�, %E3=kcX\Wgb^`fiY&Aj HIj2006.!"F�, ����kcX\Wgb^`fiY�?j#�G'��+j2006./���>: cX\Wgb^`fiYR:@2�BR�-, Zea6, 2009.Microsoft6jIntel6jNokia6QPM�.SR47

http://excalibur.brc.uconn.edu/~baynet/http://www.research.microsoft.com/research/dtg/