main.dvi

Size: px

Start display at page:

Download "main.dvi"

としはることじ
5 years ago
Views:

1 1 10,.,,.,,,.,,, 2. 1,, [1].,,,.,,.,,,.. 100,,., [2]. [3,4,5]. [6,7,8,9,10,11]. [12, 13, 14]. 1 kau@statp.is.tohoku.ac.jp

2 CDMA [15, 16] , , ,,. [17, 18]. [19, 20, 21]. [17, 22].,,.,, Neural Information Processing Systems,, Workshop [23].,,.,, ,,.,.,., M, 1, 2,,M. A, 1 A =1., A, A =1 Pr{A =1}. 1, 2,,M

3 , A = a a =1, 2,,M Pr{A = a}. Pr{A = a} Pr{A = a} 0 a =1, 2,,M, M Pr{A = } =1 1 =1. A Pr{A = a} = P a 2 a P a, P a A. 1 P a 0 a =1, 2,,M, M P =1 3 =1., 2, A 1, A 2, M 1 M 2. A 1 = a 1, A 2 = a 2, A 1 = a 1 A 2 = a 2 Pr{A 1 = a 1,A 2 = a 2 } a 1 =1, 2,,M 1 ; a 2 =1, 2,,M 2 4 A 1 A 2. Pr{A 1 = a 1,A 2 = a 2 } A 2 A 1, Pr{A 1 = a 1 } = Pr{A 2 = a 2 } = M 1 M 2 M 2 δ a1, 1 Pr{A 1 = 1,A 2 = 2 } = Pr{A 1 = a 1,A 2 = 2 } 5 1=1 2=1 M 1 1=1 2=1 2=1 M 2 M 1 δ a2, 2 Pr{A 1 = 1,A 2 = 2 } = Pr{A 1 = 1,A 2 = a 2 } 6. δ a,b 1a = b, δ a,b 0a b., Pr{A k = a k } 1=1 k =1, 2 Pr{A 1 = a 1,A 2 = a 2 } A k.. K, A k, M k. A 1 = a 1 A 1 = a 1 A K = a K. Pr{A 1 = a 1,A 2 = a 2,,A K = a K } a k =1, 2,,M k,k=1, 2,,K 7, {A k k =1, 2,,K} {a k k =1, 2,,K} A, a Pr{A = a}. Pr{A = a} Pr{A = a} 0 a k =1, 2,,M k,k=1, 2,,K, Pr{A = } =1 8

4 ., { k k =1, 2,,K} k. M 1 M 2 1=1 2=1 M K K=1 9 A Pr{A = a} = P a 10 a P a, P a A. 8 P a P a 0, P = Pr{A = a} Pr{A k = a k } = δ ak, k Pr{A = } 12 Pr{A k = a k,a k = a k } = δ ak, k δ ak, k Pr{A = } 13 Pr{A k = a k,a k = a k,a k = a k } = δ ak, k δ ak, k δ ak, k Pr{A = } 14. Pr{A 1 = a 1,A 2 = a 2 } Pr{A 1 = a 1 A 2 = a 2 } Pr{A 2 = a 2 A 1 = a 1 }. Pr{A 1 = a 1 A 2 = a 2 } Pr{A 1 = a 1,A 2 = a 2 } Pr{A 2 = a 2 } Pr{A 2 = a 2 A 1 = a 1 } Pr{A 1 = a 1,A 2 = a 2 } Pr{A 1 = a 1 } 15 Pr{A 1 = a 1,A 2 = a 2 } =Pr{A 1 = a 1 A 2 = a 2 }Pr{A 2 = a 2 } =Pr{A 2 = a 2 A 1 = a 1 }Pr{A 1 = a 1 } 16. Pr{A 2 = a 2 }. Pr{A 1 = a 1 A 2 = a 2 } = Pr{A 2 = a 2 A 1 = a 1 }P {A 1 = a 1 } P {A 2 = a 2 } 17 M 1 M 1 Pr{A 2 = a 2 } = Pr{A 1 = a 1,A 2 = a 2 } = Pr{A 2 = a 2 A 1 = a 1 }Pr{A 1 = a 1 } 18 a 1=1 a 1=1 Pr{A 1 = a 1 A 2 = a 2 } = Pr{A 2 = a 2 A 1 = a 1 }Pr{A 1 = a 1 } M 1 Pr{A 2 = a 2 A 1 = a 1 }Pr{A 1 = a 1 } a 1=1 19

5 A 1 = a 1 Pr{A 1 = a 1 } A 1 = a 1 A 2 = a 2 Pr{A 2 = a 2 A 1 = a 1 }, A 2 = a 2 A 1 = a 1 Pr{A 1 = a 1 A 2 = a 2 }. Pr{A 1 = a 1 }, Pr{A 1 = a 1 A 2 = a 2 }.,, A 1 = a 1, A 2 = a 2,,,,., 3 A 1, A 2, A 3. Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3 } 2. Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3 } =Pr{A 3 = a 3 A 1 = a 1,A 2 = a 2 }Pr{A 1 = a 1,A 2 = a 2 } =Pr{A 3 = a 3 A 1 = a 1,A 2 = a 2 }Pr{A 2 = a 2 A 1 = a 1 }Pr{A 1 = a 1 } 20 Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3 } =Pr{A 1 = a 1,A 2 = a 2 A 3 = a 3 }Pr{A 3 = a 3 } 21, Pr{A 1 = a 1,A 2 = a 2 A 3 = a 3 }Pr{A 3 = a 3 } =Pr{A 3 = a 3 A 1 = a 1,A 2 = a 2 }Pr{A 2 = a 2 A 1 = a 1 }Pr{A 1 = a 1 } 22. a 2, Pr{A 1 = a 1 A 3 = a 3 }Pr{A 3 = a 3 } M 2 = Pr{A 3 = a 3 A 1 = a 1,A 2 = a 2 }Pr{A 2 = a 2 A 1 = a 1 }Pr{A 1 = a 1 } 23 a 2=1, Pr{A 1 = a 1 A 3 = a 3 } 1 M 2 = Pr{A 3 = a 3 A 1 = a 1,A 2 = a 2 }Pr{A 2 = a 2 A 1 = a 1 }Pr{A 1 = a 1 } 24 Pr{A 3 = a 3 } a 2=1 Pr{A 3 = a 3 } = M 1 M 2 Pr{A 3 = a 3 A 1 = a 1,A 2 = a 2 }Pr{A 2 = a 2 A 1 = a 1 }Pr{A 1 = a 1 }25 a 1=1a 2=1. 24 A 1 = a 1, A 2 = a 2, A 3 = a 3, A 3 = a 3 A 1 = a 1,..

6 3 A 1, A 2, A 3 Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3 } Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3 } = P a 1,a 2,a 3 26 P a 1,a 2,a 3,, Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3 } =exp Ea 1,a 2,a 3 27 Ea 1,a 2,a 3 lnp a 1,a 2,a =expln, 27 Ea 1,a 2,a exp Ea 1,a 2,a 3 Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3 } = M 1 M 2 M 3 exp E 1, 2, 3 1=1 1=1 1=1., P a 1,a 2,a 3 > 0,. Ea 1,a 2,a 3,..,..,,. 29 3,. K A = {A k k =1, 2,,K} a = {a k k =1, 2,,K} Pr{A = a} = P a m k = a a k P a, V k = a a k m k 2 P a K, OexpK., 100, K = , OK.. Pr{A = a} = P a K P a = P k a k 30 k=1

7 . P k a k P k a k 0a k =1, 2,,M k M k k =1 P k k = 1. A, n. k 1 M l M k K M l M k n k P = P l l n k P k k P l l = n k P k k 31 l=1 l =1 k =1 l=k+1 l =1 k =1 31 m k, V k. Pr{A = a} = P a K W k,k+1 a k,a k+1 k=1 P a = K W k,k+1 k, k+1 k=1. L k 1 k a k R k+1 k a k 2. L k 1 k a k = = = M k 1 k 1 =1 k =1 M k 1 L k 1 k a k = R k+1 k a k = M k 1 k 1 =1 k =1 M k k 1 =1 k =1 R k+1 k a k = = = M k k =1 k+1 =1 M k M k+1 M k+1 k =1 k+1 =1 M 1 M 2 M k 1=1 2=1 M k M k+1 k =1 k+1 =1 k =1 δ ak, k k 1 M K K=1 M k M 1 δ ak, k W k 1,k k 1, k M 1 δ ak, k W k 1,k k 1, k 1 =1 M 2 2 =1 32 W l,l+1 l, l+1 33 l=1 δ ak, k K 1 W l,l+1 l, l+1 34 l=k M 2 1=1 2=1 M k 1 M k 2 k 2 =1 l=1 δ k 1, k 1 k 1 =1 l=1 k 2 W l,l+1 l, l+1 k 2 W l,l+1 l, l+1 M k δ ak, k W k 1,k k 1, k L k 2 k 1 k 1 35 M k M k+1 k =1 k+1 =1 M k+2 δ ak, k W k,k+1 k, k+1 M k+1 δ ak, k W k,k+1 k, k+1 k+1 =1 M k+3 k+2 =1 k+3 =1 M k+2 k+2 =1 M K M K K K=1l=k+1 K δ k+1, k+1 K =1 l=k+1 W l,l+1 l, l+1 W l,l+1 l, l+1 δ ak, k W k,k+1 k, k+1 R k+2 k+1 k L k k+1 a k+1 R k k 1 a k 1. L k k+1 a k+1 = R k k 1 a k 1 = M k M k+1 k =1 k+1 =1 M k 1 k 1 =1 k =1 δ ak+1, k+1 L k 1 k k W k,k+1 k, k+1 37 M k δ ak 1, k 1 L k+1 k k W k 1,k k 1, k 38

8 37 38 L k 1 k a k R k 1 k a k n n k P = M k k L k 1 k k R k+1 k k k =1 M k L k 1 k k R k+1 k k k =1, m k, V k..,. A = {A 1,A 2,,A K } K Ω={1, 2,,K}.., B. A a B Pr{A = a} = P a. B W k,l a k,a l kl B P a = W k,l k, l kl B B = {kl k =1, 2,,K 1, l= k +1} 41, , B kl B kl B\{kl}. B B\{kl}, B, B\{kl} 2, B. B, 40 n. Λ k ka k Λ k ka k = M k M k k =1 k =1 M k n k P = M k n k k =1 M k k =1 δ ak, k W k,k k, k M k k =1 k =1 W k,k k, k Λ l k k l c k. 42 Λ l k k l c k l c k \{k} l c k \{k} Λ l k k Λ l k k k Ω, k c k 43., c k k, c k \{k} c k k 2., Λ k ka k m k, V k. 2, 41 B, c k = {k 1,k+1}, c k \{k±1} = {k 1}.

9 a 30,. b 40 B, , 37, 38, 43 M k k =1,.. b.,,.,,.,,,. 4,,. A a 2 P a Qa, D[P Q] Q Qln 44 P., 2,. i D[P Q] 0 ii P a =Qa D[P Q] =0 x>0 lnx x 1, x =1. ln P Q P Q 1 D[P Q], D[P Q] Q Qln Q 1 P P Q =1 1 = 0 45 P Q D[P Q] 0., D[P Q] =D[Q P ], 2.

10 a Ea, Ea P a = exp Ea exp E T>0 46., Ea, T. 44, D[P Q] =F[Q]+ln exp E 47 F[Q] EQ Qln Q 48. F[Q], P a 46, D[P Q], Qa P a, F[Q] Qa. Q =1 F[Q] { Q = argmin F[Q] Q } Q =1 49,., λ. L[Q] F[Q] λ Q 1 50 L[Q] Q. Qa =exp Ea 1+λ λ, Q 46., Q F[ Q] =minf[q] = ln exp E 52 Q. 5.

11 ,.., Ω={x, y x =1, 2,,M, y =1, 2,,N}. x, y S ±1, exp B s + Cs s x+1,y + Cs s +1 Ω Pr{S = s} = P s exp Ω B + C x+1,y + C = Ω =±1. S m 54 Pr{S = } S s m s x+1,y m x+1,y 0 s m s +1 m +1 0 s = {s x, y Ω}.., s m s x+1,y m x+1,y 0. s s x+1,y s m x+1,y + m s x+1,y m m x+1,y 55 s m s +1 m , m {m } 1. m =tanh B + C m x+1,y + m x 1,y + m +1 + m B x-1,y m x-1,y m -1 m +1 m x+1,y x+1,y -1 m 1: 56.. S P s δ s, P s 57

12 . 2 Ω 1., P s Qs Q s 58 Ω Q s δ s, Qs 59 Qs, 48. F[Q] =F MFA [ {Q x, y Ω} ] Ω + Ω ζq ζ B + C ζq x+1,y ζ+c ζq +1 ζ Q ζlnq ζ 60 Q ζ =1 {Q ζ x, y Ω}, D[P Q] Qs P s {Q ζ x, y Ω} { Q ζ x, y Ω}. { Q ζ = argmin F MFA [{Q x, y Ω}] Q } Q ζ =1x, y Ω 61 D[P Q], F[Q], { Q ζ x, y Ω} {P ζ x, y Ω}. Q ζ =1 λ L [ {Q x, y Ω} ] F MFA [ {Q x, y Ω} ] Ω λ Q ζ 1 62, {Q ζ x, y Ω}, λ, { Q ζ x, y Ω}. exp B + C ζ Qx,y ζ ζ Q ζ = exp B + C x,y c ζ =±1 x,y c ζ =±1 ζ Q x,y ζ 63 ζ c { } x +1,y, x 1,y, x, y +1, x, y 1 64, S Q ζ m Q = ζq ζ 65

13 63 56,.,., B = h/t x, y Ω, C = J/T, h, J, T, Ω M N,., S m x, y, m = m. m, s, h, m. 56 h m =tanh T + 4J T m. 56,, h J T. 66 Step 1: J h. Step 2: T T =8 Step 3: m =0 Step 4: ε<10 6 h µ m, m tanh T + 4J T µ, ε m µ Step 5: T T 0.10, T/J < 0.10, Step 4 h = , J =1, m m : h = , J =1 66 m T. T ,, h = h =0

14 6,. 2,,., [9,10,11,24,25,26]., x, y, F G. Ω {x, y x =1, 2,,M, y =1, 2,,N}. 2. F = {F x, y Ω}, g = {g x, y Ω}., f = {f x, y Ω} g = {g x, y Ω} Pr{F = f G = g = Pr{G = g F = f}pr{f = f} Pr{G = g F = }Pr{F = } 67 = Ω =±1 68. Pr{G = g F = f} f g. Pr{F = f} f., Pr{F = f}, f, f Pr{G = g F = f} g., g f Pr{F = f G = g} 3. Pr{F=f} Pr{G=g F=f } Pr{F=f G=g} 3:., 1 1 2,, 2,., m =0 m =0, h = m 0.

15 g f p 1 1, Pr{G = g F = f} Ω 1 pδ f,g + p1 δ f,g 69 f 5. Pr{F = f} Ω Ω exp 12 αf f x+1,y 2 Ω exp 12 α x+1,y 2 Ω exp 1 2 αf f exp 1 2 α +1 2 f f x, y f x±1,y, x, y±1, f. 2. 4, p =0.2 20% p f = -1 g = -1 p g = 1 1-p f = 1 g = 1 p g = -1 4: 2. a b c 5: α aα =0.25. b α =0.5. c α =1.,,

16 ...,. α 5. 70,.,., 5?., , Pr{F = f G = g}. Pr{F = f G = g} = exp Ef g exp E g 71 Ef g Ω 1 2 βg f αf f x+1,y αf f β 1 1 p 2 ln p 73 f = { f x, y Ω} Pr{F = f G = g} f =argmaxpr{f = f G = g} f 74 Pr{F = f G = g} δ f, Pr{F = G = g} 75,. Pr{F = f G = g} Pr{F = f G = g} f. 75 Pr{F = f G = g}, 2 Ω 1,.. 7, 71 F Pr{F = f G = g}, [9, 10, 11, 24, 25, 26]..

17 , 71. Pr{F = f G = g} = 1 W f Z Ω Ef g = Ω W x+1,y f,f x+1,y W f W x+1,y f x+1,y Ω Ω Ω lnw f Ω W +1 f,f +1 W f W +1 f +1 lnw x+1,y f,f x+1,y lnw f lnw x+1,y f x+1,y lnw +1 f,f +1 lnw f lnw +1 f +1 Z, W ξ W x,y ξ,ξ. W ξ exp 1 2 βξ g 2 78 W x,y ξ,ξ exp 1 2 βξ g βξ g x,y αξ ξ Pr{F = f G = g} =minf[q] 80 Q F[Q] E[Q] S[Q] 81 E[Q] E gq, S[Q] QlnQ 82, Qf Q f δ f, Q 83 Q x,y f,f x,y δ f, δ fx,y, x,y Q 84. Q f Q x,y f,f x,y fs =P s h Pr{F = f G = g}. 72, E[Q] = Q ζlnw ζ Ω Q x+1,y Ω ζ =±1 ζ,ζ lnw x+1,y ζ,ζ Q ζlnw ζ Q +1 Ω ζ =±1 ζ,ζ lnw +1 ζ,ζ Q ζlnw ζ Q x+1,y ζlnw x+1,y ζ Q +1 ζlnw +1 ζ 85

18 ., E[Q] Q f, Q x+1,y f,f x+1,y Q +1 f,f +1., S[Q], F[Q] Q f, Q x+1,y f,f x+1,y Q +1 f,f +1,. 76 Qf Qf = Ω Q f Ω Ω Q x+1,y f,f x+1,y Q f Q x+1,y f x+1,y Q +1 f,f +1 Q f Q +1 f +1 Qf , S[Q] 86. S[Q] S + S x+1,y S S x+1,y + Ω Ω Ω S +1 S S S Q ζln Q ζ 88 S x,y ζ =±1 Q x,y ζ,ζ ln Q x,y ζ,ζ 89 S[Q], S. α =0. α..,, S x+1,y S S x+1,y S +1 S S F[f]. F[Q] F Bethe [{Q,Q x+1,y,q +1 x, y Ω}] Q ζln Q ζ W ζ Ω + Q x+1,y Ω ζ =±1 + ζ,ζ ln Qx+1,y ζ,ζ W x+1,y ζ,ζ Q ζln Q ζ W ζ Q x+1,y ζln Q x+1,yζ W x+1,y ζ Q +1 Ω ζ =±1 ζ,ζ ln Q+1 ζ,ζ W +1 ζ,ζ

19 Q ζln Q ζ W ζ Q +1 ζln Q +1ζ W +1 ζ 90 83, 84. Q ζ = Q x+1,y ζ,ζ = Q x 1,y ζ,ζ ζ =±1 ζ =±1 = Q +1 ζ,ζ = Q 1 ζ,ζ ζ = ±1 91, Q s, Q x+1,y F Bethe [{Q,Q x+1,y. { Q, =arg x+1,y Q, +1 Q {Q,Q x+1,y ζ =±1 ζ =±1 ζ,ζ, Q +1 ζ,ζ 91,Q +1 x, y Ω}] x, y Ω } min,q +1 Ω} { [ {Q F Bethe,Q x+1,y,q +1 } ] x, y Ω Q f = = Q x+1,y f,ζ= Q x 1,y ζ,f Q +1 Q 1 ζ,f, Q ζ = f,ζ= Q x+1,y ζ =±1 ζ,ζ = ζ =±1 Q +1 ζ,ζ =1 } 92. L Bethe [ {Q,Q x+1,y L Bethe [ {Q,Q x+1,y,q +1 [ {Q = F Bethe,Q x+1,y,q +1 Ω Λ x+1,y Λ x 1,y Ω Λ +1 Ω Λ 1 Ω Ω } ] x, y Ω,Q +1 } ] x, y Ω ζ Q ζ Q x+1,y ζ,ζ ζ =±1 ζ Q ζ ζ =±1 ζ Q ζ ζ =±1 ζ Q ζ ν Q ζ 1 ν x+1,y Ω ζ =±1 ζ =±1 Q x+1,y ν +1 Q +1 Ω ζ =±1 x 1,y ζ,ζ Q ζ,ζ Q +1 1 ζ,ζ Q ζ,ζ 1 ζ,ζ 1 x, y Ω }] Q ξ, Q x+1,y 93 ξ,ξ, Q +1 ξ,ξ

20 . 1 W ξ ξ Q ξ = W ζ x,y c exp x,y c exp,y c 1 Λx 1,y c 1 Λx, 94 ζ Q x,y ξ,ξ = W x,y ζ =±1 ξ,ξ exp Λ x,y W x,y ζ,ζ exp ξ Λ x,y exp Λ x,y ξ, ζ exp Λ x,y ζ 95 {ν,ν x,y x, y Ω, x,y c }. { Λ x,y ζ,,y Λx ζ x, y Ω, x,y c,ζ {±1} } 91. exp Λ x,y ξ = x,y c \{x,y } M x,y ξ 96, M x,y ξ, Q ξ, Q x+1,y ξ,ξ, Q +1 ξ,ξ. Q x,y ξ,ξ = Q ξ = W x,y ξ,ξ ζ =±1 M x,y ξ = W x,y ζ,ζ W ξ W ζ x,y c \{x,y } M x,y x,y c ξ x,y c M x,y M x,y x,y c \{x,y } W x,y ξ,ζ W ξ ζ =±1 W x,y ζ,ζ W ζ ξ M x,y ζ ζ, 97 x,y c x,y \{} M x,y x,y x,y c x,y \{} x,y c x,y \{} M x,y x,y c x,y \{} ζ M x,y x,y ξ M x,y x,y ζ, x,y ζ c { } x +1,y, x 1,y, x, y +1, x, y { M x,y ξ }.. 97 x, y M x,y x,y c. 6.

21 98 f x+1,y ζ,ζ, x, y x +1,y M x,y x,y c \{x +1,y}., x +1,y x, y M x,y x,y c \{x, y}. 7., 102 x,y =x +1,y, x, y x +1,y M x,y x,y c \{x +1,y}, x +1,y M x,y M +1 x-1,y M x-1,y M -1 M x+1,y -1 x+1,y 6: 97 f ζ. a x-1,y +1 M x-1,y M -1-1 M +1 M x+1,y-1 x+1,y x+1,y x+1,y+1 M x+1,y+1 x+1,y M x+2,y x+1,y x+1,y-1 x+2,y b M x-1,y x-1,y+1 x-1,y +2 M x-1,y M x+1,y-1 M M x+1,y+1 +1 M x+1,y x+1,y+1 x+1,y x+1,y-1 7: 98 f x+1,y ζ,ζ f +1 ζ,ζ. a x,y =x +1,y. b x,y =x, y , M x,y x,y ζ ζ 0, 1 M x,y x,y x,y exp λ 1 = 2coshλ x,y, Mx,y x,y x,y expλ 1 = 2coshλ x,y 101, 144. λ x,y tanhαtanh =arctanh βh + x,y c \{x,y } λ x,y ,. f = { f x, y Ω} 74. Pr{F = f G = g} 97-99, x, y Q f

22 a M x-1,y x-1,y M -1 c x-1,y M x-1,y M M +1 M x+1,y M +1 M x+1,y x+1,y x+1,y b M x-1,y x-1,y M -1 d x-1,y M x-1,y M M +1 M x+1,y M +1 M x+1,y x+1,y x+1,y : 102. a x,y =x +1,y. b x,y = x 1,y. c x,y =x, y + 1. d x,y =x, y 1.. α p. Step 1: f = {f x, y Ω}, g = {g x, y Ω} p, α, R,. Step 2: M x,y ξ =0x, y Ω, x,y c, ξ {±1}, T 1+R. Step 3: T T, r 0. Step 4: r r +1,a x,y ζ M x,y ζ x, y Ω, x,y c, ζ {±1}. { } Step 5: M x+1,y ξ, M +1 ξ x, y Ω,ξ {±1}. x,y 1/T W ξ,ζ a x,y W ξ x,y ζ M x,y x,y c x,y \{} ξ x,y 1/T W ζ,ζ W ζ a x,y x,y ζ ζ =±1 x,y c x,y \{} x,y c 103 Step 6: 1 MN Ω x,y c a x,y ζ M x,y ζ < Step 4, Step 7. Step 7: T =1 Step 3, f = { f x, y Ω} f arg max W ξ x M,y ξ x,y c 105, a α =0.5,

. α, p. 10a Mandrill., 10 2,. p, α, 70,.. a b c 9: 2 a α =0.5, 70 2. b p =0.2, 69, 2. c p =0.2, α =0.

23 . α, p. 10a Mandrill., 10 2,. p, α, 70,.. a b c 9: 2 a α =0.5, b p =0.2, 69, 2. c p =0.2, α =0.5 a b c 10: 2 a. b p =0.2. c p =0.2, α =0.35 p α, p α. 70 α α p α, p, [9, 11]. α, p =max Pr{G = g F = }Pr{F = } 106 α,p

a b c 11: a. b N [0, 30 2 ]. c σ = 30, α =0.0005 Pr{G = g} Pr{G = g F = }Pr{F = },. 0, 1,. 0, σ 2 = 1 2β N [0,σ2 ]., Pr{G = g F = f} Ω 1 exp 1 2πσ 2σ 2 f g 2 107. σ =20 11.

24 a b c 11: a. b N [0, 30 2 ]. c σ = 30, α = Pr{G = g} Pr{G = g F = }Pr{F = },. 0, 1,. 0, σ 2 = 1 2β N [0,σ2 ]., Pr{G = g F = f} Ω 1 exp 1 2πσ 2σ 2 f g σ = C URL: http : // kau/smapip-kaukau/. 8.. Pearl [3]., [2].,..., 4, 12,. K. Murphy http : // murphyk/bayes/bayes.html. 4 [ ], [ ], [ ], [ ], true T false F 2

25 A C, A S, A R, A W. Cloudy A =True or False C Sprinklet A =True or False S Rain A =True or False R Wet Grass A =True or False W 12: 4. Pr{A C = a C,A S = a S,A R = a R,A W = a W } =Pr{A W = a W A C = a C,A S = a S,A R = a R }Pr{A C = a C,A S = a S,A R = a R } =Pr{A W = a W A C = a C,A S = a S,A R = a R }Pr{A R = a R A C = a C,A S = a S } Pr{A C = a C,A S = a S } =Pr{A W = a W A C = a C,A S = a S,A R = a R }Pr{A R = a R A C = a C,A S = a S } Pr{A S = a S A C = a C }Pr{A C = a C } 108.,... Pr{A W = a W A C = a C,A S = a S,A R = a R } A C, Pr{A R = a R A C = a C,A S = a S } S Pr{A W = a W A C = a C,A S = a S,A R = a R } =Pr{A W = a W A S = a S,A R = a R } Pr{A R = a R A C = a C,A S = a S } =Pr{A R = a R A C = a C } 109., Pr{A C = a C,A S = a S,A R = a R,A W = a W }. Pr{A C = a C,A S = a S,A R = a R,A W = a W } =Pr{A W = a A A S = a S,A R = a R }Pr{A R = a R A C = a C } Pr{A S = a S A C = a C }Pr{A C = a C } Pr{A W = a A A S = a S,A R = a R },Pr{A R = a R A C = a C },Pr{A S = a S A C = a C },Pr{A C = a C } 1.,

26 1: 110 Pr{A W = a A A S = a S,A R = a R },Pr{A R = a R A C = a C },Pr{A S = a S A C = a C },Pr{A C = a C }. T true, F false. a C Pr{A C = a C } T 0.5 F 0.5 a S Pr{A S = a S A C =T} Pr{A S = a S A C =F} T F a R Pr{A R = a R A C =T} Pr{A R = a R A C =F} T F a W Pr{A W = a W A S =T,A R =T} Pr{A W = a W A S =T,A R =F} T F a W Pr{A W = a W A S =F,A R =T} Pr{A W = a W A S =F,A R =F} T F , Pr{A R = a R A W = a W } = Pr{A R = a R,A W = a W } Pr{A W = a W } Pr{A S = a S A W = a W } = Pr{A S = a S,A W = a W } Pr{A W = a W } Pr{A R = a R,A W = a W } = Pr{A S = a S,A W = a W } = Pr{A W = a W } = a C=T,Fa S=T,F a C=T,Fa R=T,F a C=T,Fa S=T,Fa R=T,F Pr{A C = a C,A S = a S,A R = a R,A W = a W } 112 Pr{A C = a C,A S = a S,A R = a R,A W = a W } 113 Pr{A C = a C,A S = a S,A R = a R,A W = a W } 114, Pr{A S =true,a W =ture=0.2781, Pr{A R =true,a W =true=0.4581, Pr{A W = true = Pr{A R =true A W =true} = Pr{a R =true,a W =true} Pr{A W =true} Pr{A S =true A W =true} = Pr{A S =true,a W =true} Pr{A W =true} = = = =

27 , Asia, [4]. Visit to Asia: A 1 Visit : A 1 = -1 No Visit : A 1 = +1 Smoking: A 2 Smoker : A 2 = -1 Non Smoker : A 2 = +1 Tuberculosis: A 3 Present : A 3 = -1 Absent : A 3 = +1 Lung Cancer: A 4 Present : A 4 = -1 Absent : A 4 = +1 Bronchitis: A 5 Present : A 5 = -1 Absent : A 5 = +1 Tuberculosis or Cancer: A 6 True : A 6 = -1 False : A 6 = +1 X-ray Result: A 7 Abnormal : A 7 = -1 Normal: A 7 = +1 Dyspnea: A 8 Present : A 8 = -1 Absent : A 8 = +1 13: A 1 A 2 A3 A4 A5 A 6 A7 A8 14: ,. Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4,A 5 = a 5,A 6 = a 6,A 7 = a 7,A 8 = a 8 } =Pr{A 7 = a 7,A 8 = a 8 A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4,A 5 = a 5,A 6 = a 6 } Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4,A 5 = a 5,A 6 = a 6 }

28 =Pr{A 7 = a 7,A 8 = a 8 A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4,A 5 = a 5,A 6 = a 6 } Pr{A 5 = a 5,A 6 = a 6 A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4 } Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4 } =Pr{A 7 = a 7,A 8 = a 8 A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4,A 5 = a 5,A 6 = a 6 } Pr{A 5 = a 5,A 6 = a 6 A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4 } Pr{A 3 = a 3,A 4 = a 4 A 1 = a 1,A 2 = a 2 } Pr{A 1 = a 1,A 2 = a 2 } Pr{A 7 = a 7,A 8 = a 8 A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4,A 5 = a 5,A 6 = a 6 } = V 56 8 a 8 a 5,a 6 V 6 7 a 7 a 6 Pr{A 5 = a 5,A 6 = a 6 A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4 } = V 2 5 a 5 a 2 V 34 6 a 6 a 3,a 4 Pr{A 3 = a 3,A 4 = a 4 A 1 = a 1,A 2 = a 2 } = V 1 3 a 3 a 1 V 2 4 a 4 a 2 Pr{A 1 = a 1,A 2 = a 2 } = V 1 a 1 V 2 a Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4,A 5 = a 5,A 6 = a 6,A 7 = a 7,A 8 = a 8 } = V 56 8 a 8 a 5,a 6 V 6 7 a 7 a 6 V 34 6 a 6 a 3,a 4 V 2 5 a 5 a 2 V 2 4 a 4 a 2 V 1 3 a 3 a 1 V 2 a 2 V 1 a 1 118, W 568 a 5,a 6,a 8 =V 56 8 a 8 a 5,a 6 W 67 a 6,a 7 =V 6 7 a 7 a 6 W 346 a 3,a 4,a 6 =V 34 6 a 6 a 3,a 4 W 25 a 2,a 5 =V 2 5 a 5 a 2 V 2 a 2 W 24 a 2,a 4 =V 2 4 a 4 a 2 V 2 a 2 W 13 a 1,a 3 =V 1 3 a 3 a 1 V 1 a 1 W 2 a 2 =V 2 a 2 W 3 a 3 =W 4 a 4 =W 5 a 5 =W 6 a 6 =1 119, 118 Pr{A 1 = a 1,A 2 = a 2,A 3 = a 3,A 4 = a 4,A 5 = a 5,A 6 = a 6,A 7 = a 7,A 8 = a 8 } = 1 W 568 a 5,a 6,a 8 W 67 a 6,a 7 W 346 a 3,a 4,a 6 W 25 a 2,a 5 W 24 a 2,a 4 W 13 a 1,a 3 Z W 2 a W 3 a 3 W 4 a 4 W 5 a 5 W 6 a 6., Z. { } C 568, 67, 346, 25, 24, 13, 2, 3, 4, 5, { µ568 = µ67 = µ346 = µ25 = µ25 = µ24 = µ13 = 1 µ2 = 2, µ3 = µ4 = µ5 = µ6 = 1, 122 } } i α i α A α {A i, a α {a i α C 123

29 . C, 2 < 24, 2 < 25, 3 < 13, 3 < 346, 4 < 24, 4 < 346, 5 < 25, 5 < 568, 6 < 346, 6 < 67, 6 < 568, 124, C. i<α i 2 α. i α. α. 118 W α a α µα α C Pr{A = a} = 125 W α α µα α C. W α a α µα Eα: 8 i=1 i=±1 126 Ea α Cµαln W α a α 127, Pr{A = a} Pr{A = a} = exp Ea exp E 128, Ea. 3 Pr{A = a}. Pr{A = a} =minf[q] 129 Q F[Q] E[Q] S[Q] 130 E[Q] EQ, S[Q] QlnQ 131, Qf Q α a α δ ai, i Q 132 {i i α}, α Q α α =1α C 133

30 Q α a α = γ {i i α} δ ai, i Q γ γ α<γ C 134 α {i i α} i=±1 135., 132 E[Q] Q α a α α C E[Q] = α Cµα α Q α α ln W α α 136. Qa Qa = Q 568a 5,a 6,a 8 Q 67 a 6,a 7 Q 346 a 3,a 4,a 6 Q 25 a 2,a 5 Q 24 a 2,a 4 Q 13 a 1,a 3 Q 2 a 2 2 Q 3 a 3 Q 4 a 4 Q 5 a 5 Q 6 a Qa = α CQ α a α µα 138, Q α a α = Q α a α Pr{A α = a α }. S[Q] S[Q] = α Cµα α Q α α ln Q α α F[Q] F CVM [{Q α α C}] µα Q α α ln Q α α α C Q α α α 140 F[Q] {Q α α C} F CVM [{Q α α C}]. { Q { α α C} arg min F CVM [{Q α α C}] Q α α =1α C, {Q α α C} α Q α a α = } δ ai, i Q γ γ α<γ C. 141 γ {i i α},,,, { Q α α C}. Q i a i = W i a i i W i i {α α c i} {α α c i} M α i a i M α i i, 142

31 Q α a α = W α a α M γ i a i α W α α {i i α}{γ γ c i\{α}} {i i α}{γ γ c i\{α}} M γ i i, 143 M α i a i = δ ai, i α α Wα α W i i Wα α W i i {j j<α}{γ γ c j\{α}} {j j<α}{γ γ c j \{α}} M γ j j M γ j j. 144?.,.. i C. ii Ω C Ω C\Ω γ γ γ<γ γ>γ. iii µγ µγ = 1 γ C\Ω, µi = 1 µγ i Ω. {γ γ>i, γ C} iv Pr{A = a} V 56 8 a 8 a 5,a 6, V 34 6 a 6 a 3,a 4, V 6 7 a 7 a 6 V 2 5 a 5 a 2, V 2 4 a 4 a 2, V 1 3 a 3 a 1, V 1 a 1, V 2 a 2 2, Q i +1, Q i P i +1,P i 1 3. Q 25 a 2,a 5, Q36 a 3,a 6 Pr{A 2 = a 2 A 5 = a 5 } Q 25 a 2,a 5 / Q 5 a 5, Pr{A 3 = a 3 A 6 = a 6 } Q 36 a 3,a 6 / Q 6 a , [22, 27].,,, 1 8,,.,,. [28]. 9,.

32 2: 118 V 56 8 a 8 a 5,a 6, V 34 6 a 6 a 3,a 4, V 6 7 a 7 a 6 V 2 5 a 5 a 2, V 2 4 a 4 a 2, V 1 3 a 3 a 1, V 1 a 1, V 2 a 2. a 1 V 1 a 2 V a 1 a 3 V 1 3 a 2 a 4 V a 2 a 5 V 2 5 a 6 a 7 V a 3 a 4 a 6 V 34 6 a 5 a 6 a 8 V [22]. if-then-else,..,,, URL:

33 3: Q i +1, Q i P i +1,P i 1 i Q i +1, Q i 1 P i +1,P i , , , , , , , , , , , , , , , , : Q 25 a 2,a 5 Pr{A 2 = a 2 A 5 = a 5 } Q 25 a 2,a 5 / Q 5 a 5 a 2 a 5 Q25 a 2,a 5 Pr{A 2 = a 2 A 5 = a 5 } [1],,,, vol.42, no.8, pp , [2],,, vol.17, no.5, pp , [3] J. Pearl: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, [4] S. L. Lauriten and D. J. Spiegelhalter, Local computations with probabilities on graphical structures and their application to expert systems, J. Royal Statistics Society B, vol.50, no.2, pp , [5],, , No.489, [6] R. Chellappa and A. Jain eds, Markov Random Fields: Theory and Applications, Academic Press, New York, : Q 36 a 3,a 6 Pr{A 3 = a 3 A 6 = a 6 } Q 36 a 3,a 6 / Q 6 a 6 a 3 a 6 Q36 a 3,a 6 Pr{A 3 = a 3 A 6 = a 6 }

34 [7] S. Z. Li, Markov Random Field Modeling in Computer Vision, Springer-Verlag, Tokyo, [8],,, vol.54, no.1, pp.25-33, [9] K. Tanaka, Statistical-mechanical approach to image processing, Journal of Physics A: Mathematical and General, vol.35, no.20, pp.r81-r150, [10],,, vol.42, no.8, pp , [11],, , No.489, pp.15-21, [12],,, vol.58, no.4, pp , [13] Y. Kabashima, T. Murayama and D. Saad, Cryptographical Properties of Tsing Spin Systems, Physical Peview Letters, vol.84, no.9, pp , [14] T. Murayama, Statistical mechanics of data compression theorem, Journal of Physics A: Mathematical and General, vol.35, no.8, pp.l95-l100, [15],,, vol.56, no.9, pp , [16] T. Tanaka, A statistical-mechanics approach to large-system analysis of CDMA multiuser detectors, IEEE Transactions on Information Theory, vol.48, no.11, pp , [17] M. Opper and D. Saad eds, Advanced Mean Field Methods Theory and Practice, MIT Press, [18], /,, [19] /,, [20],,,, [21] H. Nishimori, Statistical Physics of Spin Glasses and Information Processing: An Introduction, Oxford University Press, Oxford, [22] J. S. Yedidia, W. T. Freeman and Y. Weiss: Generalied belief propagation, Advances in Neural Information Processing Systems, vol.13, pp , 2001 Cambridge, MA: MIT Press. [23],, M. Welling, NIPS*2002 post conference workshop on Propagation Algorithms on Graphs with Cycles: Theory and Applications December 13th and 14th, 2002, Whistler, Canada, URL: [24],, A, vol.j83-a, no.10, pp , [25] K. Tanaka, Maximum Marginal Likelihood Estimation and Constrained Optimiation in Image Restoration,, vol.16, no.2, pp , [26] K. Tanaka, J. Inoue and D. M. Titterington, Probabilistic image processing by means of Bethe approximation for Q-Ising model, J. Phys. A: Math. & Gen., vol.36, no.43, pp , [27] H. J. Kappen and W. Wiegerinck: Novel iteration schemes for the cluster variation method, Advances in Neural Information Processing System, vol.14, pp , 2002 Cambridge, MA: MIT Press. [28] K. Tanaka, Probabilistic Inference by means of Cluster Variation Method and Linear Response Theory, IEICE Transactions on Information and Systems, vol.e86-d, no.7, pp , 2003.

& 3 3 ' ' (., (Pixel), (Light Intensity) (Random Variable). (Joint Probability). V., V = {,,, V }. i x i x = (x, x,, x V ) T. x i i (State Variable),

& 3 3 ' ' (., (Pixel), (Light Intensity) (Random Variable). (Joint Probability). V., V = {,,, V }. i x i x = (x, x,, x V ) T. x i i (State Variable), .... Deeping and Expansion of Large-Scale Random Fields and Probabilistic Image Processing Kazuyuki Tanaka The mathematical frameworks of probabilistic image processing are formulated by means of Markov