Kullback-Leibler

Transcription

1 Kullback-Leibler Saov : Ver Kullback-Leibler Kullback-Leibler Kullback-Leibler max-plus Laplace KL Kullback-Leibler Poisso Kullback-Leibler URL Ver Ver Ψβ = log r q e βi i Legedre Ver Boltzma e βuei. 8 3 Ver Ver.0.9. Stirlig Ver Pearso. 9 2 Ver Ver Poisso Kullback-Leibler Ver.0.22a:. 9 4 Ver : Stirlig. allowdisplaybreaks Ver : Mcmilla Ver.0.24a73 : 0.

2 2 2 Boltzma Boltzma Saov Saov Saov Saov Saov : Kullback-Leibler : Jese Kullback-Leibler L Pithagoria theorem : Cramér Cramér Cramér Saov Cramér Ψβ = log r e β i Legedre :? : Massieu Réyi Tsallis Tsallis Tsallis Csiszár f-divergece

3 3 9 : Mcmilla 7 0. Mcmilla : Stirlig Stirlig.. Kullback-Leibler Boltzma exp ν β νf ν k.,. 2. 3, 4.. [] Csiszar, Imre. A simple proof of Saov s theorem. Bull Braz Math Soc, New Series 374, , [2] Csiszár, Imre. Axiomatic characterizatios of iformatio measures. Etropy, 2008, 0, [3] Cover, M. Thomas ad Thomas, Joy A. Elemets of Iformatio Theory. Secod Editio, Joh Wiley & Sos, Ic., 2006, xxiii+748 pages. Google, Kullback-Leibler 2 2, Kullback-Leibler. Kullback-Leibler Kullback-Leibler., Kullback-Leibler. Saov., Boltzma, Saov. Boltzma e βe i. 2.

4 4. Kullback-Leibler [4] Dembo, Amir ad Zeitoui, Ofer. Large Deviatios Techiques ad Applicatios. Stochastic Modellig ad Applied Probability formerly: Applicatios of Mathematics, 38, Secod Editio, Spriger, 998, 396 pages. Google [5] Ellis, Richard, S. The theory of large deviatios ad applicatios to statistical mechaics. Lecture otes for École de Physique Les Houches, August 5 8, 2008, 23 pages. [6]. R Woderful R. 206/9/8, [7] Saov, I. N. O the probability of large deviatios of radom variables. Eglish traslatio of Matematicheskii Sborik, 4284:, pp. 44. Istitute of Statistics Mimeograph Series No. 92, March, pdf [8] Suyari, Hiroki. Mathematical structure derived from the q-multiomial coefficiet i Tsallis statistics. arxiv:cod-mat/ [9] Suyari, Hiroki ad Scarfoe, Atoio Maria. α-divergece derived as the geeralized rate fuctio i Tsallis statistics., vol. 4, o. 38, IT204-6, pp , [0]. I, II., 2008/2, [] Tim va Erve ad Peter Harremoës. Réyi divergece ad Kullback-Leibler divergece. arxiv: [2] Ramo va Hadel. Lecture 3: Saov s theorem. Stochastic Aalytic Semiar Priceto Uiversity, Blog Article, 0 October [3] Vasicek, Oldrich Alfoso. A coditioal law of large umbers. A. Probab., Volume 8, Number 980, Kullback-Leibler Stirlig Kullback-Leibler.

5 .. 5 Stirlig Stirlig., x = + y = + y/! = Γ + = e x x dx = e e y + y/ dy 0, loge y + y/ = y + log + y/ = y + y/ y 2 /2 + O / = y 2 /2 + O /, 3, e y + y/ dy Stirlig : e y2 /2 dy = 2π.! = e 2π + o, log! = log + 2 log + log 2π + o.. 0, r =. i. q = q,..., q r., i k i k i. i k i /., i. k i /,. 2 Boltzma. q = q,..., q r, k /,..., k r /,. i k i, r k i =! k! k r! qk q k r r, 0. p i 0, r p i =. i k i / p i, p i. 3 e y + y/ y < 0, y > 0. Lebesgue,.

6 6. Kullback-Leibler.2 p i., k i log k i = p i + Olog = p i + O,. logk i / = log p i + Olog / 4. Stirlig r k i = log! = log + Olog = k i log k i + Olog, log k i! = k i log k i k i + Olog k i = k i log k i k i + Olog, log q k i i = k i log. k i. :! log k! k r! qk q k k r i r = log k i log + Olog = p i log p i log + Olog = p i log p i + Olog Kullback-Leibler.2 Dp q = p i log p i :! log k! k r! qk q k r r = Dp q + Olog. k i / p i. Dp q Kullback-Leibler Kullback-Leibler divergece. Kullback-Leibler Sp q = Dp q = 4 Taylor log + x = x x 2 /2 + x 3 /3 x 4 /4 +. p i log p i

7 .4. Kullback-Leibler 7. p i. : p i = exp Dp q + Olog. Dp q > 0, Olog Dp q, exp Dp q..4 Kullback-Leibler Kullback-Leibler Dp q = r p i logp i / fx = x log x, Dp q = r fp i/, Dp q p = p,..., p r fx = x log x. f x = log x +, f x = /x > 0 fx. fx. fx f + f x = x x =. pi pi Dp q = f = 0, p i = i =,..., r. fx, Dp q p. Kullback-Leibler 0, 0 p i., p i, Dp q > 0, p i 0., k i /.. Kullback-Leibler p i. Kullback- Leibler. Kullback-Leibler, Kullback-Leibler., p i, Kullback-Leibler,. Kullback-Leibler., Kullback-Leibler. Kullback-Leibler, p i...

8 8. Kullback-Leibler. B A., A B, B A..,,...5 r = 2, q = q, q 2 = q.., p = p, p 2 = p, Kullback-Leibler : Dp q = p log p q + p log p q. p = q 0, p q. Kullback- Leibler q p. p exp Dp q + Olog., p, Dp q p 0... a.?, q. 0 a < q, a., 0 a < q, a, q. q < a.,, a 0. a p. Kullback-Leibler. p a Dp q p = a., p = a. q < a, a, a. : lim log { Dq q = 0 0 a q, q k q k = if Dp q = k p a k/ a Da q q < a., q k q k = exp if Dp q + o. k p a k/ a a. Dp q.

9 .6. max-plus Laplace 9.6 max-plus Laplace a, b a, b max{a, b}, a, b a + b semirig, semifield max-plus. max-plus tropical mathematics.,.., max 0, +, 0. log., : lim logea + e b = max{a, b}, lim logea e b = a + b... a b, b a 0, e b a, logea + e b = log e a + e b a = a + log + e b a a.. : lim log expa i + Olog = max{a,..., a r }. expa i + Olog / a i., expa i + Olog = exp max{a,..., a r } + o Laplace.. : β exp fx + Olog dx = exp if fx + o α x β α.. fx α < x = x 0 < β, f x 0 > 0, β e fx gx dx = e fx0 2π gx 0 + o. f x 0 α Laplace.

10 0. Kullback-Leibler.7 KL,. 0, r =, a, b i r b i = a,. lim N Na log p i = b i a = Nb i Na Na! Nb! Nb r! qnb q Nb r r = p i log p i. Kullback-Leibler. /Na Na! lim N Nb! Nb r! qnb qr Nbr = p /q p pr /q r pr... N Na log Na! Nb! Nb r! qnb q Nb r r = Na Nb i log k log k + Nb i log Na k= k= = Na log k Nb i Na Na log k Na + Nb i log k= = Na Na k= 0 log k Na log x dx = [x log x x] 0 pi 0 k= Na Nb i k= log x dx + log k Na + p i log p i log [x log x x] p i 0 + p i log = p i log p i. 2 Na logna r Nb i logna = 0.. : N Na p i = b i /a /Na p /q p pr /q r pr..8 Kullback-Leibler

11 .8. Kullback-Leibler, p i > 0, r = r p i =, k i r k i =.! k! k r! qk q k r r, k i = p i + ε i, p i = + x i /, ε i = o 5,.!, k i! Stirlig! = e 2π + O/, k i! = k k i i e k i 2πki + O/! k! k r! qk q kr r = e 2π + O/. k k e k 2πk k k r r e k r 2πkr e e k i r k i =. = k k r k i = k i /! k! k r! qk q kr r = k k / kr / q q r kr + O/ 2π r k / k r /.. Kullback-Leibler 6. k i = p i + ε i, p i = + x i, ε i = o 7. r x i = 0, r k i = r = r p i =, r ε i = 0, k i = ki / log + x i + ε i qi = + o, k i = + x i + ε i, ki = + x i + ε i log + x i + ε i qi q i = + x i + ε i = x i ε i + x2 i 2 + o, r x i + ε i = 0,! k! k r! qk q k r r = x i qi + ε i + 2 xi + ε 2 i + o qi x 2 i exp 2 q i + o. 2π r q q r 5 ε i /2 ε i = O ε i /2.

12 2. Kullback-Leibler dk i = dx i, x 2 i exp! 2 q k! k r! qk qr kr i dk dk r = dx dx r + o. 2π r q q r r k i =, r x i = 0.. KL Kullback-Leibler,, k i = p i + o = p i + o o. k i = p i + o = p i + o ki ki / pi + o log = p i + o log log = o! log k! k r! qk qr kr = Dp q Kullback-Leibler. = p i log p i + o p i log p i + o = Dp q + o. KL Leibler k i = p i Kullback- k k / kr / kr q q r /. k i = p i + o, Kullback-Leibler /.. 2 Kullback-Leibler. Kullback-Leibler Dp q p i = 0. Dp q Taylor 2.. r x i = 0 Dp q p i = + x i /, x i Dp q = x 2 i + x i log + x i qi

13 .8. Kullback-Leibler 3 = = r x i = 0, + xi x i qi xi + x2 i + o. 2 Dp q = 2 x2 i 2q 2 i x 2 i + o. + o p i Kullback-Leibler Taylor 3. Pearso 2 i i 2 i = k i 2 Pearso 2. k i = + x i + o, k i 2 = x 2 i + o., Pearso 2 x i r x2 i /. exp /2 r x2 i /.,, Pearso 2 2. r r x i = 0 r, 2 r. 2. Z,..., Z s. Z i fz i E[fZ i ] = R fz i e z2 i /2 2π dz i. Y = s Z2 i s. :, E[fY ] = s E[fY ] = cost. f R s z 2 i 0 fy e y/2 y s/2 dy. Γs/22s/2 e s z2 i /2 dz dz s

14 4. Kullback-Leibler = cost. 0 fr 2 e r2 /2 r s dr = cost. 0 fye y/2 y s/2 dy. 2 r 2 = s z2 i, r. s r s., cost.. 3 y = r 2. cost.. A = [a ij ] s, A = [b ij ]. X,..., X s s exp s 2 i,j= b ijx i x j det2πa., 2 Y = s b ij X i X j i,j= s., s 2 s. 2. Y s.. A U = [u ij ] D = diagα,..., α s A = UDU = UDU T. A α i D = diag α,..., α s, C = U D, A = CC T. A = C T C Y = s b ij X i X j = X T A X = C X T C X. i,j= X X i i., Z = [Z i ] Z = C X, Y = Z T Z = s Zi 2. Z i, Y s 2. Z i. Z i E[ZZ T ], X i E[XX T ] = A = CC T E[ZZ T ] = E[C XC X T ] = C E[XX T ]C T = C CC T C T = E..

15 .9. Poisso Kullback-Leibler 5.9 Poisso Kullback-Leibler [6] p.80 Poisso λ > 0. K λ Poisso, k = 0,, 2,... K = k λ λk P K = k = e k!. P K = k = e λ k=0 k=0 λ k k! = e λ e λ =. Poisso. 0 < q <!/k! k! q k q k q = λ/, k k + λ λ k = k! λ λ k k! A, 0/ / k / A = λ/ k., q = λ Poisso. Poisso P K = k k. Poisso E[K] E[K 2 ] E[K] 2 : E[K] = kp K = k = e λ k=0 E[KK ] = e λ k=2 k=0 k λk k! = e λ k= λ k k 2! = e λ λ 2 e λ = λ 2 E[K 2 ] E[K] 2 = E[K] + λ 2 λ 2 = E[K] = λ. Poisso E[e tk : E[e tk ] = e λ k=0 tk λk e k! = e λ e λet = e λet. λ k k! = e λ λe λ = λ, Poisso λ 8.,, K λ λ Poisso, K λ λ/ λ 8 X, Y λ X, λ Y Poisso, X + Y λ X + λ Y Poisso.

16 6. Kullback-Leibler λ., K i,λi λ i Poisso, s K i,λi λ i 2 λ i λ i s 2. O i = K i,λi, E i = λ i, s O i E i 2 E i, Pearso. Poisso Poisso Stirlig. fx R. K λ λ Poisso, X λ = K λ λ/ λ. E[fX λ ] = e λ = = = k=0 f x Z λ / λ x Z λ / λ x Z λ / λ k λ λ λ k k! = x Z λ / λ λ x fx λλ+ e λ λ + λ x! λ λ+ λ x e λ + O/λ fx λ + λ x λ+ λ x e λ λ x 2πλ + λ x fx + x λ+ λ x e λ x + O/λ λ 2π + x/ λ λ /2 fx e x2 + O/ λ 2π + x/ λ fx e x2 /2 2π dx λ. λ 3 Stirlig. 5 λ + λ x log + x + λ x = λ + x λ x λ x2 λ = λ x + x2 2 x2 + λ x + O. Poisso λ λ = x2 2 + O λ 2λ + O λ + λ x λ > 0, r =.! k! k r! qk q k r r =! r q k i i k i!

17 .9. Poisso Kullback-Leibler 7., k i r k i =., λ i =,! r q k i i k i! =! e r e q q i i k i =! k i! e r e λ λk i i i k i! λ i = Poisso., Poisso r k i =. Poisso KL Poisso Kullback-Leibler. λ Poisso Stirlig, λ λk e k! = e λ λ k k k k k e k 2πk + O/k = k λ + O/k e. λ 2πk. k k log e k λ = k log k λ λ + k λ. Kullback-Leibler., k = p, λ = q k log k λ + k λ = p log p q p q. Kullback-Leibler. p logp/q p = q, p q p = q. Poisso k logk λ + k λ k = λ Taylor k log k λ + k λ = k λ 2 + k λ 3 k λ λ 2 3 λ λ 3., Taylor k = λ + λ x, λ. k = λ + λ x k log k λ + k λ x2 2 λ., k = λ + λ x, k/λ = + x/ λ, dk = λ dx k λ λk k e k! dk = k λ + O/k e dk e x2 dx λ λ 2πk 2π. Poisso, Poisso KL Taylor. k logk/λ + k λ k = λ Taylor k λ 2 /λ O = = k, E = = λ k λ 2 λ = 2 = O E2 E. Poisso, E, O E 2 /E..

18 8. Kullback-Leibler Poisso KL Poisso Kullback-Leibler, Poisso Kullback-Leibler. Poisso r e λ λk i r ki i i ki = e k i λ i + O/k i. k i! 2πki, k i = p i, λ i = r ki ki log e k i λ i = k i log k i + k i λ i λ i λ i = p i log p i p i. r p i = r r ki ki log e k i λ i λ i λ i = p i log p i. Kullback-Leibler. Kullback-Leibler. Poisso r k i =. Poisso, > 0, r =, λ i =, r k i =,! k! k r! qk q k r r =! e r e λ λk i i i k i! = r 2π + O/ k i /λ i k i e k i λ i 2πki + O/k i. Stirlig.. k i = p i,! k! k r! qk q kr r = + O/ 2π r r p i r pi pi.,! log k! k r! qk q k r r = p i log p i + Olog. Kullback-Leibler Dp q = r p i logp i /. Poisso.! k! k r! qk q k r r = 2π + O/ r k i /λ i k i e k i λ i 2πki + O/k i,

19 9 logk i /λ i k i e k i λ i = k i logk i /λ i + k i λ i = k i λ i 2 + k i λ i 3 2 λ i 6 λ 2 i, k i = λ i + λ i x i = + x i, x i = k i λ i λi = k i qi, dk i = dx i! k! k r! qk qr kr dk dk r exp r 2 x2 i dx dx r. 2π r q r Poisso., Poisso. Poisso r k i =. 2 Boltzma Kullback-Leibler, Boltzma. 2. q = q,..., q r. i i k i / p i, p = p,..., p r. p = exp Dp q + Olog. : p = p,..., p r s f ν,i p i c ν ν =, 2,..., s., R r,,...,, f ν,,..., f ν,r ν =,..., s. p,?, i E i E i p i U U,?

20 20 2. Boltzma, i, E i, E i p i U U,? 2 s =. s >. 2.2 Boltzma, r p i = Kullback-Leibler Dp q = r p i logp i / p = p,..., p r. Lagrage. Kullback- Leibler p. L = p i log p s i + λ p i + β ν f ν,i p i c ν q i ν=. λ, β ν. p i L 0 0 = L λ = p i,. 3,, 0 = L β ν = f ν,i p i c ν ν =,..., s, 2 0 = L p i = log p i + λ + Z := e λ = p i = exp s β ν f ν,i i =,..., r 3 ν= λ s β ν f ν,i ν= e s ν= β νf ν,i, p i = Z e s ν= β νf ν,i 4. Z. p i Z = e λ β ν. β ν 4 2. exp s ν= β νf ν,i Boltzma. Boltzma p i. p i., p = p,..., p r..

21 s =, f,i = E i, c = U, β = β, p i = Z e βe i, Z = e βe i, log Z β = Z E i e βe i = U. Boltzma. Sp q = Dp q = r p i logp i / : logp i / = s ν= β νf ν,i log Z, r p i =, r f ν,ip i = c ν s Sp q = β ν c ν + log Z. ν= s =, f,i = E i, c = U, β = β Sp q = βu + log Z. F F = β log Z, Sp q = βu F, Boltzma, qx. px / Sp q = Dp q = px log px qx dx. px : f ν xpx dx = c ν ν =,..., s., Dp q px : px = s Z e ν= β νf ν x qx, Z = e s ν= β νf ν x qx dx, log Z = f ν xe s ν= β νf ν x qx dx = c ν. β ν Z... 9 Boltzma.

22 22 2. Boltzma : 0 < θ <, β = log θ log θ, k = 0,,..., p k = θ k θ k = e βk q k k Z, q k = k 2, Z = 2 θ : θ i 0, θ r > 0, r θ i =, β i = log θ i log θ r, k + + k r = : p k,...,k r = q k,...,k r = r! k! k r! θk θr kr = e β ik i q k,...,k r, Z! k! k r! r, Z = r θr /2σ 2 x 2 +µ/σ 2 x px = e x µ2 = e /2σ2, Z = e µ2 /2σ 2 2πσ 2π Z 2. µ = 0, σ = 2.4. px µ σ 2, px dx =, R x px dx = µ, x 2 px dx = σ 2 + µ 2, Sp = px. R R R px log px dx Gamma : x > 0 px = e x/τ x α τ α Γα = e x/τ+α log x Z, Z = τ α Γα. Gamma px px dx =, R x px dx = c, 0 0 Spx] = 0 log x px dx = c 2 px log px dx px.. Beta : x > 0 x α px = Bα, β + x = eα log x α+β log+x, Z = Bα, β. α+β Z

23 t / : t ρt dt = +/2 + t2 dt, c = π Γ/2 B/2, /2 = c Γ + /2. px dx = ρ x d x, β = + /2 px = Z Beta : + x 2 0 < x < px = xα x β Bα, β Z +/2 = e β log+x2 = eα log x+β log x Z, Z = B/2, /2., Z = Bα, β. Poisso : p k = e λ λ k k! = e log λk q k, q k = e Z k!, Z = eλ s =, f x = x 2, c =, qx = 0., x 2 x 2 + +x 2 / x x 2 =, x. px = Z e βx2, Z = R e βx2 dx = πβ /2, log Z β = 2β =. β = /2, Z = 2π, px = e x2 /2 / 2π.. R 2. : lim S fx µ dx = R fx e x2 /2 2π dx. S { x,..., x R x x 2 = }, µ, fx x x,..., x..,. 0 qx = qx.,. Maxwell-Boltzma.

24 24 3. Saov 3 Saov Saov. Stirlig. [2].. 3. Saov {, 2,..., r} P : P = { p = p,..., p r R r p,..., p r 0, p + + p r = }. P r. r = 3 P. q = q,..., q r P. X, X 2,... {, 2,..., r}, q = q,..., q r. q = q,..., q r. A #A, A P A. A B P B A. i =,..., r X,..., X i k i P #{ k =, 2,..., X k = i } = k i for each i =,..., r =! k! k r! qk q k r r. k,..., k r k i = 0,,...,, k + + k r =. k,..., k r k /,..., k r / P P : { k P =,..., k } r k i = 0,,...,, k + + k r =. P + r. #P + r. X,..., X P P = k /,, k r /. P P. p, q P 2 Dp q : Dp q = p i log p i. p i 0 0 log 0 = 0, log 0 =. Dp q Kullback-Leibler Kullback-Leibler. 3. Saov. 2 : 2 lim if, lim sup 9.

25 3.2. Saov 25 A P lim if 2 A P 3 lim sup log P P A if p A Dp q. log P P A if p A Dp q. 3 P A A lim log P P A = if Dp q. p A Kullback-Leibler Dp q if r = 2, q = q, q 2 = q, p = p, p 2 = p, Dp q = p log p q + p log p q. p = q 0, p q. 0 a < b, A = a, b = a b. P P A = q k q k k lim log a<k/<b a<k/<b q k q k = if k Dp q = a<p<b. Saov. Db q b < q, Dq q = 0 a q b, Da q q < a 3.2 Saov Stirlig k, l l! k! kl k. 3 A., A.

26 26 3. Saov. l k l k. l! k! = k + k + 2 l kl k. l! k! = l + l + 2 k = k l k. k k l Saov. Stirlig p P + r e Dp q P P = p e Dp q.. p = p,..., p r = k /,..., k r / P, Dp q = k i log p i + k i log, e Dp q = qk q k r r, P P p k = p = p k r r, : + r! k!... k r! pk p kr r.! k! k r! qk q k r r.... l i = 0,,...,, l + + l r =., p i = k i /! l!... l r! pl p lr r! k!... k r! pk p kr r. l i / p i = k i /., 3.3, = l! k! lr! k r! kk l kr kr lr k l k kr lr kr k k l kr kr lr =.., = l + +l r = + r.! l!... l r! pl p l r r + r! k!... k r! pk p k r r

27 3.3. Saov f f0 = 0, f! = ff2 f, f0! =. f. k, l,, l k, l k fl! fk! fl! fk! fkl k. = fk + fk + 2 fl fkl k l! k! = fl + fl + 2 fk = fk l k. fk k l k i Z 0 p i = fk i /f, l i Z 0, f! fl! fl r! pl p l r r f! fl! k r! pk p k r r. = fl! fk! fl r! fk r! fk k l fk r kr lr fk l k fk r lr kr fk k l fk r kr lr =. f. 3.3 Saov 3... A {, 2,..., r} P r. = P = P Q r P. A P p P A. P P A = p P A lim Dp q = if Dp q p A P P = p P P = p 3.4. log P P A Dp q r log + + r e Dp q.

28 28 3. Saov.,, lim if log P P A if p A Dp q.. 2. A {, 2,..., r} P. P P A = P P = p e Dp q + r e if p A Dp q. p P A p P A 3.4. log P P A if Dp q + r log + p A.,, lim sup log P P A if p A Dp q A B, B C, A C. B A C if p B Dp q if p A Dp q if p C Dp q. C B Dp q p, if p C Dp q = if p B Dp q. if p B Dp q = if p A Dp q = if p C Dp q., Stirlig.. : p i 0, p + + p r =! k! k r! pk p kr r k i Z 0, k + + k r =.., k i k + +k r=,. 2 : k i Z 0, k + + k r =, p i = k i /,! l! l r! pl p l r r! k! k r! pk p kr r = p + + p r r =! k! k r! pk p k r r l i Z 0, l + + l r =

29 3.3. Saov 29. k, l : l! k! kl k k, l Z 0.,, p i = k i / = l! k! lr! k k kkr r k r! k l kr lr k l k k l r k r r k k l k k r l r r =. 2. Saov. 3.4 Kullback-Leibler. k i Z 0, k + + k r =, p i = k i /. 2 l i = l + +l r =! l! l r! pl p l r r + r! k! p r! pk p k r r. + r +! r k! k r! pk p k r r. Z 0, q + + q r =, p k p k r r q k q k r r q k + r q kr r p k p kr r! k! k r! qk q kr k i = p i, Kullback-Leibler r qk qr kr. p k p kr r, Dp q = p i log p i : log qk q k r r p k p kr r = log q + r e Dp q p r p qr p r pr = Dp q.! k! k r! qk q kr r e Dp q Saov 3..

30 30 4. Saov 4 Saov 3. P {, 2,..., r} p = p,..., p r, q = q,..., q r P., i k i, i k i /. P = k /,..., k r / P E = E,..., E r R r, E = = E a < E a+ E r b < E r b+ = = E r q, q r > 0. E i i 5. β R pβ = p β,..., p r β P Zβ p i β = e βe i Zβ, Zβ = e βe i,. Uβ = E β Uβ = E β = E i p i β = log Zβ β. β, e βe i Boltzma, pβ, Zβ, Uβ. log Zβ β. 2 log Zβ = Z βzβ Z β 2 β Zβ 2, a i = e βe i 0, a, a r > 0 E < E r Z βzβ Z β 2 = i,j E 2 i a i a j i,j E i a i E j a j, = Ei 2 + Ej 2 a i a j 2E i E j a i a j = E i E j 2 a i a j > i,j i,j 2 log Zβ > 0 β., Uβ = log Zβ β 4. P, P. 5 i. i,j

31 β. Uβ. p0 = q U0 = E i. β Uβ = i E ie βe i i e βe i qi = e βe e βe i E ie βe i E q i i e βe i E e βe a E i e βe a = E. β Uβ = i E ie βe i i e βe i qi = e βer e βe r i E ie βer Ei q i i eβe r E i e βer r i=r b+ E i e βe r r i=r b+ = E r., E r U E β U = Uβ. 4.2 p = p,..., p r P, r E ip i Uβ, pβ.,, r E ip i Uβ, a > 0 : β 0, Uβ a β 0, Uβ E i p i Uβ. E i p i Uβ + a. a > 0. {, 2,..., r} P A A = { { p P Uβ a r E ip i Uβ} β 0, { p P Uβ r E ip i Uβ + a} β 0. P A P P B P A = P P A B P P A B P pβ Saov 3.., ε > 0, P B : B = { p P p pβ < ε }.

32 32 4. Saov Euclid. B pβ ε., pβ P P B P A. Kullback-Leibler Dp q P C DC q = if p P Dp q. Saov, P C C P P C = exp DC q + o. P A, B, A B A, B, A B. B A B = A B. P P B P A = P P B P P A = exp DB q DA q + o. 0. p A p = pβ Dp q, B = A B pβ, DB q > DA q = Dpβ q, P P B P A 0. Dp q p, A P, p A Dp q p = pβ, p = pβ. p A Dp q p = pβ.. pβ E i p i β = Uβ, pβ A. 6 Dpβ q = = p i β log p iβ = p i β log e βe i Zβ p i β βe i log Zβ = βuβ log Zβ. % p A Dp q. p A Dp q Dpβ q. p A., A, β 0 r E ip i Uβ, β 0 r E ip i Uβ, β β E i p i βuβ. # 6 Sp q = Dp q Sp q = βuβ + log Zβ..

33 , β > 0 Uβ < r E i, β < 0 Uβ > r E i. A a > 0. Kullback-Leibler Dp q : Dp q = p i log p i = pi p i log p i β p i β = p i log p i p i β + = Dp pβ + p i log e βe i Zβ = Dp pβ + = Dp pβ β E i p i log Zβ. p i log p iβ p i βe i log Zβ, # Kullback-Leibler Dpβ q % Kullback-Leibler 0, Dp q Dp pβ βuβ log Zβ = Dp pβ + Dpβ q Dpβ q. p A Dp q p = pβ.. 4. #. #. β Boltzma. β > 0 # = E i p i Uβ. [0] 9-2- p.39., r E ip i Uβ, #.. 4., U0 r E i. β > 0 Uβ mi{e,..., E r }, β < 0 Uβ max{e,..., E r }. β =, β =. β, β =, β = 0. β.. β,.. r, i E i. E i < 0 E i. i. U 0 = r E i

34 34 4. Saov. U 0. U > U 0.. U,. U 0 U., i p i? U 0, U U. i p i, U β p i β. U > U 0 β < 0.,,., U < U 0,, U., U 0, U, i U p i β. U < U 0 β > 0,.,,. # ,. q = q,..., q r P. X, X 2,... q {, 2,..., r}. X, X 2,..., X i k i, P = k /,..., k r /. P P. E i R 4.. E < U < E r, β R, pβ = p β,..., p r β P, Zβ : p i β = e βe i Zβ, Zβ = e βe i, log Zβ = β E i p i β = U.

35 pβ. 0 < a, A U P A U = { { p P U a r E ip i U } β 0, { p P U r E ip i U + a} β 0. r E ip i β = U pβ A U. ε > 0, pβ A U ε B ε pβ., P P B ε pβ P A U = P P B ε pβ P P A U. P pβ θ /2.. {0,,..., r} X /2 : r P X = i = i = 0,,..., r. i 2 r X, X 2,... X. X, X 2,..., X i k i, P = k 0 /, k /,..., k r /, P {0,,..., r}.,, P. E i = i., Zβ = e βe i = i=0 i=0 r i e β 2 i r i = e β + r 2 2 r, p i β = e βe i Zβ = r e βi i e β + = r r e β i r i i e β + e β +.,, p i β =, r θ i θ r i, θ = e β i e β +. Z β = re β e β + r 2 r Uβ = β Zβ = Z β Zβ = re β e β + = rθ

36 36 4. Saov θ i. /2. /2. r i i/r.. i i/r. 7 : i/r = i=0 i r p i θ. p i = k i /? p = p 0, p,..., p r θ, 4.2. pβ θ, 4.2, i = ip i = i=0 E i p i Uβ = rθ i=0 p = p 0, p,..., p r P.. θ θ.. /2 r, r. r θ 8, θ E i : = e β 0E i q 0,i Z 0, Z 0 = e β 0E i,0. q 0,i 0, r q 0,i =, E,..., E r R E E r, q, q r > 0., E i pβ : p i β = e β 0+βE i q 0,i Z 0 Zβ, Z 0 Zβ = Z 0 e βe i = e β 0+βE i q 0,i , θ / θ /2, 0,.

37 37 5 : Kullback-Leibler Cover-Thomas [3].., Kullback-Leibler. 5. : Jese fx E[fX] E[ ] : : fx, gx α, β, E[αfX + βgx] = αe[fx] + βe[gx]. 2 : fx gx E[fX] E[gX]. 3 : E[] =., α E[α] = α. E[ ]. p i 0, r p i =, E[fX] = r fx ip i. ρx 0, b ρx dx =, E[fX] = b fxρx dx a a. Jese : fx E[fX] fe[x]. fx E[fX] fe[x].. fx.. fx, µ = E[X]. fx X = µ ax µ + fµ,, fx ax µ + fµ E[fX] E[aX µ + fµ] = ae[x] µ + fµ = fe[x]. 2, , fx, X µ = E[fX] fx = µ,. 9 E[fX] = r fx ip i Jese r,.

38 38 5. : Kullback-Leibler 5.2 [3], p.3, Theorem 2.7. Log sum iequality. 0 a i, b i, a i log a i A log A b i B, A = a i, B = b i. a i /b i. 0 log 0 = 0, a loga/0 =.. a i > 0, b i > 0.. fx = x log x, f x = log x +, f x = /x fx x > 0. Jese. = b i /B. B a i log a i b i = b i a i log a i = f B b i b i a i f = f b i ai b i b i a i B b i = f A = A B B log A B. fx, a i /b i. p i, Dp q = p i log p i log = 0 Kullback-Leibler., {, 2,..., r} {, 2,..., r} = A A s, { A,..., A s } P = P,..., P s, Q j = Q,..., Q s P j = i A j p i, Q j = i A j, Dp q = s p i log p i i A j j= s j= P i log P j Q j = DP Q., Kullback-Leibler.

39 5.3. Kullback-Leibler L Kullback-Leibler L {, 2,..., r} p = p,..., p r, q = q,..., q r L p q L p q L = p i. p pa = i A p i A {, 2,..., r}, p q L = 2pA qa, A = { i {, 2,..., r} pi }. # p q L = i + i Ap p i = pa qa + qa c pa c i A c = pa qa + qa pa = 2pA qa.. KL L : Dp q 2 p q 2 L.. r = 2. 0 < a <, 0 < b < a log a b + a log a b 2a b2. fb : fb = a log a b + a log a b 2a b2. f b = a b + a 4b a = b a b b b 4. b a /4, /b b 4 0. fb b a. fb b < a, b > a. fp = 0 fb 0.. A #, {, 2,..., r} {A, A c } P = a, a, Q = b, b a = pa, b = qa. Kullback-Leibler 5.2 Dp q DP Q 2a b 2 = 2pA qb 2 = 2 p q 2 L. 2 r = 2, #.

40 40 6. : Cramér 5.4 Pithagoria theorem P = { p = p,..., p r R r 0 p + + p r = }, P {, 2,..., r}. P r. [3], p.367, Theorem.6.. Pythagoria theorem E P, q P E. p = p,..., p r E Dp q E p : Dp q = mi p E Dp q. Dp q Dp p + Dp q p E. Kullback-Leibler, Dp q Dp q p p. Pythegoria theorem. p p Kullback-Leibler p. t R. pt = p t,..., p r t = tp + tp, p i t = tp i + tp i, ft = Dpt q = tp i + tp i log tp i + tp i f t = = p i p i log tp i + tp i + p i p i p i p i log tp i + tp i. 2 r p i = r p i =. p0 = p E, p = p E, E pt E 0 t. p Dp q E p, f f 0 = = p i log p i p i p i log p i = p i log p i p i. p i log p i log p i pi p i p i p i log p i = Dp q Dp p Dp q. 6 : Cramér. formulatio Cramér.

41 6.. Cramér 4 6. Cramér H, U 0 = E[H]. H 20 : Zβ = E[e βh ].. Uβ : Uβ = E[He βh ] Zβ = log Zβ. β H, H 2 H, Zβ, 2 log Zβ = β β Uβ = Z βzβ Z β 2, Zβ 2 Z βzβ Z β 2 = 2 E[H H 2 2 e βh +H 2 ] 0, log Zβ, Uβ., U0 = U 0 = E[H], β 0 Uβ U 0, β 0 Uβ U 0. log Zβ Legedre Su Du : Su = if βu + log Zβ, β R Du = Su = sup βu log Zβ β R u R. Su u, Su. 0u log Z0 = 0 Su 0, Du 0 u R. Zβ, βu + log Zβ = u Uβ β, Uβ = u β = βu, Su = βuu + log Zβu, SUβ = βuβ + log Zβ. β = βu. Su, Du = Su Kullback-Leibler. SU0 = 0, DU0 = 0 20,, H Mt = E[e th ], t = β, Zβ.

42 42 6. : Cramér, Su u = U0 = E[H] 0, Du 0. Su Du u U0 = E[H], u U0 = E[H]. Uβ, u U0 = E[H] βu 0, u U0 = E[H] βu 0. : Su = { ifβ 0 βu + log Zβ u U0 = E[H], if β 0 βu + log Zβ u U0 = E[H]. # Du. u H U0 = E[H] β. 6. Cramér., H, H 2,..., H. : F R lim sup log P 2 G R lim if log P H k F k= H k G k= sup Su = if Du. u F u F sup Su = if Du. u G u G 3 A R, A G F A, sup u G Su = sup u F Su lim log P H k A = sup Su = if Du. u A u A k=, H + + H / A Su Kullback-Leibler Du A. 6.2 Cramér 6.2 Cramér. 6.. u H U0 = E[H], log P H k u Su u U0 = E[H], k= log P H k u Su u U0 = E[H]. k=, u U0 = E[H] Su, u U0 = E[H] Su.

43 6.2. Cramér 43 2 δ > 0 lim if log P H k u δ, u + δ Su. k=. 2..., u U0 = E[H]. H + + H u 0 H + +H u, β 0 P H k u = E[ H + +H u] k= E [ H + +H ue βh + +H u ] E [ e βh + +H u ] = e βu Zβ = e βu+log Zβ. u U0 = E[H], β 0 log P H k u βu + log Zβ. 6. # log P k= k= H k u if βu + log Zβ = Su. β 0, u U0 = E[H],. H + + H u 0 H + +H u, β 0 P H k u = E[ H + +H u] k= E [ H + +H ue βh + +H u ] E [ e βh + +H u ] = e βu Zβ = e βu+log Zβ. u U0 = E[H], β 0 log P H k u βu + log Zβ. 6. # log P k= k= H k u if βu + log Zβ = Su. β 0

44 44 6. : Cramér H R µ. µ β µ β dx = e βx µdx Zβ,. E β [ ], P β. H E β [H] = E[He βh ] = Uβ Zβ. u = Uβ, δ > 0. Su, Su = βu + log Zβ. δ ε > 0. lim P β H k u ε, u + ε =. k=, : P β H k u ε, u + ε = E[ H + +H u ε,u+ε e βh + +H ] Zβ k= Zβ E[ H + +H u ε,u+ε e βu+ β ε ] = e βu+log Zβ β δ P H k u ε, u + ε k= e βu+log Zβ β δ P H k u δ, u + δ. k= P H k u δ, u + δ e βu+log Zβ β ε + o k=. / : lim if log P H k u δ, u + δ βu + log Zβ β ε = Su β ε. k= ε > 0, lim if log P H k u δ, u + δ Su. 2. k=

45 6.2. Cramér F R, F + = { u F u U0 = E[H] }, F = { u F u U0 = E[H] }. R, F u F + u +. Su u U0 = E[H], u U0 = E[H] sup Su = Su +, u F + sup Su = Su, u F sup Su = max{su +, Su }. u F 6.2, log P H k F + k= log P H k u + Su +, log P H k F log P H k u Su, k= k= P H k F e Su + + e Su 2e sup u F Su. k= k= lim sup log P H k F k= sup Su. u F. 2. G R. ε > 0, u G Sv sup Su ε u G. G, δ > 0 v δ, v + δ G., lim if log P H k G lim if log P H k v δ, v + δ k= lim if log P k= k= Sv sup Su ε. u G H k G k= sup Su. u G A R, A G, G F. A F. G A F sup Su lim if u G log P H k G lim if log P H k A k=

46 46 6. : Cramér lim sup sup u G Su sup u A log P H k A k= Su sup Su. u F lim sup sup u G Su = sup u F Su lim log P H k A. k= log P H k F k= = sup Su. u A sup Su, u F 6.3 q = q,..., q r {,..., r} : 0, q + + q r =. H {,..., r} Hi = E i R. Zβ = E[e βh ] = e βe i. pβ = p β,..., p r β p i β = e βe i Zβ, Uβ = E β [H] = E i p i β = Zβ E i e βe i. SUβ = βuβ + log Zβ = βuβ log Zβ = p i β βe i log Zβ = p i β log p iβ = Spβ q., log Zβ Legedre Su u = Uβ. 6.4 H α > 0, τ > 0 E[fH] = Γατ α 0 fxe x/τ x α dx

47 β > /τ Zβ = E[e βh ] = Uβ = + τβα Γατ α Γατ α 0 0 e +τβx/τ x α dx = + τβ, α + τβα Γα + τ α+ = τα Γατ α + τβ α+ + τβ > 0. e +τβx/τ x α dx = 2 : Uβ = u > 0 0 e cx x s dx = Γs c s s, c > 0. SUβ = βuβ + log Zβ = = α α + τβ ταβ + τβ α log + τβ. + τβ = τα u, β = α u τ α log + τβ, Su = α u τ α log τα u = α α log α u τ α log u τ. Su u = U0 = τα 0. H, H 2,... H α, τ, 22, H + + H α, τ 2. Uβ Uβ = / β log Zβ. 22 H + + H H Zβ = + τβ α. α, τ. H + + H α, τ.. H, K, α, β, τ E[fH + K] = = = ΓαΓβτ α+β ΓαΓβτ α+β ΓαΓβτ α+β = ΓαΓβτ α+β Bα, β = ΓαΓβτ α+β = Γα + βτ α+β fx + ye x+y/τ x α y β dy dx 0 fte t/τ x α t x β dt dx x y fte t/τ x α t x β dx dy 0 fte t/τ fte t/τ t α+β dt fte t/τ t α+β dt 0 t α u α t β u β t du dy 2 y = t x y t, 4 x = tu x u. H + K α + β, τ..

48 48 6. : Cramér, [ ] E f H k = k= 0 a < b P a < H k < b = k= Γατ α Γατ α b a 0 f e y/τ y α dy = 2 y = τx. Cramér, lim log b/τ α e x x α dx = sup Su = Γα a/τ a<u<b y e y/τ y α dy. b/τ α e x x α dx. Γα a/τ Sb b/τ < α, 0 a/τ α b/τ, Sa α < a/τ. Stirlig log Γα = α log + α α log α + o, A = a/τ, B = b/τ: B lim log e x x α dx A = sup α log x x = a/τ<x<b/τ α log B B α log α α α log A A B < α, A α B, α < A. e x x α = exp α log x x + o Laplace. 6.5 Saov Cramér., Cramér. R r,. X R r, β = β,..., β r R r Massieu Zβ = E[e β,x ], Ψβ = log Zβ. Ψβ β R r. a = a,..., a r R r a = r a i / β i, X a = a, X 2 aψβ = E[X2 ae β,x ] E[e β,x ] E[X a e β,x ] 2 Zβ 2 = E[X a Y a 2 e β,x+y ] 2Zβ 2 0.

49 6.5. Saov Cramér 49 Y X X. X a 2 aψβ = 0. p R r Sp Ψβ Legedre : Sp = if β Rr β, p + Ψβ. β, p + Ψβ β i 0 p i = Ψβ = log Zβ = E[X ie β,x ] β i β i Zβ =: p i β. X i X i i. pβ = p β,..., p r β. β βp = β p,..., β r p... Spβ = β, pβ + Ψβ, Sp = βp, p + Ψβp. X, X 2,... R r, X, A R r. lim log P X k A = sup Sp p A k= Cramér. Saov Cramér. {, 2,..., r} P R r : P = { p = p,..., p r R r p,..., p r 0, p + + p r = }. e i R r i 0. e i P. q = q,..., q r R r, X e i. Cramér Saov. Sp =. p i log p i = Sp q Zβ = e β,ei = e β i, Ψβ = log Zβ. Ψβ Legedre Sp q., p i = p i β = Ψβ = log Zβ = e βi β i β i Zβ.

50 50 6. : Cramér, Ψβ = log Zβ = p i log Zβ = p i β i + log log p i = β, p p i log p i., Sp, Sp = β, p + Ψβ = p i log p i = Sp q. Cramér Saov. 6.6 Ψβ = log r e β i Legedre {, 2,..., r} P R r : P = { p = p,..., p r R r p,..., p r 0, p + + p r = }., q = q,..., q r P > 0 i =,..., r. q P,. Zβ Massieu Ψβ pβ = p β,..., p r β P : Zβ = e β i, Ψβ = log Zβ, p i β = Ψβ = e βi β i Zβ β = β,..., β r R r.. Ψβ. e =,..., R r. Zβ + λe = e λ Zβ, Ψβ + λe = Ψβ λ, pβ + λe = pβ. pβ β e. p = p,..., p r P, β R r p i = e β i λ λ R, p = pβ : e β i = e λ p i, Zβ = e β i = e λ p i = e λ, p i β = e β i Zβ = p i. pβ P. p = p,..., p r R r β R r fβ fβ = β, p + Ψβ

51 6.6. Ψβ = log r e β i Legedre 5. fβ, fβ, fβ 0. fβ β i fβ = β i β i p i + β i Ψβ = p i p i β, fβ p P. fβ if β R r fβ =., Ψβ Legedre Sp Sp = if β Rr β, p + Ψβ, Sp P. p P, p = pβp βp R r βp e, Sp = βp, p + Ψβp. β = βp p i = e β i λ β i = log p i λ, Zβ = e λ, Ψβ = λ Sp = log p i λ p i + λ = p i log p i = Sp q. Ψβ Legedre Sp q. Sp. Sp Legedre F β = sup β, p + Sp p P Ψβ. gp gp = β, p + Sp. gp p. gp P. Lagrage p λ L L = gp λ p i = β, p + Sp λ p i. L λ = p i,

52 52 7. :? L p i = β i log p i λ = β i log p i λ., r p i = gp e λ = e β i = Zβ,., Ψβ = log Zβ = λ = p i = e λ β i = e β i Zβ, p i λ = Sp Legedre F β Ψβ.. log Zβ = λ = β i log p i p i β i log p i = β, p + Sp = F β q = q,..., q r. R r Ψβ P Sp : Ψβ = log e β i, Sp = p i log p i. Legedre. Sp = if β Rr β, p + Ψβ, Ψβ = sup β, p + Sp. p P P Dp Dp = p i log p i, Dp Ψβ, Dp = sup β R r β, p Ψβ, Ψβ = sup β, p Dp p P. 7 :?.., :, Cramér : : 7.3.

53 , Cramér 6. H λ λ, H λ R µ λ : E[fH λ ] = fx µ λ dx. λ V, H λ. Zβ, λ Ψβ, λ Massieu : R Zβ, λ = E[e βh λ ], Ψβ, λ = log Zβ, λ. R µ β,λ dx = e βx µ λ dx Zβ, λ P β, β = E β [ ]. Ψβ, λ = log Zβ, λ λ Ψβ, λ = λψβ + η λ β, η λ β = o, η λβ = o, η λβ = o. H λ β = E[H λe βh λ ] Zβ, λ = β Ψβ, λ = λψ β + o Hλ λ β uβ = ψ β = uβ + o uβ λ., µ β,λ H λ, Z = Zβ.λ, 0 H λ H λ β 2 = Hλ 2 β H λ β 2 = Z ββz Z β 2 β Z 2 2 = Ψβ, λ = λψ β + o β. Ψβ, λ β. ψβ. Hλ λ µ β,λ = ψ β + o = O 0 λ. λ λ λ, µ β,λ H λ /λ., µ β,λ H λ /λ λ

54 54 7. :? uβ = ψ β, λ, 2 ψ β/λ. Ψβ, λ β, H λ β = Ψ β β, λ β. ψβ, uβ = ψ β. β 0, u = uβ = ψ β u0,. U = Uβ, λ = H λ β, su = βu + ψβ SU, λ = βu + Ψβ, λ, λ U = λu + o, Ψβ, λ = λψβ + o, SU, λ = λβu + ψβ + o = λsu + oλ λ., µ λ dx = q λ x dx,, p β,λ x = e βx q λ x Zβ, λ µ β,λ dx = e βx q λ x Zβ, λ dx = p β,λx dx, SU, λ : SUβ, λ, λ = βx + log Zβ, λp β,λ x dx R e βx = log p β,λ x dx = Zβ, λ R R p β,λ x log pβ,λ x dx. q λ x SU, λ, su λ. P H λ /λ u λ. Hλ P λ u = E[ Hλ λu] E [ Hλ λue ] βh λ λu E[e βh λ λu ] = e λβu Zβ, λ = e λβu+ψβ,λ. Hλ λu H λ λu, 0. Hλ λu Hλ λue βh λ λu β 0, 2 Hλ λue βh λ λu e βh λ λu. Ψβ, λ = λψβ + o log P Hλ λ u = λβu + ψβ + o = λsu + o.

55 λ λ lim sup λ λ P Hλ λ u su.. 0 < ε δ. µ β,µ H λ /λ λ u = uβ, Hλ P β u ε, u + ε = e o λ. λ, β 0, P β Hλ λ u ε, u + ε = E [ ] Hλ λu λε,λu+λε e βh λ Zβ, λ Zβ, λ E [ Hλ λu λε,λu+λε e λβu+λβε] = e λβu+λβε Ψβ,λ Hλ P u ε, u + ε λ = e λβu+λβε λψβ+o Hλ P u ε, u + ε λ e λsu βε+o Hλ P u δ, u + δ. λ λ Hλ P u δ, u + δ e λsu βε+o+o = e λsu βε+oλ. λ /λ λ lim if λ λ P Hλ u δ, u + δ su βε. λ ε > 0, lim if λ λ P Hλ λ u δ, u + δ su., 6 Cramér., 6, : lim λ λ P Hλ λ u = su. H λ /λ u = uβ λ, su.,.?

56 56 7. :? 7.2 [0], pp ,,.,..,. H N,V V N,. U 0 N, V = E[H N,V ]. = N/V V, U 0 N, V = V u 0 + ov. U 0 N, V =, u 0 =. V P H N,V /V u 0 ε ε > 0, H N,V /V u 0. U U 0 N, V u u 0, U u.. SU, N, V H N,V U : SU, N, V = log P H N,V U U U 0 N, V. SU, N, V U. U V P H N,V U [0], p. 05 δ u : u = U V, = N V.. [0], p SU, N, V U. SU, N, V V : ηu,, V, SU, N, V = V su, + ηu,, V, u = U V, = N V ηu,, V = ov, η u u,, V = ov V. lim V SU, N, V = lim V V, U = V u, / U = V / u, V log P H N,V U = su, S U U, N, V = V V s uu, + η u u,, V = s u u, + o.

57 s u u, u. su,. su, u. βu, 0 βu, = s u u, = S U U, N, V + o. 7. λ = V. [0], pp , P H N,V U E lim V P H N,V U. : log P H N,V U E P H N,V U = e βu,e = βu, E + o V.., 0 < θ < θ, log P H N,V U E P H N,V U = SU E, N, V SU, N, V = E S U U θe, N, V = E s u u θev, + o = E s u u, + o = βu, E + o. V. 2, 3, 4 s u u,. Boltzma..,.. U, E, U E. U E V P H N,V U E. E P H N,V U E., E V Boltzma e βu,e.,.?. 2.4 Maxwell-Boltzma.,,...

58 58 7. :? , H, H k, H res, H tot. thermal reservoir, heat bath. V. N. H, H res V HV tot, P H = E i ad H tot V = H + H res V,. U = P H = E i P H res V U E i. i, i E i. H E i : P H = E i =. 2 i,. Boltzma. V : S res U, V := log P H res V U = V s res u + ov, u = U V. S res U, V, u = U/V, s res u C. ov u ov., U = V u / U = /V / u,, E θ 0 < θ <, log P H res V, βu, U E = S res U E, V = S res U, V E Sres U θe, V U = S res U, V E V s res u θe + ov V u V P H res V = S res U, V E sres u + o. u βu = sres u u U E = P H res V U e βue+o V.,,2, P H = E i ad H tot V U = P H res V U e βue i+o.

59 59 P H tot V U = i P H = E i ad H tot V U = P H res V U i e βue i+o.,, P H = E i H tot V U = P H = E i ad H tot V U P H tot U P H = E i HV tot U = e βue i+o j q je βue j+o e βuei j q je βue j V., U, E i : lim P H = E i HV tot U = e βue i V j q je. βue j βu. e βue i Boltzma H, H 2,..., V = =, 2, 3,..., H tot = H + H H, H = H, H res = H H, Saov Cramér,.., Boltzma e βue i. 8 : 8. Massieu 8.. X : i X i M X t M X t = e tx i. X = E, t = β Zβ = e βe i.. K X t = log M X t

60 60 8. : X cumulat geeratig fuctio. F β = log Zβ β. β Fβ = log Zβ Boltzma. Fβ Massieu. 8.2 Réyi 8.2 Réyi. 2 p = p,..., p r, q = q,..., q r, Réyi S β p q S β p q = β log pi β = β log. β β r β β S βp q = pβ i q β i logp i / r pβ i q β i, β =, β β S β p q = β=. p i log p i = Sp q S p q := lim β S β p q = Sp q. p β i q β i Réyi. = Réyi Réyi. Réyi Zβ; p, q = pi q i β = e βe i, E i = log p i F β : p, q Massieu Fβ; p, q F β; p, q = β log Zβ; p, q, Fβ; p, q = log Zβ; p, q Boltzma : β S β p q = βf β; p, q = Fβ; p, q = log Zβ; p, q.

61 8.3. Tsallis 6 Réyi divergece Réyi []. β S β p q = log Zβ; p, q β : 2 r i,j= log Zβ; p, q = E i E j 2 e βe i+e j q j 0 β 2Zβ 2 p i = i =,..., r., β S β p q = log Zβ; p, q β = log Z; p, q = log = 0, β S β p q = log Zβ; p, q β = Sp q,. β S β p q β Sp q. 8.3 Tsallis 8.3 Tsallis. p = p,..., p r, q = q,..., q r, Zβ; p, q : Zβ; p, q = e βe i = pi q i β = p β i q β i, E i = log p i. E i 2 p q i. pβ = p β,..., p r β p i β = e βe i Zβ; p, q = pβ i q β i Zβ; p, q, β = p i 0 p i = p i., Réyi S β p q S β p q = log Zβ; p, q β = β log p β i q β i, Sp q Sp q = β log Zβ; p, q = β= β Zβ; p, q = β=. 2 Z; p, q =. x q : p i log p i D x,q fx = fx fqx qx. q q D x,q fx fx/ x.

62 62 8. : log Zβ; p, q Zβ; p, q β q 23, Tsallis 24 q q α : Z; p, q Zα; p, q T α p q = D β,α β= Zβ; p, q = α = r pα i q α i α α α, Tsallis. T α p q = p i / p i / α, α x x α lim α α = α α α= x α = x log x.,. Tsallis x log x x x α / α. Tsallis Réyi :. T β p q = Zβ; p, q β = exp βs βp q. β Réyi Tsallis S β p q = log Zβ; p, q β = log + βt βp q β. Tsallis Réyi x log x = log + x.,, Réyi, Tsallis Zβ; p, q. Tsallis Réyi. Tsallis T β p q Réyi S β p q Z β p q = p β i q β i T β p q = Z βp q β, S β p q = log Z βp q. β. β > β < Z β p q, Z β p q. e β x = + β x /β, l β x = xβ β Tsallis. Réyi, Massieu, Tsallis.

63 8.3. Tsallis 63, Z β p q /β = e β T β p q = exp S β p q. Tsallis Réyi e β x expx., T β p q = l β Zβ p q /β, S β p q = log Z β p q /β p i, 0, r p i = r = =, β > 0., Réyi, Tsallis Sp q =. l β x, Tsallis p i log p i S β p q = β log p β i q β i, r T β p q = pβ i q β i β l β x = xβ β T β p q = pi p i l β. Sp q, S β p q, T β p q 0. fx = x log x, f x = log x +, f x = /x fx, f = 0, f =, fx x. Sp q = f pi pi = 0. Tsallis gx = xl β x, g x = l β x + x β, g x = βx β 2 β >, gx, g = 0, g =, gx x. T β p q = g pi pi = 0. 8.

64 64 8. : Réyi β >, S β p q 0 p β i q β i = pi β. hx = x β, h x = βx β, h x = ββ x β 2 β >, hx, h =, h = β hx + βx. pi β pi + β =. Jese ν =, 2 R ν = {, 2,..., r ν } p ν = p ν,,..., p ν,rν, p ν = p ν,,..., p ν,rν, Réyi Sp ν q ν = r ν p ν,i log p ν,i q ν,i, S β p ν q ν = log Z βp ν q ν, Z β p ν q ν = β r ν p β ν,i q β ν,i. R R 2 = { i, j i R, j R 2 } i, j p,i p 2,j, i, j q,i q 2,j. Réyi : Sp, p 2 q, q 2 = i,j p,i p 2,j log p,ip 2,j q,i q 2,j, S β p, p 2 q, q 2 = log Z βp, p 2 q, q 2, Z β p, p 2 q, q 2 = β i,j : p,i p 2,j β q,i q 2,j β. Sp, p 2 q, q 2 = Sp q 2 + Sp 2 q 2, S β p, p 2 q, q 2 = S β p q 2 + S β p 2 q 2. Z β p, p 2 q, q 2 = Z β p q Z β p 2 q 2. : Sp, p 2 q, q 2 = i,j p,i p 2,j log p,i q,i i,j p,i p 2,j log p 2,j q 2,j = i p,i log p,i q,i j p 2,j log p 2,j q 2,j = Sp q + Sp 2 q 2,

65 8.5. Tsallis 65 Z β p, p 2 q, q 2 = i,j = i p,i p 2,j β q,i q 2,j β = i,j p β,i q β,i j p β,i q β,i p β 2,j q β 2,j p β 2,j q β 2,j = Z β p q Z β p 2 q 2. Tsallis T β p ν q ν = Z βp ν q ν, T β p, p 2 q, q 2 = Z βp, p 2 q, q 2 β β, T β p, p 2 q, q 2 = T β p q + T β p 2 q 2 + βt β p q T β p 2 q 2. : T β p, p 2 q, q 2 T β p q ] T β p 2 q 2 = Z βp q Z β p 2 q 2 Z β p q Z β p 2 q 2 β = Z βp q Z β p 2 q 2 Z β p q Z β p 2 q 2 + β = Z βp q Z β p 2 q 2 β = β T β p q T β p 2 q 2. q x q = q x / q x + y q = x q + y q + q x q y q. Tsallis 25 [8]. Tsallis. 8.5 Tsallis Suyari [8] Tsallis. Tsallis 26.. h β h = β, h + = β. h = β e h x l h x e h x = + hx /h, l h x = xh h 25 Tsallis = Tsallis. 26.

66 66 8. :., h 0 β., 8.3 Tsallis T β p q 27 : pi p i l h = h = T β p q = p i pi r pβ i q β i β pi p i l h. h p i = β = r pβ i q β i β p β i q β i p i = T β p q.. a; k =! k k r q k q kr r k = k,..., k r, k i 0, k + + k r =, : log a; k = log ν ν= k i log ν i = νi. l h a h ; k : l h a h ; k = l h ν ν= k i ν i = νi l h., l h x = x h /h /h /h, l h a h ; k = k i h ν h νi = k i ν h q h i νi h. h h ν= ν i = h > 0 β >, k i / k i = p i + O = p i + O/. l h a h ; k. h > 0, ν= ν i = ν= ν h = h+ h + + Oh = β β + Oβ. k i = p i + O, k i ν i = ν h i = p i h+ h + + Oh = β β pβ i + Oβ. 27 Tsallis., Tsallis T β p q Sp q = r p i logp i / h = β.

67 8.6. Tsallis 2 67 l h a h ; k = β β r pβ i q β i β + O β = β β T βp q + O β. 28.! log a; k = log k! k r! qk q k r r = Sp q + Olog. 8.6 Tsallis 2 Suyari-Scarfoe [9] a h ; k. [9] q 8.6 h = β h = q, h + = β = 2 q. e h x l h x e h x = + hx /h > 0 x >, l h x = xh > x > 0 h h h Tsallis T β p q = r pβ i q β i β = h p h+ i q h i = pi p i l h. a; k =! k k r q k q k r r k = k,..., k r, k i 0, k + + k r =, : log a; k = ν= log ν ν= ki ν i = ν i = log ν i + k i log q i l h a h ; k 29 : ki l h a h ; k = l h ν l h ν i + kh+ i h + l hq i. : l h a h ; k = ν h h ν= ki ν i = νi h + kh+ i h + q h i. 28 Tsallis Saov., h = β a h ; k ,..

68 68 8. : h > 0 β >, k i / k i = p i + O = p i + O/. l h a h ; k. h > 0, ν h = h+ h + + Oh = β β + Oβ. ν= k i = p i + O, q h i k i ν i = ν h i = p i h+ h + + Oh = β β pβ i + Oβ, k h+ i h + = p i h+ h + + Oh = β β pβ i + Oβ. = q β i.,, l h a; k = β h β β β pβ i q β i + O β = β β T βp q + O β..! log a; k = log k! k r! qk qr kr = Sp q + Olog. 8.7 Csiszár f-divergece, Csiszár [2]. fx 0 < x <, f = 0. {, 2,..., r} p = p,..., p r, q = q,..., q r, q p f-divergece D f p q D f p q = f pi. fx = x log x, f-divergece Kullback-Leibler divergece pi Dp q = p i log. fx = xl h x = x xh h, f-divergece p i / β p i / D f p q = = β = xβ x β, h = β p β i q β i p i = β r pβ i q β i β = T β p q Tsallis divergece Tsallis. f-divergece.

69 69 {, 2,..., r} A, p, q A pa = p i, qa = i A i A. A,..., A s {, 2,..., r}, {A, A 2,..., A s } P = P,..., P s, Q = Q,..., Q r P j = pa j, Q j = qa j. 5.2 Kullback-Leibler Dp q DP Q.. f-divergece D f p q D f p q fx Jese : D f p q = s pi f q j= i i A j s Q j f i Aj j= = p i Q j s Q j j= = pi qi f Q j i A j Pj s f j= Q j Q j = D f P Q. s =, A = {, 2,..., r} P = Q =, f = 0 D f P Q = 0 D f p q 0. Kullback-Leibler f-divergece. 9 : lim sup lim if ±,.,.,. 9. a, a 2,.... a, a +, a +2,... sup k a k sup a k = a, a +, a +2,... α. k

70 70 9. : α 30 : sup a k = mi{ α R { } a k α k }. k sup k a k., sup k a k ± 3. a limit superior, : lim sup a = lim sup k limit iferior : a k. if a k = max{ α R { } a k α k }, k lim if a = lim if a k. k sup if : lim if a lim sup a. a, sup k a k if k a k 0, lim if a = lim sup a = lim a. lim sup a lim if a a. 9.. : lim sup =, lim sup lim sup lim if =, =, + 2 =, lim if lim if =, + 2 = a. a B, C B a C

71 7!. : lim B lim if a lim sup a lim C. a. B C, a., a, a. Saov 3. 0 Mcmilla Mcmilla A A A : A = { a a 2... a l a, a 2,..., a l A, l = 0,, 2,... }. a = a a 2 a l l la = l. A. r, b 2. S, T r, b : S = {s,..., s r }, T = {t,..., t b }. w,..., w r T, C : S T Cs i s im = w i w im. C S b. C : S T, C. 0. Mcmilla. C S b. S s i Cs i = w i l i,. b l i

72 72 0. Mcmilla. a = r b l i. a. l l,..., l r,. a = i,...,i = l b l i + +l i = N k b k. N k l i + + l i = k s = s i s i. w = Cs = w i w i b k, w b k. C : S T, s b k. k= l l l a = N k b k b k b k = l. k=. a. k= k= a = l / = e / logl e 0 = 0.2 p i, 0, r p i = r =. Kullback-Leibler 0 Gibbs : Dp q = p i log p i 0. fx = x log x Jese : pi p i Dp q = f f = f = 0. a i 0, t a i = a, = a i /a a i = a p i log pi a i = : pi p i log = Dp q log a 0. a a i 0, t a i = p i log a i p i log p i. log b b >.

73 C S = {s,..., s r } b,, w i = Cs i l i. Mcmilla b l i, a i = b l i b, p i l i p i log b p i. S p i. S p i b., S p i, S p i. S a i 0, r a i, l i = log b a i s i. b = 2 s i l i bit. l i = log b a i s i b., b Kullback-Leibler q a D b p a = pi p i log b = a i p i log b a i p i log b p i 0 a i b i l i = log a i 32,. Kullback-Leibler. Kullback-Leibler Dp a p a. 32 Mcmilla, r a i.