17
i 1 1 1.1..................................... 1 1.2....................................... 1 1.3..................................... 2 2 5 2.1................................ 5 2.2 (1)......................... 7 2.3 (2)......................... 12 3 17 3.1................................... 17 3.2................................... 17 19
iii 2.1............................ 15
v 2.1 F (p)............................... 5 2.2.................................. 7
1 1 1.1 19 1898 1902 20 1926 1 1988 1 11 1 10 10 2 1 3 5 3 2 2 1 1.2
2 1 1.3 (2004) 80 K. 2 2 A x 1 x B B B y y A B 1 y A A 1 2 A 1 a B b a b 2 A B 50 B A 55 B 45 B B b =0.6 A 50
1.3. 3 50 b =30 30 B 1 1 3 B 3 A B 1 2 A x B y A y ax B y B 1 x b(1 y) A x y = ax, 1 x = b(1 y) x y A B A 1 A x B A x B 1 x = b(1 a)/(1 ab) ( )
5 2 2.1 p p F (p) F (1) = 0 F (p) p F (p) F (p) 2.1: F (p) y = log x x 0 1 y F (p) log p
6 2 p F (p) = log p(0 p 1) p F (p) 2 2 2 p 1 p I(p) = pf (p)+(1 p)f(1 p) = p log p (1 p) log(1 p) A = {a 1,a 2,,a k } a 1,a 2,,a k P (a 1 )=p 1,P(a 2 )= p 2,,P(a k )=p k 001 999 L=999 999 999 099 1 999 1 L=999 a 1 =0.999 0 a 1 A, B a 1,a 2 A log 2 a 1 B log 2 a 2 2 a 1 log 2 a 1 a 2 log 2 a 2 2
2.2. (1) 7 2.2 2.2: 2.2 (1) 2 A, B 2 1 A m, B 1 m A p = m 2 +2m 2 (1 m) = m 2 (3 2m) B q = (1 m) 2 +2m(1 m) 2 = (1+2m)(1 m) 2 1 A A p 1 = m + m(1 m) = m(2 m)
8 2 B q 1 =(1 m) 2 B 1 A p 2 = m 2 B q 2 = (1 m)+m(1 m) = 1 m 2 2 1 1 2 A p 3 = m B p 3 =1 m m m =1.0 I p = 1.0(3 2 1.0) = 1.0 q = (0+2 0)(1 0) 2 = 0 I = log 2 1.0 = 0 (2.1) 1, 2 I A m =0.1I p = 0.01(3 0.2) = 0.028 q = (1 2 0.1)(1 0.1) 2 = 0.972
2.2. (1) 9 I = 0.028 log 2 0.028 0.972 log 2 0.972 = 0.184 (2.2) 1 I 1 A p 1 = 0.1 1.9 = 0.19 q 1 = 0.9 2 = 0.81 I 1 = 0.19 log 2 0.19 0.81 log? 2 0.81 = 0.701 1 B p 2 = 0.1 2 = 0.01 q 2 = 1 0.01 = 0.99 I 2 = 0.01 log 2 0.01 0.99 log 2 0.99 = 0.081 1 I I = 0.1 0.701 + 0.9 0.081 = 0.143 (2.3) 2 I 1 1 I = 0.1 log 2 0.1 0.9 log 2 0.9 = 0.469 2 I (0.01 + 0.81) +0.18 0.469 = 0.0842 (2.4)
10 2 m =0.3 I p = 0.09(3 0.6) = 0.216 q = (1+0.6)(1 0.3) 2 = 0.784 I = 0.216 log 2 0.216 0.784 log 2 0.784 = 0.753 (2.5) 1 I 1 A p 1 = 0.3 1.7 = 0.51 q 1 = (1 0.3) 2 = 0.49 I 1 = 0.51 log 2 0.51 0.49 log 2 0.49 = 0.997 1 B p 2 = 0.3 2 = 0.09 q 2 = 1 0.09 = 0.91 I 2 = 0.09 log 2 0.09 0.91 log 2 0.91 = 0.436 1 I I = 0.3 0.997 + 0.7 0.436 = 0.604 (2.6)
2.2. (1) 11 2 I 1 1 I = 0.3 log 2 0.3 0.7 log 2 0.7 = 0.881 2 I (0.09 + 0.49) 0+0.42 0.881 = 0.371 (2.7) m =0.5 I p = 0.5 2 (3 2 0.5) = 0.5 q = (1+2 0.5)(1 0.5) 2 = 0.5 I = log 2 0.5 = 1 (2.8) 1 I 1 A p 1 = 0.5(2 0.5) = 0.75 q 1 = (1 0.5) 2 = 0.25 I 1 = 0.75 log 2 0.75 0.25 log 2 0.25 = 0.811 1 B p 2 = 0.5 2 = 0.25 q 2 = 1 0.5 2 = 0.75
12 2 I 1 = 0.75 log 2 0.75 0.25 log 2 0.25 = 0.811 1 I I = 0.5 0.811 + 0.5 0.811 = 0.811 (2.9) 2 I 1 1 I = log 2 0.5 = 1 2 I 0.5 0+0.5 1=0.5 (2.10) (1) m 0.5 1.0 0 2.3 (2) 4 A, B (3 3 ) 1 A a, B b A 2 A a 2 B b 2 1 a 2 b 2 A a 4 +4a 4 b +10a 4 b 2 +20a 5 b 3 {1+ (1 a 2 b 2 ) n } = a 4 +4a 4 b +10a 4 b 2 +20a 5 b 3 {1+(1 a 2 b 2 /a 2 + b 2 )}
2.3. (2) 13 B A 4 A, B (3 3 ) A a 4 +4a 4 b +20a 4 b 2 +20a 4 b 3 B A,B 1 a, b a = 0.6 b = 0.4 A 0.6 4 + 4 0.6 4 0.4+10 0.6 4 0.4 2 + 20 0.6 5 0.4 3 {1+(1 0.6 2 0.4 2 /0.6 2 +0.4 2 )} = 0.7357 0.6 4 + 4 0.6 4 0.4+10 0.6 4 0.4 2 +20 0.6 4 0.4 3 = 0.7102 1 A I I d I nd I d = 0.7537 log 2 0.7537 0.2463 log 2 0.2463 = 0.8054 (2.11) I nd = 0.7102 log 2 0.7102 0.2898 log 2 0.2898 = 0.8685 (2.12) A, B 1 a, b a = 0.7 b = 0.3 A 0.7 4 + 4 0.7 4 0.3+10 0.7 4 0.3 2 + 20 0.7 5 0.3 3 {1+(1 0.7 2 0.3 2 /0.7 2 +0.3 2 )} = 0.901
14 2 0.7 4 + 4 0.7 4 0.3+10 0.7 4 0.3 2 +20 0.7 4 0.3 3 = 0.874 1 A I d, I nd I d = 0.901 log 2 0.901 0.099 log 2 0.099 = 0.466 (2.13) I nd = 0.874 log 2 0.874 0.126 log 2 0.126 = 0.546 (2.14) A, B 1 a, b a = 0.8 b = 0.2 A 0.8 4 + 4 0.8 4 0.2+10 0.8 4 0.2 2 + 20 0.8 5 0.2 3 {1+(1 0.8 2 0.2 2 /0.8 2 +0.2 2 )} = 0.978 0.8 4 + 4 0.8 4 0.2+10 0.8 4 0.2 2 +20 0.8 4 0.2 3 = 0.967 1 A I d, I nd I d = 0.978 log 2 0.978 0.022log 2 0.022 = 0.152 (2.15) I nd = 0.967 log 2 0.967 0.033 log 2 0.033 = 0.209 (2.16) A, B 1 a, b a = 0.9 b = 0.1
2.3. (2) 15 A 0.9 4 + 4 0.9 4 0.1+10 0.9 4 0.1 2 + 20 0.9 5 0.1 3 {1+(1 0.9 2 0.1 2 /0.9 2 +0.1 2 2)} = 0.999 0.9 4 + 4 0.9 4 0.1+10 0.9 4 0.1 2 +20 0.9 4 0.1 3 = 0.997 1 A I d, I nd I d = 0.999 log 2 0.999 0.001 log 2 0.001 = 0.011 (2.17) I nd = 0.997 log 2 0.997 0.003 log 2 0.003 = 0.029 (2.18) 2.1 2.1: m =0.6 0.806 0.869 m =0.7 0.466 0.546 m =0.8 0.152 0.209 m =0.9 0.011 0.029
17 3 3.1 (1) m =1.0 m =0.5 1.0 m A, B 3 m =0.3 m =0.5 m =0.3 m =0.5 75.3 (2) 1 A B 0 1 A B 2 3.2 (1975)
18
1975 2004 http://www.yobology.info/text/entropy/entropy.htm 19