IT 1 2 1 2 27 11 24 15:20 16:05 ( ) 27 11 24 1 / 49
1 1940 Witsenhausen 2 3 ( ) 27 11 24 2 / 49
1940 2 gun director Warren Weaver, NDRC (National Defence Research Committee) Final report D-2 project #2, Study of errors in T-10 gun director, 1945 there are surprisingly close and valid analogies between the fire control prediction probelm and certain basic problems in communiation engineering. Department of National Defence (CANADA) ( ) 27 11 24 3 / 49
1940 T-10 gun director 1941 2 T-10 gun director T-15 gun director Hendrik Bode NDRC (National Defence Research Committee) 2 MIT (Norbert Wiener) David A. Mindell, Automation s finest hour: Bell Labs and automatic control in World War II, IEEE Control Systems, 1995 ( ) 27 11 24 4 / 49
1940 Norbert Wiener (MIT) 1940 1942 2 Extrapolation, Interpolation, and Smoothing of Stationary Times Series with Engineering Applications 1942 12 10 20 Norman Levinson Claude E. Shannon 1948 Cybernetics Stuart Bennett, Nobert Wiener and control of anti-aircraft guns, IEEE Control Systems, 1994. ( ) 27 11 24 5 / 49
Wiener Shannon = = Nyquist Bode 1990 ( ) 27 11 24 6 / 49
Witsenhausen 2 Shannon ( ) 27 11 24 7 / 49
Witsenhausen x 1 = x 0 + u 1 x 2 = x 1 u 2 x 0 : N(0, σ 2) 0 0, σ 2, 0 v : N(0, 1) x 0 v y 0 = x 0 y 1 = x 1 + v u 1 = γ 1 (y 0 ) u 2 = γ 2 (y 1 ) E { k 2 u 2 + } 1 x2 2 k 1 2 ( ) 27 11 24 8 / 49
Witsenhausen x 1 = x 0 + u 1 x 2 = x 1 u 2 x 0 : N(0, σ 2) 0 0, σ 2, 0 v : N(0, 1) x 0 v y 0 = x 0 y 1 = x 1 + v u 1 = γ 1 (y 0 ) u 2 = γ 2 (y 1 ) E { k 2 u 2 + } 1 x2 2 k LQG 2 1 ( ) 27 11 24 8 / 49
x 1 = x 0 + u 1 y 0 = x 0 x 2 = x 1 u 2 y 1 = x 1 + v 0 u 1 = ay 0, u 2 = by 1 a, b u 1 = ay 0 x 1 = (1 + a) x 0 Ex 2 2 u 2 = E (x 1 y 1 ) = Ex 2 1 Ex 2 1 + Ev2 y 1 =: by 1 b = Ex 2 1 Ex 2 1 + Ev2 = (1 + a) 2 σ 2 0 (1 + a) 2 σ 2 0 + 1 } = k 2 a 2 σ 2 0 + (1 + a) 2 σ 2 0 E { k 2 u 2 1 + x2 2 (1 + a) 2 σ 2 0 + 1 a a, b ( ) 27 11 24 9 / 49
x 1 = x 0 + u 1 y 0 = x 0 x 2 = x 1 u 2 y 1 = x 1 + v (Witsenhausen 1968) u 1 = γ 1 (y 0 ) = y 0 + σ 0 sgn(y 0 ) u 2 = γ 2 (y 1 ) = σ 0 tanh(σ 0 y 1 ) E { k 2 u 2 + } 2 σ 1 x2 2 2k 2 σ 2 0 1 π + σ2 exp 0 2 0 2 ( ) 27 11 24 10 / 49
2 1.5 optimal linear cost UB nonlinear cost E { k 2 u 2 1 + x2 2 } cost 1 0.5 min a (1 + k2 a 2 σ 2 a)2 σ 2 0 + 0 (1 + a) 2 σ 2 + 1 0 0 0 0.1 0.2 0.3 k σ 0 = 6 Witsenhausen k = 0.2 0.9725 0.5821 2 σ 2k 2 σ 2 0 1 π + σ2 0 2 exp 0 2 ( ) 27 11 24 11 / 49
x t+1 = Ax t + Bu t y t = Cx t x 0 Λ 0 y t n t u t dt m t 2 R R ( ) 27 11 24 12 / 49
x t+1 = Ax t + Bu t y t = Cx t x 0 Λ 0 y t n t u t dt m t R max { 0, log 2 λ i } i [ S. Tatikonda, and S.K. Mitter: Control under communication constraints; 2004 ] ( ) 27 11 24 13 / 49
x t+1 = Ax t + Bu t, y t = Cx t, x 0 Λ 0 det A vol(λ 0 ) vol(λ 0 ) AΛ 0 + Bu 0 ( ) 27 11 24 14 / 49
x t+1 = Ax t + Bu t, y t = Cx t, x 0 Λ 0 det A vol(λ 0 ) vol(λ 0 ) AΛ 0 + Bu 0 ( ) 27 11 24 14 / 49
x t+1 = Ax t + Bu t, y t = Cx t, x 0 Λ 0 det A vol(λ 0 ) vol(λ 0 ) vol(λ 1 ) = 2 R det A vol(λ 0 ) AΛ 0 + Bu 0 Λ 1 ( ) 27 11 24 14 / 49
x t+1 = Ax t + Bu t, y t = Cx t, x 0 Λ 0 det A vol(λ 0 ) vol(λ 0 ) vol(λ 1 ) = 2 R det A vol(λ 0 ) AΛ 0 + Bu 0 Λ 1 vol(λ t ) = 2 Rt det A t vol(λ 0 ) R > log 2 det A = log 2 Π i λ i = log 2 λ i i ( ) 27 11 24 14 / 49
1 2 Bode 3 ( ) 27 11 24 15 / 49
A λ i Π λi >1 λ i Bode ( ) 27 11 24 16 / 49
x(t + 1) = f(x(t)) (n, ε) f : X X X d d n, f (x 1, x 2 ) = max 0 k n 1 d( f k (x 1 ), f k (x 2 )) ε > 0 K X F X x K z F d n, f (z, x) < ε F K (n, ε) K x f(x) F z F f(z) f 2 (x) f 3 (x) d n, f (z, x) f 3 (z) f 2 (z) f k = f f f } {{ } k times ( ) 27 11 24 17 / 49
x(t + 1) = f(x(t)) f : X X X d K (n, ε) F r(n, ε, K, f) 1 h top ( f) = sup lim lim sup ln r(n, ε, K, f) K Xcompact ε 0 n n f f(x) = Ax λ i >1 λ i [ R. Bowen: Entropy for group endomorphisms and homogeneous spaces; 1971 ] ( ) 27 11 24 18 / 49
dx(t) = f(x(t), u(t)), u U dt R n U = {u : u(t) U} U R n u Q R n x 0 Q u U x(t) Q ( ) 27 11 24 19 / 49
dx(t) = f(x(t), u(t)), u U dt K Q Q T, ε > 0 (K, Q) (T, ε) x 0 K Q u S U x(t), 0 t T Q ε S (K, Q) (T, ε) Q K Input u 1 S Input u 2 S ( ) 27 11 24 20 / 49
dx(t) = f(x(t), u(t)), u U dt K Q r inv (T, ε, K, Q): (K, Q) (T, ε) 1 h inv (K, Q) = lim lim sup ε 0 T T ln r inv(t, ε, K, Q) dx(t) = Ax(t) + Bu(t) dt K Q. Q. K Lebesgue. h inv (K, Q) = Re λ i Re λ i >0 [ F. Colonius, and C. Kawan: Invariance entropy for control systems; 2009 ] ( ) 27 11 24 21 / 49
A λ i d dt x = Ax e Ah x((k + 1)h) = e Ah x(kh) e Ah e λ ih e λ i h >1 e λ i t = Re λ i >0 e λ i t = exp h Re λ i >0 Re λ i ( ) 27 11 24 22 / 49
r + e C(z) P(z) y L(z) = P(z)C(z) S(z) = 1 1 + L(z) r e S(e jθ ) 1 y r ( ) 27 11 24 23 / 49
Bode r + e C(z) P(z) y L(z) = P(z)C(z) 1 S(z) = 1 + L(z) L(z) L(z) p i, i = 1,..., n Bode 1 π ln S(e jθ ) dθ = 2π π n ln p i i=1 ( ) 27 11 24 24 / 49
Bode Bode 1 π ln S(e jθ ) dθ = 2π π n ln p i S(e jθ ) 1 i=1 ( ) 27 11 24 25 / 49
x p x (x) Shannon x h (x) = p x (x) log 2 p x (x)dx x, y p x y=y h (x y = y) ω h (x y) = Eh (x y = y(ω)) x y I (x; y) = h (x) h (x y) ( ) 27 11 24 26 / 49
t x t = (x 0, x 1,..., x t ): x t, y t I(x T y T ) = T I 1 (x t ; y t y t 1 ) t=0 ( ) 27 11 24 27 / 49
u t + w t + y t p i : u t y t 1 lim T T I(uT y T ) = 1 2π π π ln S(e jθ ) dθ = ln p i [ N. Elia: When Bode meets Shannon: control-oriented feedback communication schemes; 2004 ] i Bode Shannon ( ) 27 11 24 28 / 49
1 2 3 ( ) 27 11 24 29 / 49
w t xt u t + w t x t x t ( ) 27 11 24 30 / 49
w = (w 0, w 1,..., w t,...) Gauss w t : N(0, δ 2 ), Gauss δ 0 δ w t : [ δ, δ] δ 0 δ ( ) 27 11 24 31 / 49
Gauss lim sup E (x t ) 2 γ 2 t MS magnitude γ 2 Time lim sup ess sup w x t γ t Magnitude 0 γ γ ( ) 27 11 24 32 / 49 Time
u t + x t ( ) 27 11 24 33 / 49
0 1 Rényi h 0 (x) = log µs x h 1 (x) = p x (ξ) log p x (ξ)dξ (Shannon ) δ 0 δ [ δ, δ] h 0 (x) = log 2 (2δ) δ 0 δ Gauss N(0, δ 2 ) h 1 (x) = log 2 ( 2πeδ ) ( ) 27 11 24 34 / 49
y x y 2 h 0(x y) h 0 (x y) = ess sup h 0 (x y = y(ω)) h 1 (x y) = Eh 1 (x y = y(ω)) (Shannon ) 2 h 0(x) 0 x ( ) 27 11 24 35 / 49
x y I r (x; y) = h r (x) h r (x y) y r = 1 : Shannon r = 0 : 2 h 0(x y) 2 h 0(x) x 2 I 0(x;y) = 2h 0(x) 2 h 0(x y) ( ) 27 11 24 36 / 49
0 1 x, y x y 2 (1+r)h r(x) + 2 (1+r)h r(y) 2 (1+r)h r(x+y), r = 0, 1. x 0 ess sup x δ h 0 (x) log 2 (2δ) 1 Ex 2 δ 2 h 1 (x) 1 2 log 2 2πeδ 2 ( ) 27 11 24 37 / 49
u t + x t (MI) I r ( xt ; u t u t 1) R (AI) 1 T 1 ( lim sup I r xt ; u t u t 1) R T T t=0 (MI) (AI) u = (u 0, u 1,..., u t,...), u t 1 = (u 0, u 1,... u t 1 ) ( ) 27 11 24 38 / 49
m t m t = n t, S mt <, n t mt S mt (MC) S mt 2 R (AAC) (GAC) lim sup T lim sup T 1 T 1 T T 1 S mt 2 R t=0 T 1 log Smt 2 R t=0 (GAC) (MC) (AAC) (GAC) ( ) 27 11 24 39 / 49
m t n t m t = n t + v t, n t v t N(0, ε 2 ) (MC) (AAC) (GAC) Em 2 t ε 2 lim sup T lim sup T 2 2R 1 T 1 T T 1 t=0 T 1 t=0 Em 2 t ε 2 2 2R 1 Em 2 2 log t 2 ε 2 R (GAC) (MC) (AAC) (GAC) ( ) 27 11 24 40 / 49
u t + m t Smt x t (MI) I r ( xt ; u t u t 1) R (AI) lim sup T T 1 1 T t=0 I r ( xt ; u t u t 1) R (MC) Smt 2 R (AAC) lim sup T (GAC) lim sup T 1 T T 1 S mt 2 R t=0 T 1 1 T t=0 log 2 Smt R (MC) (AAC) (GAC) (MI) (AI) Gauss ( ) 27 11 24 41 / 49
w t xt u t + x t+1 = ax t + w t + u t x 0 γ 0, w t δ w t ( x 0, w t 1, u t) I 0 ( xt ; u t u t 1) R lim sup t x t γ ( ) 27 11 24 42 / 49
w t xt u t + Gauss x t+1 = ax t + w t + u t x 0 N(0, γ 2 0 ), w t N(0, δ 2 ) w t ( x 0, w t 1, u t) I 1 (x t ; u t u t 1 ) R lim sup t E x t 2 γ 2 ( ) 27 11 24 42 / 49
a 0 R γ u γ > δ, R log 2 a 1 (δ/γ), ( ) γ > δ, R 1 2 log 2 a 2, (Gauss ) 2 1 (δ/γ) [ H. Shingin, and Y. Ohta: Disturbance rejection with information constraints; 2012 ] ( ) 27 11 24 43 / 49
a 0 R γ u γ > δ, R log 2 a 1 (δ/γ), ( ) γ > δ, R 1 2 log 2 a 2, (Gauss ) 2 1 (δ/γ) [ H. Shingin, and Y. Ohta: Disturbance rejection with information constraints; 2012 ] R a δ γ ( ) 27 11 24 43 / 49
a 0 R γ u γ > δ, R log 2 a 1 (δ/γ), ( ) γ > δ, R 1 2 log 2 a 2, (Gauss ) 2 1 (δ/γ) [ H. Shingin, and Y. Ohta: Disturbance rejection with information constraints; 2012 ] R a δ γ T 1 ( lim sup I 0 xt ; u t u t 1) R T t=0 ( ) 27 11 24 43 / 49
a 0 R γ u γ > δ, R log 2 a 1 (δ/γ), ( ) γ > δ, R 1 2 log 2 a 2, (Gauss ) 2 1 (δ/γ) [ H. Shingin, and Y. Ohta: Disturbance rejection with information constraints; 2012 ] Gauss ( ) 27 11 24 43 / 49
a 0 R γ u γ > δ, R log 2 a 1 (δ/γ), ( ) γ > δ, R 1 2 log 2 a 2, (Gauss ) 2 1 (δ/γ) [ H. Shingin, and Y. Ohta: Disturbance rejection with information constraints; 2012 ] Gauss Gauss Rényi ( ) 27 11 24 43 / 49
2 1 2 ( ) 27 11 24 44 / 49
w t u t xt + I 0 ( xt ; u t u t 1) R T 1 1 ( lim sup I 0 xt ; u t u t 1) R T T t=0 x t+1 = Ax t + w t + u t, det A 0 x 0 γ 0 w t δ w t ( x 0, w t 1, u t) lim sup t x t γ R R n ( ) 27 11 24 45 / 49
x t+1 = Ax t + w t + u t x 0 γ 0 w t δ w t ( x 0, w t 1, u t) lim sup t x t γ ( I 0 xt ; u t u t 1) R 1 T 1 ( lim sup I 0 xt ; u t u t 1) R T T t=0 R n γ α R n log 1 δ γ α = max n k a j=1 k j [ : ; 2010 ] ( ) 27 11 24 46 / 49
x t+1 = Ax t + w t + u t x 0 γ 0 w t δ w t ( x 0, w t 1, u t) lim sup t x t γ ( I 0 xt ; u t u t 1) R 1 T 1 ( lim sup I 0 xt ; u t u t 1) R T T t=0 R n [Nair et al, 2007] γ det A R log ( ) 1 δ n γ [ G.N. Nair et al: Feedback control under data rate constraints: an overview; 2007 ] ( ) 27 11 24 47 / 49
α det A n log log 1 δ ( ) n γ 1 δ γ n = 1 x t+1 = Ax t + w t + u t B = I C = I ê u = Aê ( ) 27 11 24 48 / 49
1940 Wiener, Bode MIT, Bell Shannon, Nyquist 1990 Witsenhausen Shannon Bode ( ) 27 11 24 49 / 49