2012 : DP(1) 24 5 6 1 (Dynamic Programming) (Dynamic Programming) Bellman Stokey and Lucas with Prescott(1987) 1.1 max {c t,k t+1 } T o T β t U (c t ) (1) subject to c t + k t+1 = Af (k t ), (2) k 0 given, t k t 0, 0 < β < 1. (2) max {k t+1 } T o T β t U (Af (k t ) k t+1 ) (3) 1
k 0 given, k t 0. 1 β t U (Af (k t ) k t+1 ) ( 1)+β t+1 U (Af (k t+1 ) k t+2 ) Af (k t+1 ) = 0 (4) t=1,2,3,...,t-1 t=t terminal point k T +1 = 0 2 T k 1 k T T T (1) closed form closed fomr U (c t ) = ln c t (5) Af (k t ) = Ak α t. 0 < α < 1 (6) αak α 1 t+1 β t 1 ( 1) + β t+1 = 0, (7) Akt α k t+1 Akt+1 α k t+2 k T +1 = 0. (8) X t = k t+1 Ak α t (9) 1 2? 2
(7) k t+1 X t+1 = 1 + αβ αβ X t (10) k T +1 = 0 X T = 0 X T 1 = X T 1 = αβ 1 + αβ αβ 1 + αβ X T (11) = αβ 1 αβ 1 (αβ) 2. (12) X T 2 = αβ 1 + αβ αβ 1+αβ (1 + αβ) = αβ 1 + αβ + (αβ) 2 = αβ (1 + αβ) (1 αβ) (1 αβ) ( 1 + αβ + (αβ) 2). (13) X T 2 = αβ 1 (αβ)2 1 (αβ) 3. (14) t 1 (αβ)t X t = αβ (15) 1 (αβ) T t+1 terminal condition (=T) T 1.2 Bellman Equation and Policy Function T = max {k t+1 } o β t ln (Akt α k t+1 ). (16) terminal point X T = 0 3
(15) T αβ 3 t X t αβ k t+1 = αβak α t (17) (17) k t+1 = h (k t ), t = 0, 1, 2,... (18) h (k t ) Policy Function Policy Function {k t+1 } Policy Function k 0 (16) V (k 0 ) = max {k t+1 } β t ln (Akt α k t+1 ) (19) V (k 0 ) k 0 V (k 1 ) = max {k t+1 } t=1 β t 1 ln (Akt α k t+1 ) (20) t=1 V (k 1 ) k 1 max k 1 [ln (Ak α 0 k 1 ) + βv (k 1 )] (21) V (k 1 ) k 1 = h (k 0 ) V (k 0 ) k 0 3 0 < αβ < 1 V (k 0 ) = max k 1 [ln (Ak α 0 k 1 ) + βv (k 1 )] (22) 4
t V (k t ) = max k t+1 [ln (Ak α t k t+1 ) + βv (k t+1 )] (23) index V (k) = max [ln (Ak α k ) + βv (k )] (24) k (24) Bellman V (k) Value Function Value Function Policy Function Policy Function 1.3 Ljungqivist and Sargent (2004) return function r transition function g max {u t } β t r (x t, u t ) (25) x t+1 = g (x t, u t ) (26) x 0 : given. (27) V V = max {u t } β t r (x t, u t ) s.t.x t+1 = g (x t, u t ) and x 0 (28) Bellman V (x t ) = max (r (x, u) + βv (x )) s.t.x = g (x, u). (29) u (1)Bellman V (2) (3) 5
V? Bellman V V Bellman V Bellman Bellman V Stokey and Lucas with Prescott(1987), Chapter 4, Section 2 (1) r (2) { (x t+1, x t ) : x t+1 g (x t, u t ), u t R k} (1) (2) Bellman V 1. V (x) 2. V (x) Policy function single-valued function 3. V (x) V 4. Bellman V iteration V j+1 = max (r (x, u) + βv j (x )) s.t.x = g (x, u). (30) u 5. iteration Value Function V (x) V (x) = r g (x, h (x)) + β x x (x, h (x)) V (g (x, h (x))). (31) 5 Benveniste and Scheinkman r g Value Function Policy Function Value Function Benveniste and Scheinkman (24) Benveniste and Scheinkman V (k) = αakα 1 Ak α k. (32) 6
0 = 1 Ak α k + βv (k ) (33) V (k) k k 1 Ak α k + β αak α 1 = 0. (34) Ak α k DP L = β t ln (Akt α k t+1 ), Aαkα 1 t β Akt α k t+1 1 Ak α t 1 k t = 0 Benveniste and Scheinkman (2) (1) (1) Hall 2 Bellman Stokey and Lucas with Prescott(1987) 4.4 Value Function Value Function 4. iteration Value Function Iteration Dynamic Programming 7
1.4 Value Function Iteration (30) j=0 V 1 = max (r (x, u) + βv 0 (x )) s.t.x = g (x, u) (35) u V 0 V 0 V 0 V 1 j=1 V 2 = max (r (x, u) + βv 1 (x )) s.t.x = g (x, u) (36) u V j V j+1 Policy Function h j (x) h j+1 (x) iteration Value Function Policy Function Value Function Iteration V 0 = 0 j=0 k = 0 V 1 = max ln (Ak α k ) + β 0. (37) k V 1 = ln Ak α = ln A + α ln k (38) j=1 V 1 V 2 = max ln (Ak α k ) + β (ln A + α ln k ) (39) k 1 Ak α k + βα 1 k = 0 (40) k = αβ 1 + αβ Akα (41) 8
(( V 2 = ln 1 αβ ) ) Ak α + β 1 + αβ ( ln A + α ln ) αβ 1 + αβ Akα (42) V 2 = ln A 1 + αβ + β ln A + αβ ln αβ 1 + αβ + ( α + α 2 β ) ln k (43) j=2 V 3 = max ln (Ak α k ) + β ( Const1 + ( α + α 2 β ) ln k ) (44) k V 3 1 Ak α k + β ( α + α 2 β ) 1 = 0 (45) k k = αβ (1 + αβ) 1 + αβ (1 + αβ) Akα (46) V 3 = Const2 + α ( 1 + αβ + (αβ) 2) ln k (47) V (k) = 1 ( ( ln A (1 αβ) + αβ )) ln (Aαβ) + α 1 β 1 αβ 1 αβ ln k (48) Value Function Policy Function k = αβk α (49) Value Function lnk Value Function Value Function 9
1.5 Guess and Verify DP Guess and Verify Value Function Guess Bellman Equation Verify 4 Value Function Closed Form (16) Guess and Verify Value Function Guess V (k) = E + F ln k (50) E F Value Function Bellman Equation E + F ln k = max ln (Ak α k ) + β (E + F ln k ) (51) k Verify 0 = 1 Ak α k + βf k (52) k = βf Akα 1 + βf Bellman Equation (53) E + F ln k = ln A 1 + βf E = ln + α ln k + βe + βf ln βf A 1 + βf A 1 + βf E,F + αβf ln k (54) F = α + αβf, (55) F = + βe + βf ln βf A 1 + βf. (56) α 1 αβ (57) 4 Long and Plosser(1983) Guess and Verify Dynamic Programming 10
E = 1 ( A ln 1 β 1 + βf ) βf A + βf ln 1 + βf (58) E F Bellman Equation Guess Guess 1.6 Policy Function Iteration Value Function Policy Function iterate Value Function Howard s Policy Improvement Algorithm 1. Policy Function u = h 0 (x 0 ) (59) 2. Policy Function Value Function V 0 (x 0 ) = max {u t } β t r (x t, h 0 (x t )) s.t.x t+1 = g (x t, h 0 (x t )) and x 0. (60) 3. Value Function Policy Function max (r (x, u) + βv 0 (x )) s.t.x = g (x, u) (61) u 4. 1. Policy Function 11
(16) Policy Function Iteration 1. Policy Function 1/2 k t+1 = 1 2 Akα t (62) 2. Value Function ( V 0 (k 0 ) = β t ln Akt α 1 ) 2 Akα t (63) = = ( ) 1 β t ln 2 Akα t ( ) ) 1 β (ln t 2 A + α ln k t (64) (65) k t = 1 2 Akα t 1 (66) = ( ) 1+α 1 A 1+α kt 2 α2 (67) 2 k t = ln D + α t k 0 (68) D ( ) ) 1 V 0 (k 0 ) = β (ln t 2 A + α ln D + α t+1 ln k 0 (69) V 0 (k 0 ) = const + α 1 αβ ln k 0. (70) 12
3. Value Function Bellman Equation ( V 0 (k) = max ln (Ak α k ) + β const + α ) ln k k 1 αβ (71) 1 Ak α k + αβ 1 αβ 1 k = 0 (72) k = αβak α (73) iteration Polify Function Iteration Value Function Policy Policy Value Function 0.0001 Value Function 0.00001 Policy Policy Value Value Policy Value Policy Policy Value 13