II (2011 ) ( ) α β û i R

Transcription

1 II α β 3 û i 4 R F 5 7 F 6 8 GLS 8 8 heil and Goldberger Model 9 MLE 9 9 I 3 93 II 3 94 AR i

2 Wald, LM, LR AR MA 5 84 ARMA ARIMA SARIMA , Identification ARCH Unit Root 6 9 Cointegration 69 GMM Generalized Mothod of Moments ii

3 X, Y, X, Y,, X n, Y n n X i Y i Y i α + βx i, X i Y i α, β α β {X i, Y i, i,,, n} i i α, β Y i α βx i, X i Y i α βx i, Y i n α + β X i, 3 X i Y i α X i + β Xi, n Y i n n X n iy i X i a b d b c d ad bc c a α, β α β n X i n X i α β, i i i n n X i n X i n X i n Y i n X iy i α β Sα, β Sα, β min Sα, β α,β u i Y i α βx i α, β α, β Sα, β α Sα, β β α, β α, β n n X i n X i n X i n n X i n X i n Y i n X iy i β β n n X iy i n X i n Y i n n X i n X i n X iy i nxy n X i nx n X i XY i Y n X i X 3 α Y βx X X i, n Y n Y i,

4 Y i α + βx i α β α β i Y i X i α β n β X iy i nxy n X i nx α Y βx X Y n Xi X i Y i i Y i X i X i Y i Xi Y i Xi Xi Y i X i Y X β α α, β α, β α, β Ŷ i α + βx i, Ŷ i X i Y i i Y i X i X i Y i Xi Ŷ i Y i Xi Xi Y i X i Ŷi Y X Ŷ i Y i X i Ŷi 5 5 X i Ŷ i Y i û i Y i Ŷi, û i Y i Ŷi + û i α + βx i + û i, Y Y i Y Ŷi Y + û i, 3 û i û i Y i α βx i û i, X i û i,

5 Ŷ i α + βx i Ŷ i û i, Ŷ i û i α + βx i û i α û i + β X i û i i Y i X i Ŷ i û i X i û i Ŷ i û i Y i Xi Ŷi ûi Xi û i Ŷi û i R Y i Y Ŷi Y + û i, Y i Y Ŷi Y + û i Ŷi Y + Ŷi Y û i + Ŷi Y + û i Y i Y Ŷi Y + n Ŷi Y n n Y i Y + û i n Y i Y û i û i 3 Y i Y y Ŷi Y Ŷi û i Ŷi R n R Ŷi Y n Y i Y R n û i n Y i Y, Y i Ŷi + û i Ŷi Y Ŷi Y Y i Y û i Ŷi Y Y i Y Ŷi Y Y i Y n R Ŷi Y n Y i Y Ŷi Y û i n Ŷi Y n Y i Y n Ŷi Y n Ŷi Y Y i Y n Y i Y n Ŷi Y R Y i Ŷi Y i Y Ŷi Y + R, û i 3

6 R t 9 y t β x t + β x t + + β k x kt + u t, t,,, u t R n û i n Y i Y n û i n Y i ny û i Y i α + βx i Y Y i i Y i X i Ŷ i û i û i Yi Y i Xi Ŷi ûi û i Y i û i 3 X 3 Y 875 R 5 α β Y i n β X iy i nxy n X i nx α Y βx X Y n Xi X i Y i R n û i n Y i Y û i Y n n û i n Y i ny Y i a u t iid, σ b x t, x t,, x kt u t c x t, x t,, x kt, y t d β u t iid N, σ 3 x t β 4 Sβ, β,, β k y t β x t β x t β k x kt t Sβ, β,, β k β, β,, β k β, β,, β k Sβ, β,, β k β t x t y t β x t β x t β k x kt t Sβ, β,, β k β t x t y t β x t β x t β k x kt t Sβ, β,, β k β kt x kt y t β x t β x t β k x kt t 4

7 xt y t β x t + β xt x t + + β k xt x kt xt y t β xt x t + β x t y x β + u y x β + u + + β k xt x kt xkt y t β xkt x t + β xkt x t + + β k x kt x t xt x t xt x kt xt x t x t xt x kt xkt x t xkt x t x kt β xt y t β xt y t β k xkt y t 5 y t β x t + β x t + + β k x kt + u t β β x t x t x kt x t β + u t x t x t x t x kt, β β β β k, y x β + u β k + u t y y y y Xβ + u y X u y y y x x x u u u, x x x, β + u u u 6 x t : k, β: k, y:, X: k, u: 5

8 3 A :, B : n m, C : m k, D : k n, tra a ii, where A [a ij ] If A is idempotent, A A A A 3 A is idempotent if and only if the eigen values of A consist of and 4 If A is idempotent, ranka tra 5 trbcd trcdb 3 a, x :, y : K, A :, B : K a x x x a x a a x x a x Ax x A + A x x Ax x x A + A x By B tr A A log A A xy I 3 a, x, y, µ :, Σ, A, B :, σ : scalar A If x Nµ, Σ, then a x Na µ, a Σa If x Nµ, Σ, then x µ Σ x µ χ 3 x: n, y: m x Nµ x, Σ x, y Nµ y, Σ y, x y E x µ x y µ y x µ x Σ x x µ x /n y µ y Σ F n, m y y µ y /m 4 If x N, σ I and A is a symmetric idempotent matrix of rank G, then x Ax/σ χ G 5 If x N, σ I, A and B are symmetric idempotent matrices of rank G and K, and AB, then x Ax / x Bx Gσ Kσ x Ax/G x Bx/K 4 y :, X : k, β : k, u :, u u, u,, u t u t u u t F G, K Regression model: y Xβ + u, u, σ I min β y Xβ y Xβ u u y Xβ y Xβ y y β X y + β X Xβ u u β X y + X Xβ β β β X X X y 6

9 3 u u β β X X c Xd d c c d X Xd > * a x x Ax > A positive definite matrix b x x Ax < A positive definite matrix 4 β X X X y X X X Xβ + u β + X X X u 5 β X X X y y 6 E β β + X X X Eu β 7 V β E β β β β E X X X ux X X u X X X Euu XX X σ X X 8 u N, σ I β Nβ, σ X X β β X X β β χ k σ * x k x Nµ, Σ x µ Σ x µ χ k 9 Properties of β : BLUE best linear unbiased estimator, ie, Unbiased and efficient estimator in linear class Gauss-Markov theorem β Cy β Cy CXβ + u CXβ + Cu E β CXβ + CEu CXβ β Cy E β β CX I β Cy β CXβ + u β + Cu V β E β β β β ECuu C σ CC D C X X X V β σ CC σ D + X X X D + X X X CX I D + X X X X DX + I DX 7

10 V β σ CC σ D + X X X D + X X X σ X X + σ DD V β + σ DD V β V β β σ s s k y X β y X β, because and y X β y XX X X y I XX X X y I XX X X Xβ + u I XX X X u E k y X β y X β k E I XX X X u I XX X X u k E u I XX X X I XX X X u k E u I XX X X u k E tru I XX X X u k tr I XX X X Euu k σ tr I XX X X k σ k σ A: k k tra k a ii a ii A i j a: tra a 3 A: k, B: k trab trba 4 tr[xx X X ] tr[x X X X] tri k 5 X: E[trX] tr[ex] F H : β If u N, σ I, then β Nβ, σ X X herefore, β β X X β β σ a β β X X β β σ χ k β β X X X u β β X X β β χ k X X X u X XX X X u u XX X X XX X X u u XX X X u XX X X idempotent, ie, A A A u XX X X u σ χ tr XX X X 8

11 tr XX X X tr X X X X tri k k u XX X X u σ b * χ k x N, I k x Ax χ RankA A idempotent, ie, A A A RankA tra x Ax χ tra u N, σ I σ u N, I Rank û y X β y XX X X y I XX X X y I XX X X Xβ + u I XX X X Xβ + I XX X X u X XX X X X β + I XX X X u I XX X X u I û û XX X X u I XX X X u u I XX X X I XX X X u u I XX X X u û û σ u I XX X X u σ χ tr I XX X X tr I XX X X tri tr XX X X tri tr X X X X tri tri k k û û σ ks σ χ k s k û û 3 β û [ ] u N, σ I 9

12 Covû, β Covû, β E û β β I E XX X X u X X X u I E XX X X uu XX X I XX X X Euu XX X I XX X X σ IXX X σ I XX X X XX X σ XX X XX X X XX X σ XX X XX X β û 4 β β X X β β σ û û σ χ k χ k, β β X X β β û û a y y y y y y y y y ỵ y y ii y I ii y y i,,, 4 R û û y I ii y F H : β R y t x t β + u t y t β + x t β + u t y t y x t x β + u t β β X X β β / σ k û û / σ k R R R t û t t y t y t û t û û 3 t y t y y I ii I ii y y I ii y F k, k x t x t, x t,, x kt x t, x t, x t, x x t,, x kt, β β β β k β β, y t x t β + u t y Xβ + u y t y x t x β + u t y X β + u û t û t [ ] t ût,

13 t y t β β x t β k x kt, t y t β x t β, y β + x β, y t β + x t β + û t y t y x t x β + û t 3 y t y x t x β + u t H : β y X β + u 4 β X X β /k û û / k y y β X X β + û û y I ii y û û F k, k β X X β /k û û / k y I ii y û û /k û û/ k F k, k 5 y Xβ + u R R û û y I ii y 6 F β X X β /k û û / k y I ii y û û /k û û/ k R /k R F k, k / k y Xβ +u H : β F If u N, σ I, then β Nβ, σ X X Consider testing the hypothesis H : Rβ r R : G k, rankr G k R β NRβ, σ RX X R herefore, R β r RX X R R β r χ G σ Note that Rβ r a β Nβ, σ X X ER β RE β Rβ b β Nβ, σ X X VR β E R β RβR β Rβ E R β β β β R RE β β β β R RV βr σ RX X R We have the following: R β r RX X R R β r G y X β y X β k F G, k 3

14 a t G, r, R,,,, R i β i s y X β y X β k R β r RX X R R β r G s β i s a ii F, k R β β i a ii X X i i β i β i s a ii t k * Recall that t k F, k b { x t β + u t, t,,, n y t x t β + u t, t n +, n +,, u i N, σ y x y x y n x y n n+ x n+ y n+ x n+ y x β β Y X β Y X Y Xβ + u β + + u u u u n u n+ u n+ u H : β β R I I, r F G rankr k, β k F k, k c R,,,,, r G rankr F, k d y α + α D + α D + α 3 D 3 + Xβ + u D i i α α α 3 β α, α, α, α 3, β k R, r G rankr 3, β k F 3, k e Q t K t L t logq t β + β logk t + β 3 logl t + u t, β + β 3 H : β + β 3 H : β + β 3

15 R, r f n n + Y t α + βx t + γd t + δd t X t + u t, {, t,,, n d t, t n +, n +,, n + H : γ δ H : γ δ R, r g Y t α + βx t + γz t + u t, X t Z t Y t H : β γ H : β γ R, r Define û as û y X β he coefficient of determinant, R, is R û û y My, where M I ii, I is a identity matrix and i is a vector consisting of, ie, i,,, Note that y y y y My y y M is idempotent DurbinWatson ratio, DW, is defined as t DW û t û t t û t where û û, û,, û û Aû û û, A 5 Rβ r y Xβ y Xβ L L y Xβ y Xβ λ Rβ r β, λ β, λ L β X y Xβ R λ L λ Rβ r L/ β β X X X y + X X R λ β + X X R λ Rβ R β + RX X R λ 3

16 r R β + RX X R λ λ RX X R r R β β β + X X R RX X R r R β a Eβ E β + X X R RX X R r RE β β + X X R RX X R r Rβ β b β β β β + X X R RX X R Rβ R β β β X X R RX X R R β Rβ β β X X R RX X R R β β [I X X R RX X R R] β β W β β Vβ E β ββ β E W β β β β W W E β β β β W W V βw σ W X X W σ I X X R RX X R R X X I X X R RX X R R σ I X X R RX X R R X X I R RX X R RX X σ X X X X R RX X R RX X I R RX X R RX X σ X X X X R RX X R RX X X X R RX X R RX X +X X R RX X R RX X R RX X R RX X σ X X X X R RX X R RX X σ X X σ X X R RX X R RX X V β σ X X R RX X R RX X a L β X y Xβ R λ L λ Rβ r X Xβ R λ X y Rβ r X X R β X y R λ r β X X R X y λ R r 4

17 b A B E F B D F G E, F, G E A BD B A + A BD B A B B A F A BD B BD A BD B A B G D B A B D + D B A BD B BD c E, F E X X X X R RX X R RX X F X X R RX X R β EX y + F r β β + X X R RX X R r R β d β X V λ σ X R R Eβ σ E Vβ σ X X X X R RX X R RX X e Rβ r V β Vβ σ X X R RX X R RX X 6 F R β r RX X R R β r G y X β y X β k F G, k R β r RX X R R β r β β X X β β β β + X X R RX X R r R β y Xβ y Xβ y Xβ β X β y Xβ β X β y X β y X β + β β X Xβ β y X β Xβ β β β X y X β y X β y X β + β β X Xβ β X û β β X Xβ β y Xβ y Xβ y X β y X β 5

18 û u û y X β u y Xβ R β r RX X R R β r G y X β y X β k β β X Xβ β G y X β y X β k y Xβ y Xβ y X β y X β G y X β y X β k u u û û/g û û/ k 7 F cons99txt PROGRAM LINE ************************************************ freq a; smpl ; 3 readfile cons99txt year cons yd price; 4 rconscons/price/; 5 rydyd/price/; 6 d; 7 smpl ; 8 d; 9 smpl ; drydd*ryd; dconsrcons-rcons-; olsq rcons c ryd; 3 olsq rcons c d ryd dryd; 4 olsq rcons c ryd rcons-; 5 olsq dcons c; 6 end; EXECUION ****************************************************** Equation Method of estimation Ordinary Least Squares Dependent variable: RCONS Current sample: 956 to 997 Number of observations: 4 Mean of dependent variable 4938 Std dev of dependent var Sum of squared residuals 795E+ Variance of residuals 39878E+8 Std error of regression R-squared Adjusted R-squared Durbin-Watson statistic 6873 F-statistic zero slopes Schwarz Bayes Info Crit 74 6

19 Log of likelihood function Estimated Standard Variable Coefficient Error t-statistic C RYD E Equation Method of estimation Ordinary Least Squares Dependent variable: RCONS Current sample: 956 to 997 Number of observations: 4 Mean of dependent variable 4938 Std dev of dependent var Sum of squared residuals 445E+9 Variance of residuals 64343E+7 Std error of regression R-squared 9994 Adjusted R-squared Durbin-Watson statistic 4979 F-statistic zero slopes 959 Schwarz Bayes Info Crit 5933 Log of likelihood function Estimated Standard Variable Coefficient Error t-statistic C D RYD DRYD Equation 3 Method of estimation Ordinary Least Squares Dependent variable: RCONS Current sample: 956 to 997 Number of observations: 4 Mean of dependent variable 4938 Std dev of dependent var Sum of squared residuals 465E+9 Variance of residuals 6396E+7 Std error of regression 556 R-squared 9997 Adjusted R-squared Durbin-Watson statistic 547 Durbin s h 665 Durbin s h alternative F-statistic zero slopes 98 Schwarz Bayes Info Crit 585 Log of likelihood function Estimated Standard Variable Coefficient Error t-statistic C RYD RCONS Equation 4 Method of estimation Ordinary Least Squares Dependent variable: DCONS Current sample: 956 to 997 Number of observations: 4 Mean of dependent variable Std dev of dependent var 7348 Sum of squared residuals 36647E+9 Variance of residuals 74799E+7 Std error of regression 7348 R-squared 369E-49 Adjusted R-squared Durbin-Watson statistic 387 Schwarz Bayes Info Crit 5895 Log of likelihood function Estimated Standard Variable Coefficient Error t-statistic C ****************************************************** Equation vs Equation 974 Equation RCONS β + β D + β 3 RYD + β 4 RYD D H : β β 4 OLS Equation OLS Equation u u û û/g û û/ k 795E + 445E + 9/ 445E + 9/ F, 38 F, 38 % 5 < 843 H : β β

20 Equation vs Equation 3 Equation 3 RCONS β + β RYD + β 3 RCONS H : β 3 OLS Equation OLS Equation 3 u u û û/g û û/ k 795E + 465E + 9/ 465E + 9/ F, 39 F, 39 % 7333 < 6368 H : β 3 RCONS- RCONS RCONS- t-statistic 3 Equation 3 vs Equation 4 Equation 3 RCONS β + β RYD + β 3 RCONS H : β and β 3 OLS Equation 4 OLS Equation 3 u u û û/g û û/ k 36647E E + 9/ 465E + 9/ F, 39 F, 39 % 594 > 49 H : β and β 3 8 GLS Regression model: y Xβ + u, u, σ Ω Heteroscedasticity σ σ Ω σ σ First-Order Autocorrelation u t ρu t + ɛ t, ɛ t iid N, σ ɛ σ Ω σ ɛ ρ Vu t σ ρ ρ ρ ρ ρ ρ ρ ρ ρ 3 ρ ρ ρ 3 σ ɛ ρ 3 β GLS b min β GLSE of β is b y Xβ Ω y Xβ b X Ω X X Ω y 4 Ω Ω Ω A ΛA Λ A Ω Λ x x Ωx > Ω 5 here exists P shuch that Ω P P take P Λ / A Multiply P on both sides of y Xβ + u 8

21 We have: y X β + u, where y P y, X P X, and u P u Note that Vu VP u P VuP σ P ΩP I, because Ω P P Accordingly, the regression model is rewritten as: y X β + u, u, σ I Apply OLS to the above model hat is, min β is equivalent to: min β y X β y X β y Xβ Ω y Xβ b X X X y X Ω X X Ω y b β + X X X u β + X X X Ω u Eb β Vb σ X X σ X Ω X 6 y Xβ + u, u N, σ Ω, β X X X y β + X X X u V β σ X X X ΩXX X GLS OLS a E β β Eb β β b b V β σ X X X ΩXX X Vb σ X Ω X V β Vb σ X X X ΩXX X σ X Ω X σ X X X X Ω X X Ω Ω X X X X Ω X X Ω σ AΩA Ω u Ω I AΩA b β 7 If u N, σ Ω, then b Nβ, σ X Ω X Consider testing the hypothesis H : Rβ r R : G k, rankr G k R β NRβ, σ RX Ω X R herefore, R β r RX Ω X R R β r σ χ G 9

22 8 9 We have: y Xb Ω y Xb σ χ k R β r RX Ω X R R β r G y Xb Ω y Xb k F G, k b b e e Ω X V β X R Ψ R X Ω X + R Ψ R 9 MLE Maximum Likelihood Estimation MLE he distribution function of {x i } is fx; θ, where x x,, x and θ µ, Σ Likelihood function L is defined as Lθ; x fx; θ Maximum likelihood estimate MLE of θ is θ such that: e y Xb, e y Xb max θ Lθ; x F e Ω e e Ω e/g e Ω e/ k F G, k 8 heil and Goldberger Model r Rβ + v, v, Ψ y Xβ + u, u, Ω y X r R u v β + Ω, Ψ u v Ω X β X R Ψ R Ω y X R Ψ r X Ω X + R Ψ R X Ω y + R Ψ r MLE satisfies the following: a log Lθ; x θ b log Lθ; x θ θ is a negative definite matrix Fisher s information matrix is defined as: log Lθ; x Iθ E θ θ Note as follows: log Lθ; x E θ θ log Lθ; x log Lθ; x E θ θ log Lθ; x V θ θ Lθ; xdx Lθ; x dx θ x θ L/ θ

23 log Lθ; x Lθ; xdx θ log Lθ; x E θ θ log Lθ; x θ θ Lθ; xdx log Lθ; x Lθ; x + θ dx θ log Lθ; x θ θ Lθ; xdx log Lθ; x log Lθ; x + θ θ Lθ; xdx log Lθ; x E θ θ log Lθ; x log Lθ; x E θ θ log Lθ; x V θ 3 Cramer-Rao Iθ: θ sx Esx sxlθ; xdx θ Esx Lθ; x sx dx θ θ log Lθ; x sx Lθ; xdx θ log Lθ; x Cov sx, θ sx θ Esx θ log Lθ; x Cov sx, θ log Lθ; x ρ V sx V θ log Lθ; x V sx V θ log Lθ; x ρ sx θ ρ log Lθ; x Cov sx, θ ρ log Lθ; x V sx V θ Esx log Lθ; x Vsx V θ θ Esx θ Vsx log Lθ; x V θ Esx θ Vsx Iθ log Lθ; x E θ sx Vsx Iθ 4 Iθ θ θ N, lim θ N θ, Iθ

24 sx θ V sx Iθ 5 log Lθ; x θ log Lθ ; x θ + log Lθ ; x θ θ θ θ θ θ log Lθ ; x log Lθ ; x θ θ θ Replace the variables as follows: θ θ i+ θ θ i hen, we have: θ i+ θ i log Lθ i ; x log Lθ i ; x θ θ θ log Lθ i ; x θ θ log Lθ i ; x E θ θ θ i+ θ i log Lθ i ; x log Lθ i ; x E θ θ θ log Lθ θ i Iθ i i ; x θ 9 y t α + βx t + u t, u t u t N, σ u t fu t exp πσ σ u t u, u,, u fu, u,, u fu fu fu exp πσ / σ t 3 y,, y u t y t α βx t fy, y,, y exp πσ / σ Lα, β, σ y, y,, y u t y t α βx t t Lα, β, σ y, y,, y log Lα, β, σ y, y,, y log Lα, β, σ y, y,, y logπ logσ σ y t α βx t t 4 x f x x x gz z f z z f z z f x gz dgz dz

25 x U, z logx f x x 9 I Regression model: y Xβ + u, u N, σ I Log-likelihood function is: x exp z f z z dx dz f x gz exp z exp z 5 y, y,, y Lα, β, σ y, y,, y log Lα, β, σ y, y,, y α, β, σ α, β, σ α, β, σ log Lα, β, σ y, y,, y α σ y t α βx t t log Lα, β, σ y, y,, y β σ y t α βx t x t t log Lα, β, σ y, y,, y σ σ + σ 4 y t α βx t t t β x t xy t y t x t x α y βx σ y t α βx t t OLS ML σ log Lθ; y, X logπσ where θ β, σ max θ log Lθ; y, X log Lθ; y, X θ σ y Xβ y Xβ, We obtain MLE of β and σ : β X X X y, σ y X β y X β 3 Fisher s information matrix is defined as: log Lθ; y, X Iθ E θ θ he inverse of the information matrix, Iθ, provides a lower bound of the variance - covariance matrix for unbiased estimators of θ σ X X Iθ σ 4 93 II Regression model: y Xβ + u, u N, σ Ω Log-likelihood function is: log Lθ; y, X logπσ log Ω σ y Xβ Ω y Xβ, where θ β, σ 3

26 max θ log Lθ; y, X log Lθ; y, X θ We obtain MLE of β and σ : β X Ω X X Ω y, σ y X β Ω y X β 3 Fisher s information matrix is defined as: log Lθ; y, X Iθ E θ θ he inverse of the information matrix, Iθ, provides a lower bound of the variance - covariance matrix for unbiased estimators of θ σ X Ω X Iθ σ 4 94 AR AR t, 3,,, φ < y t φ y t + u t, u t N, σ y, y,, y fy, y,, y fy, y,, y fy fy t y t,, y t fy t y t,, y y t φ y t +u t Ey t y t,, y φ y t, Vy t y t,, y σ fy t y t,, y exp πσ σ y t φ y t fy t y t,, y Ey t y t,, y Vy t y t,, y fy t y t,, y fy t Ey t, y t φ y t + u t φ y t + u t + φ u t φ t y + u t + φ u t + + φ t u u t + φ u t + φ u t + Vy t σ + φ + φ 4 + φ, fy t πσ / φ exp σ / φ y t y, y,, y fy, y,, y fy, y,, y fy fy t y t,, y t t πσ / φ exp σ / φ y exp πσ σ y t φ y t σ Lφ, σ ; y, y,, y logπσ / φ σ / φ y logπσ σ y t φ y t t < ρ < grid search 4

27 95 y t x t β + u t u t ρu t + ɛ t ɛ t iid N, σ ɛ u, u,, u f u ; log f u u, u,, u ; ρ, σɛ log fu ; ρ, σɛ + log fu t u t,, u ; ρ, σɛ t logπσ ɛ / ρ σɛ / ρ u logπσ ɛ σ ɛ u t ρu t t u, u,, u y, y,, y Lρ, σ ɛ, β; y, y,, y log f y y, y,, y ; ρ, σ ɛ, β log f u y x β, y x β,, y x β; ρ, σɛ u y log πσɛ / ρ σɛ / ρ y x β logπσɛ σɛ y t ρy t x t ρx t β t logπσ ɛ + log ρ σɛ ρ y ρ x β logπσɛ σ ɛ y t ρy t x t ρx t β t logπσ ɛ + log ρ σɛ y x β σɛ yt x t β t logπ logσ ɛ + log ρ σ ɛ yt x t β t yt, x t { yt ρ y t, for t, y t ρy t, for t, 3,,, { x ρ x t, for t, t x t ρx t, for t, 3,,, β Lρ, σ ɛ, β; y, y,, y β x t x t x t yt t t X X X y y X β + ɛ, ɛ N, σ ɛ I σ ɛ Lρ, σ ɛ, β; y, y,, y σ yt x t β t y X β y X β y ρ y y y y ρy x y x x x y ρy ρ x x ρx x ρx ρ Lρ, σ ɛ, β; y, y,, y max Lρ, β,σɛ,ρ σ ɛ, β; y 5

28 max Lρ, σ ɛ, β; y ρ Lρ, σ ɛ, β; y concentrated loglikelihood function Lρ, σ ɛ, β; y logπ log σ ɛ + log ρ logπ σ log ɛ ρ + log ρ ρ σ ɛ σ ɛ ρ 96 y t x t β + u t u t iid N, σ t σ t z t α u, u,, u f u ; log f u u, u,, u ; σt log f u u t ; σt t logπ logσ t logπ logz tα t ut t σ t ut z t α u, u,, u y, y,, y Lσ t, β; y, y,, y log f y y, y,, y ; σ t, β log f u y x β, y x β,, y x β; σt u y logπ logz tα yt x t β z t α β, α : t x, x, F, F, lim F t F t x, x, F convergence in distribution a {z :,, } z θ convergence in probability lim Prob z θ < ɛ, ɛ θ z limit plim z θ, probability b θ θ θ θ θ θ consistent estimator 3 Chebyshev gx Prob gx k E gx k 6

29 k gx k U gx < k U U gx ku E gx keu EU EU Prob gx k + Prob gx < k P gx k E gx kprob gx k Prob gx k E gx 4 X gx X µ X µ EX µ VarX Σ Prob X µ X µ k trσ k E X µ X µ E tr X µ X µ E tr x µx µ tr E x µx µ trσ k 5 X i µ, σ, i,,, X µ Chebyshev gx X µ, ɛ k E gx VX σ P X µ ɛ σ ɛ ɛ 6 lim P X µ < ɛ x y plim x c, plim y d a plim x + y c + d b plim x y cd c plim x /y c/d, d d plim gx gc, g Slutsky s heorem 7 Lidberg-Levy Central Limit heorem x, x,, x x t µ Σ independent and identically distributed, iid t x t µ N, Σ 8 Central Limit heorem Greenberg and Webster 983 7

30 x, x,, x x t µ Σ t t Σ lim x t µ N, Σ Σ t t 9 : θ θ θ θ N, Σ θ Nθ, Σ/ : 3 θ consistent uniformly asymptotically normal a θ b θ θ N, Σ c : θ θ consistent, uniformly, asymptotically normal Σ/, Ω/ Ω Σ positive semidefinite θ θ asymptotically efficient : consistent, uniformly, asymptotically normal estimator asymptotically efficient consistent, uniformly, asymptotically normal estimator asymptotically efficient 3 asymptotically efficient consistent, uniformly, asymptotically normal estimator 4 x, x,, x fx; θ θ θ regularity conditions θ θ θ θ Iθ N, lim 5 Regularity Conditions a x t θ b fx; θ θ 3 6 i consistent ii asymptotically normal iii asymptotically efficient, 7 Slutsky s heorem θ θ g θ gθ g well-defined continuous function 8 Invariance of Maximum Likelihood Estimation θ, θ,, θ k θ, θ,, θ k α α θ, θ,, θ k, α α θ, θ,, θ k,, α k α k θ, θ,, θ k α, α,, α k α α θ, θ,, θ k, α α θ, θ,, θ k,, α k α k θ, θ,, θ k β X X X y β β X X M xx X u 8

31 Chebyshev s inequality gx Prob gx k E gx k x X u, gx x x E X u X u E u XX u E tr u XX u E tr XX uu tr XX Euu σ trxx σ trx X σ tr X X k Prob X u X u k σ k tr X X X X M xx X u X u X u trm xx 3 X X M xx X X M xx Slutsky s heorem * Slutsky s heorem θ θ g θ gθ 4 β β + X X X u β + X X X u β β + M xx β 5 i ii iii 6 Asymptotic Normality of OLSE a Central Limit heorem: Webster 983 Greenberg and z, z,, z µ Σ t t Σ lim z t µ N, Σ Σ t t z t b z t x t u t Σ t σ x tx t c Σ Σ lim σ x tx t t σ lim X X σ M xx 9

32 d Central Limit heorem Greenberg and Webster 983 t x tu t X u N, σ M xx β β X X X u β β N, σ M xx u t Errors in Variables a X X plim X V plim X X plim X X + plim V V Σ + Ω b u X u X plim V u plim X u 6 β β β + X X X u V β β + X X X + V u V β y Xβ + u X X + V V : Measurement Errors 3 X V 4 plim β β Σ + Ω Ωβ 7 y t α + β x t + u t x t x t + v t y Xβ + u V β β β X X X y β + X X X u V β 5 Σ plim X X plim xt µ µ µ + σ xt x t 3

33 µ, σ x t Ω plim V V plim v t σv plim α β α β µ µ µ + σ + σv α σv β α µσ v β β σ + σv σvβ β σ vβ plim β β σ + σv β + σv/σ Instrumental Variable IV y Xβ + u, u N, σ I EX u β β + X X X u β + M xx M xu 3 Z u M zu X Z Z Z y Z Xβ + Z u plim plim Z y β plim + plim Z u plim Z X β Z X β plim Z X plim Z y β IV Z X Z y β 4 5 β IV β IV Z X M zx Z Z M zz Z u Z X Z y Z X Z Xβ + u β + Z X Z u X X M xx X u M xu βiv β Z X Z u 3

34 Central Limit heorem Z u N, σ M zz β IV Z X Z u N, σ M zx M zz M zx 6 Central Limit heorem: Greenberg and Webster 983 z, z,, z t Σ lim µ Σ t z t µ N, Σ Σ t t 7 β IV Vβ IV s Z X Z ZX Z s y Xβ IV y Xβ IV k 3 y Xβ + u, u N, σ I EX u Z u M zu X Z 3 Z X X X W X W B + V B X W B B W W W X X W W W W X Z 4 β IV X X X y X W W W W X X W W W W y β IV β + X W W W W X X W W W W u βiv β X W X W W W W W XW N, M xz M zz M xz 5 W u plim W u W u 3

35 6 X X X W W W W X c h θ R h θ N, R θ Iθ θ X W W W W W W W W X X X d χ R h θ R θ Iθ h θ θ χ G β IV X X X y X X X y I θ Iθ convergence in probability R h θ R θ I θ h θ χ θ G 3 3 Wald, LM, LR θ : K hθ : G vector function, G K θ : K he null hypothesis H : hθ θ : k, restricted maximum likelihood estimate θ : k, unrestricted maximum likelihood estimate Iθ : K K, information matrix log Lθ Iθ E θ θ log Lθ : log-likelihood function R θ hθ θ : G K log Lθ F θ : K θ Wald : W h θ R θ I θ R θ h θ a hθ h θ + h θ θ θ θ h θ h θ θ θ θ R θ θ θ b θ Iθ θ θ N, lim θ θ N, Iθ Lagrange Multiplier : LM F θ Iθ F θ a hθ max θ log Lθ, subject to hθ L log Lθ + λhθ b c L log Lθ θ θ L λ hθ + λ hθ θ log Lθ θ log Lθ E, θ log Lθ log Lθ V E θ θ θ Iθ d e log Lθ θ N, Iθ log Lθ log Lθ Iθ θ θ θ F θ Iθ Fθ χ G χ G θ 33

36 3 Likelihood Ratio : LR log λ χ G λ Lθ L θ For proof, see heil 97, p396 4 All of W, LM and LR are asymptotically distributed as χ G random variables under the null hypothesis H : hθ 5 Under some comditions, we have W LR LM See Engle 98 Wald, Likelihood and Lagrange Multiplier ests in Econometrics, Chap 3 in Handbook of Econometrics, Vol, Grilliches and Intriligator eds, North-Holland 3 cons99txt PROGRAM LINE *********************************************** freq a; smpl ; 3 readfile cons99txt year cons yd price; 4 rconscons/price/; 5 rydyd/price/; 6 lydlogryd; 7 olsq rcons c ryd; 8 ar rcons c ryd; 9 olsq rcons c lyd; param a a a3 ; frml eq rconsa+a*ryd**a3-/a3; lsqtol,maxit eq; 3 a35; 4 rrydryd**a3-/a3; 5 ar rcons c rryd; 6 end; EXECUION ***************************************************** Equation Method of estimation Ordinary Least Squares Dependent variable: RCONS Current sample: 955 to 997 Number of observations: 43 Mean of dependent variable 467 Std dev of dependent var 7937 Sum of squared residuals 9697E+ Variance of residuals 36335E+8 Std error of regression R-squared 9959 Adjusted R-squared Durbin-Watson statistic 5 F-statistic zero slopes 839 Schwarz Bayes Info Crit 7397 Log of likelihood function Estimated Standard Variable Coefficient Error t-statistic C RYD E Equation FIRS-ORDER SERIAL CORRELAION OF HE ERROR MAXIMUM LIKELIHOOD IERAIVE ECHNIQUE 34

37 CONVERGENCE ACHIEVED AFER 7 IERAIONS EQUAIONS: EQ Dependent variable: RCONS Current sample: 955 to 997 Number of observations: 43 Statistics based on transformed data Mean of dependent variable 3685 Std dev of dependent var 5495 Sum of squared residuals 4587E+9 Variance of residuals 35567E+7 Std error of regression 8858 R-squared 8854 Adjusted R-squared 8836 Durbin-Watson statistic 3875 Rho autocorrelation coef 9454 Standard error of rho 4839 t-statistic for rho 345 F-statistic zero slopes 356 Log of likelihood function Statistics based on original data Mean of dependent variable 467 Std dev of dependent var 7937 Sum of squared residuals 4586E+9 Variance of residuals 35567E+7 Std error of regression R-squared Adjusted R-squared Durbin-Watson statistic 3874 Estimated Standard Variable Coefficient Error t-statistic C RYD CONVERGENCE ACHIEVED AFER 7 IERAIONS Log of Likelihood Function Number of Observations 43 Standard Parameter Estimate Error t-statistic A A A Standard Errors computed from quadratic form of analytic first derivatives Gauss Dependent variable: RCONS Mean of dependent variable 467 Std dev of dependent var 7937 Sum of squared residuals 593E+9 Variance of residuals 47553E+8 Std error of regression 3847 R-squared Adjusted R-squared Durbin-Watson statistic 5334 Equation 4 FIRS-ORDER SERIAL CORRELAION OF HE ERROR MAXIMUM LIKELIHOOD IERAIVE ECHNIQUE Equation 3 Method of estimation Ordinary Least Squares CONVERGENCE ACHIEVED AFER Dependent variable: RCONS Current sample: 955 to 997 Number of observations: 43 5 IERAIONS Dependent variable: RCONS Current sample: 955 to 997 Number of observations: 43 Mean of dependent variable 467 Std dev of dependent var 7937 Sum of squared residuals 564E+ Variance of residuals 64487E+9 Std error of regression R-squared 93 Adjusted R-squared 9737 Durbin-Watson statistic 975 F-statistic zero slopes 387 Schwarz Bayes Info Crit 3798 Log of likelihood function Estimated Standard Variable Coefficient Error t-statistic C -58E LYD NONLINEAR LEAS SQUARES Statistics based on transformed data Mean of dependent variable 339 Std dev of dependent var 437 Sum of squared residuals 3778E+9 Variance of residuals E+7 Std error of regression 8375 R-squared 9784 Adjusted R-squared Durbin-Watson statistic Rho autocorrelation coef Standard error of rho 663 t-statistic for rho 366 F-statistic zero slopes 3994 Log of likelihood function Statistics based on original data Mean of dependent variable 467 Std dev of dependent var 7937 Sum of squared residuals 439E+9 Variance of residuals 3447E+7 Std error of regression 8545 R-squared Adjusted R-squared Durbin-Watson statistic Estimated Standard 35

38 Variable Coefficient Error t-statistic C RRYD E ***************************************************** Equation vs Equation Equation H : ρ RCONS t β + β RYD t + u t, u t ρu t + ɛ t, ɛ t iid N, σ ɛ MLE Equation MLE Equation log Lβ, σ ɛ, ρ logπ logσ ɛ + log ρ σ ɛ RCONS t β CONS t β RYD t t RCONS t CONS t RYD t { ρ RCONS t, t RCONS t ρrcons t, { ρ, t t, 3,, ρ, t, 3,, { ρ RYD t, t RYD t ρryd t, t, 3,, ρ MLE Equation max log Lβ, σ β,σɛ ɛ, β, σ ɛ Log of likelihood function ρ MLE Equation max log Lβ, β,σɛ,ρ σ ɛ, ρ β, σ ɛ, ρ Log of likelihood function logλ L β, σ ɛ, log L β, σ ɛ, ρ log L β, σ ɛ, log L β, σ ɛ, ρ logλ χ G G G logλ χ chi % > 6635 ρ Equation vs NONLINEAR LEAS SQUARES NONLINEAR LEAS SQUARES RCONS t a + a RYDa3 t + u t a3 a3 RCONS t a a + aryd t + u t Equation H : a3 G 36

39 a3 MLE Equation Log of likelihood function a3 MLE NONLINEAR LEAS SQUARES Log of likelihood function logλ logλ χ chi % > 6635 a3 3 Equation 3 vs NONLINEAR LEAS SQUARES NONLINEAR LEAS SQUARES RCONS t a + a RYDa3 t + u t a3 a3 RCONS t a + a logryd t + u t Equation 3 H : a3 G a3 MLE Equation 3 Log of likelihood function a3 MLE NONLINEAR LEAS SQUARES Log of likelihood function logλ χ chi % > 6635 a3 4 Equation vs Equation 4 Equation 4 RCONS t a + a RYDa3 t + u t, a3 u t ρu t + ɛ t, ɛ t iid N, σɛ H : a3, ρ MLE Equation MLE Equation 4 PROGRAM 3 5 a3 a3 5 logλ logλ χ chi % > 9 a3 ρ logλ

40 4 y t α + βx t + u t x t y t u t u, u,, u σ z t u t σ z t y t α + βx t + u t y t z t α z t + β x t z t + u t z t α z t + β x t z t + u t u t σ Eu ut t E Eu t z t u t Eu t Vu ut t V Vu t σ z t z t z t u t Vu t σ z t y t,, x t z t z t z t û t γz t + ɛ t γ γ t z t u t σ x t x t y t x t α x t + β + u t x t α x t + β + u t β y t x t β + u t, u t N, σt σ σ Ω σ σ σ σ σ σ σ σ σ σ y t x t β + u t, u t N, σ t, σ t z t α L logπ logz tα yt x t β t z t α 3 y t x t β + u t, u t N, σ t, σ t σ x t β p 4 y t x t β + u t, u t N, σ t, σ t expz t α 5 ARCH autoregressive conditional heteroscedasticity y t x t β + u t, u t N, σ t, σ t α + α u t α >, α σ 38

41 5 DW u, u,, u DW u t u t u, u,, u u, u,, u u, u,, u u, u,, u t u t+ u, u,, u y t α + βx t + u t, u t ρu t + ɛ t, H : ρ, H : ρ ɛ, ɛ,, ɛ t DW û t û t t û t t ûtû t t û t ρ, ρ û t û t u t ρu t + ɛ t u t, u t û t, û t ρ ρ DW ρ DW DW 3 DW k k DW t DW û t û t t û t t û t t ûtû t + t û t t û t û t 4 t û t û + û t ûtû t t û t t û t ρ, û + û t û t, t ûtû t t û t t ûtû t t t û t + û 39

42 : 5 % û t 5 t k A B C D E dl dl du du 4 du 4 du 4 dl 4 dl k A B C D E dl dl du du 4 du 4 du 4 dl 4 dl Y t α + βx t + u t, u t ρu t + ɛ t, ɛ, ɛ,, ɛ u t Y t ρy t α ρ + βx t ρx t + ɛ t, Y t Y t ρy t, X t X t ρx t Y t α + βx t + ɛ t, ɛ, ɛ,, ɛ α α ρ Y t β X t + β X t + + β k X kt + u t, u t ρu t + ɛ t, ɛ, ɛ,, ɛ 3 k 3 A B C D E dl dl du du 4 du 4 du 4 dl 4 dl k 4 A B C D E dl dl du du 4 du 4 du 4 dl 4 dl k 5 A B C D E dl dl du du 4 du 4 du 4 dl 4 dl A: B: C: D: E: 4

43 u t Y t ρy t β X t ρx,t : 5 % k k k 3 k 4 k 5 k 6 k 7 k 8 k 9 k k k k 3 n dl du dl du dl du dl du dl du dl du dl du dl du dl du dl du dl du dl du dl du n k clint/bench/dwcrithtm + β X t ρx,t + Y t Y t ρy t, X t X t ρx,t, X t X t ρx,t,, X kt X kt ρx k,t Y t + β k X t ρx k,t + ɛ t, β X t + β X t + + β k X kt + ɛ t ɛ, ɛ,, ɛ ρ ρ Y t β X t + β X t + + β k X kt + ɛ t, Y t Y t Y t, X t X t X,t,, X kt X kt X k,t DW DW ρ DW ρ ρ Y t Y t ρy t, X t X t ρx,t, X t X t ρx,t,, X kt X kt ρx k,t Y t β X t + β X t + + β k X kt + ɛ t, 3 4

44 i Y t β X t + β X t + + β k X kt + u t, û, û,, û ii û t ρû t + ɛ t, ρ iii Y t Y t ρy t, X t X t ρx,t, X t X t ρx,t,, X kt X kt ρx k,t Y t β X t + β X t + + β k X kt + ɛ t, β, β,, β k iv û t Y t β X t β X t β k X kt, û t û t ρû t + ɛ t, ρ v iii iv iv ρ H : ρ Wald y t x t β + u t, u t ρu t + ɛ t, ɛ t iid N, σ ɛ u t σ Ω σ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ + ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ ρ σ σ ɛ ρ ρ 3 u ɛ ρ ρ ρ ρ 4 u u u ɛ ɛ ɛ a h b H : ρ LM, LR, W y t α + βx t + γy t + u t, 4

45 x t y t β y t y t x t y t y t α + βx t + γy t + u t, y t y t α γ + β γ + γ u t, β γ x t y t y t x t y t y t y t x t 3 DW DW h h h ρ s γ, γ ρ s γ t ρ ûtû t û t t û t u t ρu t + ɛ t H : ρ H : ρ H : ρ h a h > z α/ % H b h < z α/ % H z α/ α % 6 omitting relevant variables including irrelevant variable y X β + X β + u, β : k, β : k β, β, s y X β + u β, σ omitting relevant variables y X β + X β + u y X β + u a β OLSE β X X X y X X X X β + X β + u β + X X X X β + X X X u Eβ β + X X X X β β 43

46 β β β β, σ, efficient and unbiased β, σ, biased β β, β, s, unbiased but inefficient β, β, s, efficient and unbiased including irrelevant variable y X β + u y X β + X β + u β X X + u Xβ + u β a β β, β OLSE β X X X y X X X X β + u X X X β X + u β + X X X u b β β E β c V β σ X X β X V σ β X X X X X X X d A B X Y B D Y Z X, Y, Z X A BD B A + A BD B A B B A Y A BD B BD A BD B A B Z D B A B D + D B A BD B BD e β V β σ X X X X X X X X σ X X V β Vβ f β 7 y Xβ + u, u N, σ I x x x k x x x k X a x x x x x x k + a x x x + + a k x k x k x k a, a,, a k β X X X y X X X X 44

47 3 6 Ridge y t x t β + x t β + u t, u t iidn, σ x t x t r r t x t x t x t x t t x t x t t x t x t t x tx t t x t t x t x t x t V β σ X X σ t x t t x tx t t x t t x t t x tx t t x tx t t x t t x tx t t x t t x t t x tx t t x t t x tx t t x tx t t x t t x t t x t r t x t t x tx t t x tx t t x t V β V β σ t x t r σ t x t r 4 x t x t r V β V β t 5 Ridge β X X X y Q ΛQ X y Λ Q X X X X > β R Q Λ + Q X y X X + Q Q X y X X + R X y X X X X + R Ridge 8 8 stationarity : y, y,, y a Ey t µ Ey t µy t τ µ γτ, τ,,, b y t, y t,, y tr r fy t, y t,, y tr fy t, y t,, y tr fy t +τ, y t +τ,, y tr +τ 45

48 3 Ey t µy t τ µ γτ, τ,,, γτ γ τ log Ly t y t,, y Ey t y t,, y Vy t y t,, y 4 ρτ γτ γ 5 µ t γτ y t ρτ γτ γ tτ+ 6 Lag Operator y t µy t τ µ L τ y t y t τ, τ,, 7 Innovation Form y, y,, y Ly,, y,, y Ly y,, y Ly,, y Ly y,, y Ly y,, y Ly,, y Ly Ly t y t,, y t log Ly, y,, y log Ly + log Ly t y t,, y t 8 AR AR : Autoregressive Model ARp y t φ y t + φ y t + + φ p y t p + ɛ t φly t ɛ t φl φ L φ L φ p L p φx p 3 y t y t k φ k,k y t,, y t k+ y t y t k φ, ρ ρ φ, ρ ρ ρ φ, ρ ρ ρ ρ ρ ρ φ 3, φ 3, φ 3,3 ρ ρ ρ3 46

49 ρ ρk ρk ρ ρk 3 ρk ρk ρk ρ φ k, φ ρ k, ρ φ k,k ρk φ k,k φ k,k ρ ρk ρ ρ ρk 3 ρ ρk ρk ρ ρk φ k,k ρ ρk ρk ρ ρk 3 ρk ρk ρk ρ 4 AR y t φ y t + ɛ t a φx φ x x /φ φ < b AR y t φ y t + ɛ t φ y t + ɛ t + φ ɛ t φ 3 y t 3 + ɛ t + φ ɛ t + φ ɛ t φ s y t s + ɛ t + φ ɛ t + + φ s ɛ t s+ s φ s c y t φ y t + ɛ t y t ɛ t + φ ɛ t + φ ɛ t + AR MA d AR µ Ey t Eɛ t + φ ɛ t + φ ɛ t + Eɛ t + φ Eɛ t + φ Eɛ t + e AR y t φ τ y t τ + ɛ t + φ ɛ t + + φ τ ɛ t τ+ γτ Ey t µy t τ µ Ey t y t τ E φ τ y t τ + ɛ t + φ ɛ t + + φ τ ɛ t τ+ y t τ φ τ Ey t τ y t τ + Eɛ t y t τ + φ Eɛ t y t τ + + φ τ Eɛ t τ+ y t τ φ τ γ ρτ γτ γ φ τ AR y t τ Ey t y t τ φ Ey t y t τ + Eɛ t y t τ { φ γτ, τ γτ φ γτ + σ, τ γτ γ τ τ γ φ γ + σ φ γ + σ σ γ φ 47

50 f AR φ, ρ φ ρ ρ ρ φ, ρ ρ g AR i ρ ρ ρ log Ly,, y log Ly + log Ly t y t,, y t logπ σ log φ σ / φ y logπ logσ σ t y t φ y t logπ logσ log φ σ / φ y σ y t φ y t t Ly πσ / φ exp σ / φ y Ly t y t,, y exp πσ σ y t φ y t log Ly,, y σ σ + σ 4 / φ y + σ 4 y t φ y t t log Ly,, y φ φ φ + φ σ y + σ y t φ y t y t t φ, σ σ φ y + y t φ y t t t φ y ty t t y t + φ y σ φ / φ t ii OLS Sφ y t φ y t t y t φ φ t y t y t t y t t φ + y t ɛ t t y t φ t + ɛ t + φ ɛ t 3 + φ ɛ t 3 + ɛ t t ɛ t + φ ɛ t + φ ɛ t 3 + φ φ + E ɛ t + φ ɛ t + φ ɛ t 3 + ɛ t Eɛ t + φ ɛ t + φ ɛ t 3 + OLSE y t ɛ t + φ ɛ t + φ ɛ t 3 + E ɛ t + φ ɛ t + φ ɛ t 3 + ɛ t yt t ɛ t + φ ɛ t + φ ɛ t + t 48

51 Eɛ t + φ ɛ t + φ ɛ t + σ ɛ φ iii φ y t ɛ t, t,,,, σ ɛ / φ / t y t ɛ t σɛ 4 / φ / t t y t ɛ t φ φ iv N, σ 4 ɛ N, φ y t Ey t σ ɛ φ / t y t ɛ t / t y t N, φ A x, x,, x µ σ x / t x t x µ σ/ N, B x, y y x xy y Nµ, σ, xy Nµ, c σ h AR +drift: y t µ + φ y t + ɛ t φly t µ + ɛ t x c φl φ L y t φl µ + φl ɛ t Ey t φl µ + φl Eɛ t φ µ µ φ 5 AR y t φ y t + φ y t + ɛ t a φx φ x φ x b φ L φ L y t ɛ t φx /α, /α AR α L α Ly t ɛ t y t α L α L ɛ t α /α α α L c AR µ Ey t EφLɛ t d AR γτ Ey t µy t τ µ + α /α α α L Ey t y t τ E φ y t + φ y t + ɛ t y t τ φ Ey t y t τ + φ Ey t y t τ ɛ t + Eɛ t y t τ φ γτ + φ γτ, τ φ γτ + φ γτ + σ, τ 49

52 γ φ γ + φ γ + σ γ φ γ + φ γ γ φ γ + φ γ φ σ γ + φ φ φ e τ γ φ γ+φ γ+σ f γ σ φ ρ φ ρ log Ly,, y log Ly, y + log Ly t y t,, y t3 Ly, y V / exp π y y V y y Ly t y t,, y exp πσ σ y t φ y t φ y t φ / φ V γ φ / φ g AR +drift: y t µ + φ y t + φ y t + ɛ t φly t µ + ɛ t φl φ L φ L y t φl µ + φl ɛ t Ey t φl µ + φl Eɛ t φ µ µ φ φ 6 ARp y t φ y t +φ y t + +φ p y t p +ɛ t a ARp γ b ARp : : σ φ ρ φ p ρp min y t φ y t φ y t φ p y t p φ,, φ p : log Ly,, y log Ly p,, y, y + log Ly t y t,, y tp+ Ly p,, y, y V / π y exp y y y p V y y p Ly t y t,, y exp πσ σ y t φ y t φ y t φ p y t p YuleWalker : φ y t + φ y t + + φ p y t p + ɛ t y t y t, y t,, y t p γ ρ ρp ρp ρ ρp 3 ρp ρp ρp ρ φ ρ φ ρ φ p ρp φ p 5

53 γτ ρτ γτ γ tτ+ y t µy t τ µ c ARp +drift: y t µ + φ y t + φ y t + φ p y t p + ɛ t φly t µ + ɛ t φl φ L φ L φ p L p y t φl µ + φl ɛ t Ey t φl µ + φl Eɛ t φ µ µ φ φ φ p d AR p k p +, p +, φ k,k AR, AR 83 MA MA Moveing Average Model MA q y t ɛ t + θ ɛ t + θ ɛ t + + θ q ɛ t q y t θlɛ t θl + θ L + θ L + + θ q L q θx + θ x + θ x + + θ q x q q AR 3 MA y t ɛ t + θ ɛ t a MA Ey t Eɛ t + θ ɛ t Eɛ t + θ Eɛ t b MA γ Eyt Eɛ t + θ ɛ t Eɛ t + θ ɛ t ɛ t + θɛ t Eɛ t + θ Eɛ t ɛ t + θeɛ t + θσ γ Ey t y t Eɛ t + θ ɛ t ɛ t + θ ɛ t θ σ γ Ey t y t Eɛ t + θ ɛ t ɛ t + θ ɛ t 3 θ ρτ + θ, τ, τ, 3, MA ρ c ɛ t θ ɛ t + y t θ ɛ t + y t + θ y t θ 3 ɛ t 3 + y t + θ y t + θ y t 5

54 θ s ɛ t s + y t + θ y t + θ y t + + θ t s+ y t s+ θ s ɛ t s MA AR y t θ y t θ y t d θ t s+ y t s+ + ɛ t γ + θ σ γ θ σ τ γτ y, y,, y fy, y,, y π V / / exp Y V Y Y y y y, + θ θ θ + θ θ V σ θ + θ θ θ + θ 4 MA +drift: y t µ + ɛ t + θ ɛ t y t µ + θlɛ t θl + θ L Ey t µ + θleɛ t µ 5 MA y t ɛ t + θ ɛ t + θ ɛ t a MA + θ + θσ τ θ + θ θ σ τ γτ θ σ τ b θx /β, /β β, β MA y t ɛ t + θ ɛ t + θ ɛ t + θ L + θ L ɛ t + β L + β Lɛ t MA AR c V σ ɛ t + β L + β L y t β /β β + β L β /β β + β L y t + θ + θ θ + θ θ θ θ + θ θ + θ + θ θ + θ θ θ θ + θ θ θ + θ + θ θ + θ θ θ θ + θ θ + θ + θ d MA +drift: y t µ + ɛ t + θ ɛ t + θ ɛ t y t µ + θlɛ t θl + θ L + θ L Ey t µ + θleɛ t µ 6 MA q y t ɛ t +θ ɛ t +θ ɛ t + +θ q ɛ t q a MA q Ey t Eɛ t + θ ɛ t + θ ɛ t + + θ q ɛ t q 5

55 b MA q σ θ θ τ + θ θ τ+ + + θ q τ θ q, γτ τ,,, q,, τ q +, q +,, q τ σ θ i θ τ+i, τ,,, q γτ i, τ q +, q +, θ c MA q d MA q +drift: y t µ + ɛ t + θ ɛ t + θ ɛ t + + θ q ɛ t q y t µ + θlɛ t θl + θ L + θ L + + θ q L q Ey t µ + θleɛ t µ 84 ARMA ARMA : Autoregressive Moving Average Model ARMA p, q y t φ y t + φ y t + + φ p y t p + ɛ t + θ ɛ t + θ ɛ t + + θ q ɛ t q φly t θlɛ t Y V 3 ARMA, : y t φ y t + ɛ t + θ ɛ t y t Ey t φ Ey t + Eɛ t + θ Eɛ t, 3 Ey t y t τ Ey t y t τ φ Ey t y t τ Ey t y t τ γτ + Eɛ t y t τ + θ Eɛ t y t τ Ey t y t τ γτ { σ, τ Eɛ t y t τ, τ,, φ + θ σ, τ Eɛ t y t τ σ, τ, τ, 3, γ φ γ + + φ θ + θ σ γ φ γ + θ σ γτ φ γτ, τ, 3, γ, γ φ γ φ γ + σ φ θ + θ θ γ γ σ φ + φ θ + θ φ θ σ φ + φ θ + θ φ φ θ σ + φ θ + θ φ + φ θ φ + θ 53

56 ρ + φ θ φ + θ + φ θ + θ ρτ φ ρτ 86 SARIMA SARIMA : Seasonal ARIMA Model SARIMA p, d, q φl d s y t θlɛ t s y t L s y t 85 ARIMA ARIMA : Autoregressive Integrated Moving Average Model ARIMA p, d, q φl d y t θlɛ t d y t d Ly t d y t d y t L d y t, d,, y t y t ARMA p, q +drift: y t µ + φ y t + φ y t + φ p y t p + ɛ t + θ ɛ t + θ ɛ t + + θ q ɛ t q φly t µ + θlɛ t φl φ L φ L φ p L p θl + θ L + θ L + + θ q L q y t φl µ + φl θlɛ t Ey t φl µ + φl θleɛ t φ µ µ φ φ φ p y t y t s s 4 s 87 AR p y t φ y t + + φ p y t p + ɛ t a Ey t+k I t y t+k t y t+k φ y t+k + + φ p y t+k p + ɛ t+k y t+k t φ y t+k t + + φ p y t+k p t s t y s t y s b MA q y t ɛ t + θ ɛ t + + θ q ɛ t q a ɛ, ɛ,, ɛ b y t ɛ t + θ ɛ t + + θ q ɛ t q y t+k ɛ t+k + θ ɛ t+k + + θ q ɛ t+k q c y t+k t ɛ t+k t + θ ɛ t+k t + + θ q ɛ t+k q t s > t ɛ s t s t ɛ s t ɛ s 3 ARMA p, q y t φ y t + + φ p y t p + ɛ t + θ ɛ t + + θ q ɛ t q 54

57 a y t+k φ y t+k + + φ p y t+k p + ɛ t+k + θ ɛ t+k + + θ q ɛ t+k q b y t+k t φ y t+k t + + φ p y t+k p t + ɛ t+k t + θ ɛ t+k t + + θ q ɛ t+k q t s t y s t y s ɛ s t ɛ s s > t ɛ s t 88, Identification d, s AIC SBIC p, q a AIC Akaike s Information Criterion AIC log s p + q + b SBIC Shwarz s Bayesian Information Criterion SBIC log s + p + q log ρτ, τ,,, p, d, q, s AR p MA q ρk, k q +, q +, φk, k, k p +, p +, a s y t s y t s L s b τ c AR τ d MA 55

58 ARIMA 985 IDEN RC Date: / ime: 7:3 SMPL range: Number of observations: 57 Autocorrelations Partial Autocorrelations ac pac ************* ************* ************ * ************ ** ************ *** ************ ****** *********** * *********** * *********** * *********** **** ********** * ********** * 79 6 ********** 786 ********** ** ********* * ********* ********* ********* ** ******** * ******** ******** 63-9 Box-Pierce Q-Stat 4636 Prob SE of Correlations 8 Ljung-Box Q-Stat 64 Prob DCRC-RC - IDEN DC Date: / ime: 7:3 SMPL range:

59 Number of observations: 56 Autocorrelations Partial Autocorrelations ac pac ***** ***** *** ****** ***** ************ ************ ******* ***** * *** * **** ** ************ ** ***** *** **** ************ * **** *** **** ** 5-3 *********** * **** *** **** * *********** ** 84-5 Box-Pierce Q-Stat 867 Prob SE of Correlations 8 Ljung-Box Q-Stat 9386 Prob SDCDC-DC -4 IDEN SDC Date: / ime: 7:4 SMPL range: Number of observations: 5 Autocorrelations Partial Autocorrelations ac pac *** *** * **** *** ******* ******

60 * ** * ** * * ** * ** * * * * * * * * * * * ** * Box-Pierce Q-Stat 7 Prob SE of Correlations 8 Ljung-Box Q-Stat 7439 Prob LS // Dependent Variable is SDC Date: / ime: 7:4 SMPL range: Number of observations: 5 Convergence achieved after 7 iterations VARIABLE COEFFICIEN SD ERROR -SA -AIL SIG MA R-squared Mean of dependent var Adjusted R-squared SD of dependent var 575 SE of regression Sum of squared resid Log likelihood -566 Durbin-Watson stat IDEN RESID Date: / ime: 7:4 SMPL range:

61 Number of observations: 5 Autocorrelations Partial Autocorrelations ac pac *** *** * ** ** * * * * ** ** * * * * * -5-9 * ** * * 7 58 * ** * * * * * * * * ** * 9 38 Box-Pierce Q-Stat 96 Prob 847 SE of Correlations 8 Ljung-Box Q-Stat 37 Prob 53 LS // Dependent Variable is SDC Date: / ime: 7:4 SMPL range: Number of observations: 5 Convergence achieved after 7 iterations VARIABLE COEFFICIEN SD ERROR -SA -AIL SIG MA MA R-squared 3397 Mean of dependent var

62 Adjusted R-squared SD of dependent var 575 SE of regression Sum of squared resid Log likelihood F-statistic Durbin-Watson stat Prob F-statistic IDEN RESID Date: / ime: 7:5 SMPL range: Number of observations: 5 Autocorrelations Partial Autocorrelations ac pac 6 6 ** ** * * * * * * ** * - -6 * * * ** 9 * * * * * * * * * * * * * * 5 7 Box-Pierce Q-Stat 53 Prob 768 SE of Correlations 8 Ljung-Box Q-Stat 664 Prob

63 89 : Frequency Domain fλ π τ γτ cos λτ π γτ exp iλτ τ y t fλ π σ 3 γτ π π fλ cos λτdλ 4 wj < s y t w j x t j j r f x λ x t W λ s W λ w j e iλj j r y t f y λ W λ f x λ W λ transfer function W λ W λw λ s s w j e iλj w j e iλj j r j r W λ W λ 5 MA q y t ɛ t + θ ɛ t + + θ q ɛ t q + θ L + + θ q L q ɛ t θlɛ t f y λ θe iλ θe iλ f ɛ λ θe iλ θe iλ σ π 6 AR p : φly t ɛ t y t φl ɛ t f y λ φe iλ φe iλ f ɛλ φe iλ φe iλ π 7 ARMA p, q : φly t θlɛ t y t φl θlɛ t σ f y λ θe iλ θe iλ φe iλ φe iλ f ɛλ σ θe iλ θe iλ φe iλ φe iλ π 8 ARCH Autoregressive Conditional Heteroskedasticity ARCH p ɛ t ɛ t, ɛ t,, ɛ N, h t h t α + α ɛ t + + α p ɛ t p ɛ t Unconditional Variance : α σ α α α p 6

64 GARCH p, q ɛ t ɛ t, ɛ t,, ɛ N, h t h t α + α ɛ t + + α p ɛ t p + β h t + + β q h t q 3 ARCH y t x t β + ɛ t, ɛ t ɛ t, ɛ t,, ɛ N, α + α ɛ t ɛ,, ɛ fɛ,, ɛ fɛ fɛ t ɛ t,, ɛ t / π / α α exp α / α ɛ π exp α + α ɛ t / t t ɛ t α + α ɛ t log Lβ, α, α ; y,, y logπ log α α α / α y x β logπ log α + α y t x t β t t y t x t β α + α y t x t β α, α, β α >, α > Eɛ t ɛ t, ɛ t,, ɛ α + αɛ t ARCH a y t x t β + u t OLS β û t y t x t β b û t α + α û t α ARCH LM 9 9 Unit Root JD Hamilton, 994 ime Series Analysis, Princeton University Press a X X b AR -consistent -consistent c y t a + a t + ɛ t y t b + y t + ɛ t k y t+ t a + a t + y t+k t b k + y t 6

65 φ < : y t φ y t + ɛ t, ɛ t iid N, σ, y, t,, φ OLSE φ y t y t t t y t φ < φ φ + y t ɛ t t t y t y t ɛ t t φ φ y t ɛ t t t s σ γ t y t y s ɛ t ɛ s y t γ φ φ σ N, γ N, φ φ, t t y t ɛ t y t σ γ φ 3 φ φ φ degenetate distribution 4 φ : y t y t + ɛ t, y y t ɛ t + ɛ t + ɛ t + + ɛ y t N, σ t a y t ɛ t y t y t + ɛ t y t + y t ɛ t + ɛ t y t ɛ t y t y t ɛ t y y t ɛ t y t t σ σ y t ɛ t t y σ σ y t N, σ t y χ σ ɛ t σ t ɛ t t ɛ t 63

66 σ y t ɛ t t y σ X σ X χ b yt E t t y t y t t t Ey t ɛ t σ t t σ 5 φ φ 6 a y t y t + ɛ t, y, ɛ t N, y t ɛ t + ɛ t + + ɛ y t N, t y s y t s > t y s y t ɛ s + ɛ s + + ɛ t+ + ɛ t+ y s y t N, s t b y t y t + e,t + e,t + + e N,t ɛ t e,t +e,t + +e N,t t t+ N N Standard Brownian Motion t W t t t + N, W t : Standard Brownian motion W W t, t [, ] i W ii r < r < < r k W r W r, W r 3 W r,, W r k W r k W s W t N, s t iii W r r Zt σw t N, σ t σ Brownian motion Zt W t t χ c X r r [, ], r < u, r < X r u + u, r < 3 u + u + + u, r X r [ r] u t t X r N, rσ [ r] r X r X r σ N, r r X W σ 64