09_toukei08.dvi

Size: px

Start display at page:

Download "09_toukei08.dvi"

もりよりこしの
5 years ago
Views:

1 c Cleridy E. Lennert-Cody Minami et al zero-inflated 2 1. Lambert, 1992 Frankenberg and Thomas, 2000; Lam et al., 2006 Welsh et al., 1996 zero-inflated Lambert, 1992 zero-inflated 2 Greene, 1994 Mullahy, Inter-American Tropical Tuna Commission, 8606 La Jolla Shores Dr. La Jolla, CA , U.S.A.

2 Lawless, 1987; Hilbe, 2007 μ 2 μ x g( ) g(μ)=x T β 2 ( ) θ ( ) y Γ(θ + y) θ μ (1.1) f NB(y μ,θ)= Γ(θ)Γ(y +1) θ + μ θ + μ μ θ (> 0) 2 θ θ θ (1.2) log(μ)=x T β Var(Y )=μ + 1 θ μ2 θ θ = θ =2 θ = R θ R = r Y m log(m)=β T x +logr Y f(y;x,β,θ) f(y;x,β,θ)= exp( re βt x ) ry e (β T x)y θ θ 0 y! Γ(θ) rθ 1 e θr dr ( ) θ ( ) y Γ(θ + y) θ e βt x = Γ(θ)Γ(y +1) θ + e βt x θ + e βt x 2 2 2

3 θ =2 θ = , 1, 2, 5 Minami et al Minami et al , Minami

4 ˆθ MM = ˆθ ML =0.318 zero-inflated 2 partial dependence 2 2 zero-inflated ˆθ MM = ˆθ ML =0.318 zero-inflated 2 partial dependence Hastie et al., 2009 A Partial dependence Maunder and Punt, A B x 2 W w i(i =1,...,2N) y i(i =1,...,2N) N N = [0,1] N u i (i =1,...,N) 2 x i =5.5u 2 i 2.5 (i =1,...,N) 3 μ A i,μb i (i =1,...,N) θ =0.6 2

5 2 275 w A i,wb i (i =1,...,N) log(μ A i )=x i +3 log(μ B i )=x i +1.5 w A i wb i (i =1,...,N) w i (i =1,...,2N) μ A i μb i (i =1,...,N) μ i I B(i) i B 1 0 μ i log(μ i)=x i I B(i) 4 p i ( ) pi log = 3x i 5+2.5I B(i) 1 p i w i p i 0 y i { 0 with probability p i y i = w i with probability 1 p i y i w i y i w i (i =1,...,2N) w y 2 R 2 MASS glm.nb mgcv Ver gam w y 2 A B

6 w, 2 y, 2 A 2 log(μ i)=β 0 + β xx i + β BI B(i) 2 w 2 (β 0 =3,β B = 1.5, β x =1 and θ =0.6) 2 y 2 B 2 A (ˆθ MLE,0.422) (ˆθ MME,0.305) = 2 w y i w i y i y i A w i A ŷ i ŵ i β x β B θ = y 2 w y zero-inflated 2 Greene, 1994 μ 2 p zero-inflated 2 μ p

7 p p 1 p 2 zero-inflated 2 (2.1) f(y b,g,β,γ,θ)= { p +(1 p)fnb(0 b,β,θ) for y =0 (1 p)f NB(y b,β,θ) for y =1,2,... f NB(y b,β,θ)= Γ(θ + y) ( θ ) θ ( μ ) y Γ(θ)Γ(y +1) θ + μ θ + μ log(μ) =β 0 + b 1β b kβ β kβ = b T β logit(p) =γ 0 + g 1γ g kγ γ kγ = g T γ b g 2 β γ y y Román-Verdesoto and Orozco-Zöller, 2005; Minami et al.,

8 Z 1,Z 2,Z 3,Z 4 Minami et al., 2007 Ȳ partial dependence Hastie et al., Minami et al zero-inflated 2 2 Zero-inflated Zero-inflated Zero-inflated AIC zero-inflated Zero-inflated 2 0 AIC zero-inflated 2 Vuong Vuong, zero-inflated 2 2 ˆθ ML =

9 ˆθ ML = ˆθ ML =0.318 ˆθ MM = zero-inflated Zero-inflated w w zeroinflated 2 x I B 0.6 x I B zero-inflated 2 2 zero-inflated Zero-inflated 2 2

10 p,μ 0,θ 0(0 <p<1,0 <μ 0,0 <θ 0) zero-inflated 2 μ,θ 2 n + 1 ˆθ MM θ (<θ) ( ) 1 θ p0 = θ 0 1+θ 0p 0 2 ˆθ ML θ <θ 0 θ C zero-inflated 2 zero-inflated 2 zero-inflated 2 θ ZINB =0.6 2 NB NB ˆθ MM =0.305 ˆθ ML =0.422 zero-inflated 2 ˆθ ZINB = NB NB ˆθ MM =0.157 ˆθ ML = Hampel et al., ( ) log f NB(β,θ y,b) θ =(y μ) b μ =exp(b T β) β θ + μ θ + log f Poi(β,θ y,b) =(y μ)b β 2 θ/(θ + μ) μ μ θ μ μ Williams, 1987; Chambers and Hastie,

11 ˆθ ML = zero-inflated ( ) (1 p i)μ i =exp{b T 1 (4.1) i β} 1+exp{g T i γ} (4.2) ( =exp{b T i β g T i γ} 1 1+exp{ g T i γ} 4.1 g T i γ p i 0 (1 p i)μ i exp{b T i β} 4.2 g T i γ p i 1 (1 p i)μ i exp{b T i β g T i γ} 2 b i = g i exp{b T i β g T i γ} =exp{b T i (β γ)}. )

12 A B x β γ Lambert 1992 zero-inflated β γ j β j γ j β j γ j β j p i 1 (1 p i)μ i exp { b T j (β γ) } 1 2 β j μ i p i 1 6 zero-inflated 2 y 2 A B x A μ = 1 exp{x +3} 1+exp{ 3x 5} μ =exp{1.111x } 2 μ =exp{1.414x } 2 x x x 2 A

13 Lambert 1992 zero-inflated γ = τβ ZIP τ 2 AIC 2 2 A. Partial dependence Partial dependence Hastie et al., 2009 X s X c x s,x c g(x s,x c) g(x s,x c) X s partial dependence g s(x s)=e Xc [g(x s,x c)] x 1c,x 2c,...,x nc ĝ 1 n s(x s)= g(x s,x ic) n i=1 B.

14 / f(y;η,φ)=exp{(yη b(η))/a(φ)+c(y,φ)} η μ g(μ)=η(μ) x β η(μ)=x T β b (η)=μ log f(y;η,φ) β =(xy xμ)/a(φ) x i,y i (i =1, 2,...,n) ˆμ i(i =1, 2,...,n) n n x iy i = x i ˆμ i i=1 L j j y i = ˆμ i i L j i L j i=1 C NB(μ,θ) zero-inflated 2 ZINB (p,μ 0,θ 0) μ =(1 p)μ 0 μ + 1 θ μ2 =(1 p)μ 0 +(1 p) (p + 1θ0 ) μ 2 0. μ θ μ θ ( ) μ =(1 p)μ 0 θ 1 p = θ 0. 1+pθ 0 (1 p)/(1 + pθ 0) < 1 θ <θ 0 2 ZINB (p,μ 0,θ 0), 0 <p<1, 0 <μ 0,0<θ NB(μ,θ) g(μ,θ) μ θ μ θ g(μ,θ) θ θ 0 g(μ,θ) E ZINB(p,μ0,θ 0 ) [log(f NB(Y μ,θ))] =(1 p)e NB(μ0,θ 0 ) [log (f NB(Y μ,θ))] + pθ [log(θ) log(θ + μ)]

15 2 285 =(1 p)e NB(μ0,θ 0 ) [log (f NB(Y μ 0,θ))] +(1 p)e NB(μ0,θ 0 ) [log(f NB(Y μ,θ)) log (f NB(Y μ 0,θ))] +pθ [log(θ) log(θ + μ)] h(μ,θ) h(μ,θ) =(1 p)e NB(μ0,θ 0 ) [log (f NB(Y μ,θ)) log (f NB(Y μ 0,θ))] +pθ [log(θ) log(θ + μ)] g(μ,θ)=(1 p)e NB(μ0,θ 0 ) [log(f NB(Y μ 0,θ))] + h(μ,θ) h(μ,θ) =pθlog(θ)+(1 p)(θ + μ 0)log(θ + μ 0) θ log(θ + μ) (1 p)μ 0 log(θ + μ)+(1 p)μ 0(log μ log μ 0) g(μ,θ) μ g(μ,θ) μ = h(μ,θ) μ 0 = (1 p)μ0 μ μ =(1 p)μ 0 θ +(1 p)μ0 θ + μ μ = μ μ θ θ 0 g(μ,θ) g(μ,θ 0) g(μ,θ) θ h(μ,θ) μ = μ < 0 θ=θ0 h(μ,θ)=pθlog(θ)+(1 p)(θ + μ 0)log(θ + μ 0) (θ + μ )log(θ + μ )+μ log(1 p) θ Jensen h(μ,θ) θ = p log(θ)+(1 p)log(θ + μ 0) log(θ + μ )log(pθ +(1 p)(θ + μ 0)) < log(pθ +(1 p)(θ + μ 0)) log(θ + μ ) =log(θ + μ ) log(θ + μ ) =0 h(μ,θ)/ θ h(μ,θ) E NB(μ0,θ 0 ) [log (f NB(Y μ 0,θ))] θ = θ 0 θ = θ 0 0 g(μ,θ) g(μ,θ 0) for θ θ 0 g(μ,θ) θ = h(μ,θ) θ=θ0 θ < 0. θ=θ0 g(μ,θ) (0, ) θ lim θ 0+ g(μ,θ)= g(μ,θ) θ θ (0, θ 0)

16 JSPS Chambers,J.M.andHastie,T.J Statistical Models in S, Wadsworth, Pacific Grove. Frankenberg, E. and Thomas, D The Indonesian Family Life Survey: Study Design and Results from Waves 1 and 2, RAND, Santa Monica. Greene, W Accounting for excess zeros and sample selection in Poisson and negative binomial regression models, Working Paper 94 10, Department of Econometrics, Stern School of Business, New York University, New York. Hampel, F. R., Ronchetti, E. M., Rousseuw, P. J. and Stahel, W. A Robust Statistics The Approach Based on Influence Functions, Wiley, New York. Hastie, T., Tibshirani, R. and Friedman, J The Elements of Statistical Learning; Data Mining, Inference and Prediction, 2nd ed., Springer, New York. Hilbe, J. M Negative Binomial Regression, Cambridge University Press, Cambridge. Lam,K.F.,Xue,H.andCheung,Y.B Semiparametric analysis of zero-inflated count data, Biometrics, 62, Lambert, D Zero-inflated Poisson regression, with an application to defects in manufacturing, Technometrics, 34, Lawless, J Negative binomial and mixed Poisson regression, The Canadian Journal of Statistics, 15, Maunder, M. and Punt, A. E Standardizing catch and effort data: A review of recent approaches, Fisheries Research, 70, Minami, M., Lennert-Cody, C. E., Gao, W. and Roman-Verdesoto, M. H Modeling shark bycatch : The zero-inflated negative binomial regression model with smoothing, Fisheries Research, 84, Mullahy, J Specification and testing of some modified count data models, Journal of Econometrics, 33, Román-Verdesoto, M. and Orozco-Zöller, M Bycatches of sharks in the tuna purse-seine fishery of the eastern Pacific Ocean reported by observers of the Inter-American Tropical Tuna Commission, , Data Report 11, Inter-American Tropical Tuna Commission, La Jolla, California. Vuong, Q Likelihood ratio tests for model selection and non-nested hypotheses, Econometrica, 57, Welsh, A., Cunningham, R., Donnelly, C. and Lindenmayer, D Modelling the abundance of rare species: statistical models for counts with extra zeros, Ecological Modelling, 88, Williams, D. A Generalized linear model diagnostic using the deviance and single case deletions, Applied Statistics, 36 2,

17 Proceedings of the Institute of Statistical Mathematics Vol. 61, No. 2, (2013) 287 Analysis of Data with Many Zero-valued Observations: Over-estimation of Temporal Trend by Negative Binomial Regression Mihoko Minami 1 and Cleridy E. Lennert-Cody 2 1 Department of Mathematics, Keio University 2 Inter-American Tropical Tuna Commission In ecological and environmental studies, count data such as the number of animals per unit area or unit effort often contain many zero-valued observations. Such data unfortunately may be analyzed without any special consideration given to how the zeros arose. In particular, the negative binomial regression model has been a commonly used model for count data with overdispersion. However, we found that the negative binomial regression model over-estimated temporal trends in species relative abundance. Such over-estimation could be problematic, for example, for the development of management guidelines for conservation. In this paper, we investigate this phenomena of over-estimation. We show that when the negative binomial regression model is fitted to data with excess zeros, the estimate of the size parameter becomes too small and the observations with small fitted values have more influence. This results in estimated coefficients of predictors that are too large in absolute value, and it produces exaggerated estimates of the marginal effects. Key words: Over-dispersion, influence function, size parameter, zero-inflated negative binomial regression model, Cook s distance, leverage.

dvi

dvi 2017 65 2 185 200 2017 1 2 2016 12 28 2017 5 17 5 24 PITCHf/x PITCHf/x PITCHf/x MLB 2014 PITCHf/x 1. 1 223 8522 3 14 1 2 223 8522 3 14 1 186 65 2 2017 PITCHf/x 1.1 PITCHf/x PITCHf/x SPORTVISION MLB 30