Statistical Inference of Stratified Linear Regression Model Based on Nested Structure

1. Introduction

In actual observation data, data often has a hierarchical structure, and data having nested and hierarchical structures can be analyzed using a multi-layer linear model, and the multilayer linear model is in processing lack of data. It does not affect the characteristics of parameter estimation accuracy, compared to the multi-linear regression model, the longitudinal data has a large advantage, and there is no strict requirement for the number of repeated measurements and time spans. For the description of the layered linear regression model, the names in different documents are different. In sociological studies, it is often referred to as multi-layer linear models (Goldstein [1], 1997), in biometric application, called For the random effect model (laird [2], 1982), in statistical literature, it is generally referred to as a covariance component model (Dempster [3], 1981). The unknown parameters of the layered linear regression model are generally estimated to estimate the regression coefficients and variance coefficient differences corresponding to the random effect and the fixed effect. In this paper, the regression coefficient diagnosis problem of the hierarchical linear regression model is mainly considered. The first layer of regression coefficient of the layer linear regression model performs regression diagnosis, that is, the corresponding diagnosis problem of multivariate linear regression coefficients, has a small number of references to the second layer coefficient of the layered linear regression model, and this article is hierarchical The regression diagnosis of the second layer coefficient is primarily inferred by constructing a method of laminated linear regression model likelihood functions. In the multi-linear regression model, Wu Xi Zhi [4] (2016) uses a linear model to approximate the variable and the linear relationship of the variable, and give the parameter estimation in the linear model and the multi-variable coefficient composite inspection process, thank you Yu [5] (2013) introduces the specific meaning of the nested model in the multi-linear model, and gives the combined test of the combined test of multiple regression coefficients of nested models, and gives two-point due variable nested models. Model evaluation method, but does not give a test process of a layered linear nested model, Lindley & Smith [6] (1972) gives a specific form of a layered linear model for complex nested data, and is targeted The unknown parameters in the regression model give a specific estimate, and Tian Mao is again [7] (2006) proposes the algorithm of the Conditional Points of GuAss-SeiDel iteration, solving the layered model cannot be fully portrayed. VisionThe condition of responding variables, Wu Mi Xia [8] (2013) introduces the likelihood test of the model parameters, gives the specific hypothesis process, Liu Hongyun [9] (2005) introduces in tracking data analysis For multi-layer linear models, the WALD test can be utilized to make a significant test of the fixed partial parameters.

For the statistical inferior promotion of the regression model, Ma Haiqiang [10] (2008) gave the statistical diagnosis problem of variable coefficient model, Dai Lin sent [11] (2013) to study the generalized Poisson regression model Statistical diagnostic method, Li & Lee [12] (2019) Maximize the blend of two-parameter zero expansion and negative two distribution regression models through the likelihood function, and evaluate a consecutive coarction effect of a hypothetical functional form Proficiency, using data to prove the effectiveness of its methods. Feiyu [13] (2013) introduces the statistical diagnosis method of linear hybrid model and generalized linear hybrid model. Zeng et al [14] (2017) proposes three integrated residuals, leverage values, and coefficient variations to diagnose abnormal points or strong impact points of data in the logistic regression model. Chown & Ursula [15] (2019) Introduced a multi-poorer test method for non-parameter regression models, using local polynomial smooth structure residual, design and detection function verification parallelism.振 等 [16] (2016) Diagnosis of abnormal dots in the big data set after the leverage value sample. Liang Jin Wen [17] (2020) Based on the diagnosis of abnormal points based on data deletion model and mean drift model, the number of abnormal points was studied by volume sampling.

The least squares method based on the linear regression model is estimated, and the linear regression nested model is mainly discussed whether the regression coefficient of increased variables is remarkable, and whether there is a substantial change in the original regression coefficient. This determines the variable reservation or not. Based on the variable discussion in the linear regression model, the layered linear regression model is further discussed. This is also the main innovation point of this paper, using the ratio of the likelihood function of the nesting structure, the ratio of the likelihood function is judged to the layered linear embedded Set of rationality of the model assumption. At the same time, this article is based on the same variance. There are also other articles on the differential differences in linear regression models.Detailed description.

In this article, according to the meaning of the multivariate linear nested model, the multi-line nested model is mainly used to verify the significance of the restrictive model and the non-limiting model, and through the Boston Rate The validity of the test statistic, and according to the nesting structure of the multi-linear model, a reasonable hypothesis of the nested structure of the hierarchical linear model is mainly hypothetical, mainly by constructing a laminated linear regression model of a nesting structure. The ratio is determined from the card distribution, and the significance problem of the restriction layered regression model and the non-limiting layered regression model is determined by a given reject domain.

2. Statistical Inference of Linear Nested Model

If we only consider the regression diagnosis of the first layer coefficient of the layered linear model, the layered linear model can directly understand the multivariate linearity. Regression model of the regression model. In order to better perform the statistical diagnosis of the first layer model, the first layer of model is now in the form of transformation, and its practical significance has no effect.

We consider that there is a general form of a random linear model. The linear model means that the relationship between the variable Y and the variable X can be used to approximate (Wu Xi [4] 2016):

= X I + ε i , = 1 , , N y i


× N × P The known vector matrix, is an unknown parameter to be estimated, ε is a random error term that the model cannot be described. Normally, random error i satisfies 3 assumptions: 1) ( ε i ) V a R ( ε i = σ 2 ; 3) ( ε I ) = i j . Often, people put unknown Assumble as equal In the case of the traditional minimum multiplier estimation model parameters, the obtained parameter estimate is not an effective estimate, and the regression diagnosis of the difference in the case of the difference is not specifically described. The following linear regression diagnosis is based on the same time. For linear regression models to be estimated parameters The method of commonly used estimation parameters is a normal least squares method, and its purpose is to make ε x β to minimize, That is, ( β ) = ( Y – β 2

The minimum is achieved. That is, the unknown parameter

β s ( β is zero, it can be obtained: β ^ = ( X ” – 1 If the argument in a model is another model The linear combination of the self-variable subset or subset is called two models of nested models (Xie Yu [5] 2013). The model of a linear combination of a model subset or a subset is referred to as a restrictive model, and another model corresponding to a non-limiting model is referred to as a non-limiting model nested in a non-limiting model. The coefficient of multivariate linear regression model is proposed as follows: H 0 : y X 1 β


+ 2 β 2 Vs : Y = X 1 + X 2



3 Corresponding test statistic : F J = S E H – S S E ) / K – g S S H 1 / ( – K – 1 wherein s S



is the original hypothesis H The corresponding residual fraction, S S H is prepared hypothesis [ 10] H The corresponding residual square and the residual fraction of the residual model and the non-limiting model (Xie Yu[5] 2013). Here, the number of regression coefficients contained in the alternative hypothesis model N – K – Corresponding to the alternative hypothesis of residuals and the degree of freedom. This freedom increment is between optional hypothesis and the original hypothesis corresponding model The difference between the number of regression coefficients. α , the rejection domain of the test is F J F α ( 1 , N – ) . Since the original hypothesis removes the partial argument, theoretical assumptions corresponding to the residual fraction and the residual sum of the residuals are not less than the hypothesis. Since the original hypothesis is only one parameter between the corresponding model, T inspection statistics can also be used T = F 1 ~ T (

) For a given significance level α , the rejection domain of the test is | | > T ( N ) . Where the F. The first degree of freedom of the F status is 1, then the F statistic can also be used to use T statistics. For the linear regression model of the nested structure, the determination coefficient increment can be used to explain the problem of returning model fit, and the determination coefficient of the non-limiting model is subtracted to reduce the restriction model. The decision coefficient (Xie Yu [5] 2013), the detailed process will be reflected in the simulation of Boston Rate factors, and consider the addition of the original hypothesis and alternative hypothesis of the multivariate limit model and the non-limiting model. The average number of residential rooms This argument is determined by variable analysis to determine this variable. Whether the problem is retained. The following simulation data is derived from part of the Boston house price, with the number of the number of housing prices as the variable Y, with the average room number of each house x , the five employment center of Boston is X

3 We constructed linear nesting model test, original assumptions are restricted models, with x 2 3 is an independent variable, and the alternative assumption is not The limit model is added to the average room number of each residential, with x 1 ,

X 3 is self-variable, specifically expression: h 0 : Y β 2 X 2 3 + H 1 : y β 1 X 1 2 X 2 X 3 + Fighe 1 . Scatter Plot of Restricted Model Fig. 1 . Scatter points of the restriction model Figure 2

. Scatter Plot of Unrestricted Model Fig. 2 . Scatter plots of non-limiting model From

Fig. 1 and Figure 2 It can be seen that in the factors affecting the price of Boston, the average room number X 1 2 and students in town There is a significant linear relationship with the median ym of the home price. In response to this multivariate linear regression model, we construct a restriction model with non-limiting models, and constructive statistics, which can be seen from the following variance analysis table. Table 1 . Analysis of Variance of Restrictive Model and Unrestricted MoDEL Table 1 . Restriction model and non-limiting model variance analysis table According to Table 1 variance analysis table Get, the test statistic of the nested model is calculated to obtain a F value of 28.255, so that the P value corresponding to the nested model F is significantly less than the significant level 0.05, so rejecting the original hypothesis, accepting optional assumptions, ie non-limiting The model passes a significant test. At the same time, the T value obtained by the F value is 5.316 is greater than the rejection domain 2.160. At the same time, it can also be based on the increment of the determination coefficient according to the nested model, it can be seen that when the restriction model is added to the self-variable x 1 , the determination coefficient R 2 increases, meaning more square and is explained by the non-limiting model. In the multivariate linear regression model statistics, it is generally included in two aspects: one is the overall inspection of the regression model and the other is the test of the regression coefficient. The variance analysis of multi-linear regression is roughly the same, for multivariate linear nested models, we often use the construction of F statumerate to detect the hypothesis of the restriction model, if the original hypothesis of the nested model, the returning independent variable Only one regression coefficient can also be tested using T statistics, but the F statistic inspection cannot be used for two unscading models. At the same time, for the nested model, the restrictive model independent variable can not only be a non-limiting model, but also the non-limiting model independent variable can also be a linear combination of arguments in the restrictive model. 3. Stratified linear model statistics with nested structures Data often have a layered structure during statistical data, such as studying different students in universities, or The difference in the study of national economic development is interacting with adult education, or the difference in treatment methods of clinical drugs, etc., these cases have a study of nested problems, and the layered linear regression model gives a good model structure.The layered linear model is proposed by Lindley and Smith [6] (1972) as an important contribution to the estimation of the linear model, and the form of a universal layered linear model is given to complex nested structural data. . Here, the two-layer data model is used as an example, the specific form of two layered linear models is given, and there is a hypothesis ( X

of a set of independent distribution observation

X 1 , 1 ) ( 2 , y , ⋯ ( x N , Y } , wherein Y is the value of the real number of variables, X 1 × D β i is unknown × DirectionQuantity, meet the first layer model (Tian Mao repeated [7] 2006): Y i



I ~


0 ,

σ 2 wherein R i is the IID unbaraciible random effect variable, assuming to interpret variables independent, combined with average value is 0, the variance is σ 2 Normal distribution. In the second layer model, the first layer model is in the coefficient vector as the interpretable variable, is a fixed effect vector, is a known interpretation variable matrix in the second layer:


W i



, U ~ N (


wherein U is the second layer D × 1 Dimensional random effect vector, assuming and second layer interpretation variables and R Independent, the multi-pylon distribution of the same value is 0, the covariance is T. A second layer model is brought into the first layer model to obtain the following form (Raudenbush and Bryk [18] 1992): Y = i W + X i u + R i , = , ⋯ , N The above model is also referred to as a linear hybrid model, which can be used to analyze various types of repeated measurement data such as longitudinal data and panel data, compared to linear models, and the observing. The variance matrix can have a more flexible setting, and more convenient and reasonable assumptions are given for the random effect section. The statistical method for nested model analysis of the prolonged factor (Xie Yu [5] 2013), which is an impact on estimation and predicting the probability of success or failure. Differentiation variables are interpreted as having values, and is also commonly referred to as 0-1 variables, which are often used to handle the two-point factor variables to Logit model. In response to the two-binary variables in the nesting relationship, it is often more preferred to determine the method of fitting the model of the model, that is, the difference between the two nested models. Statistic, its statistic is from distributed, corresponding to statistic forms: δ g 2 = g – g U wherein g R g U The alignment ratio of the unconstrained model is obedient with the two-point due variable nested model, and χ

The different degree of freedom of freedom is the residual degree of freedom and the residual degree of freedom of residuality and constraint model. Nested model for the hypothetical test and the two-point due variable for the above multi-linear regression modelTest, expanded to a layered linear regression model, and the unknown parameter estimate in the layered linear model is mainly estimated to estimate the regression coefficient of the fixed effect and the variance of the random effect, specifically according to the use of Raudenbush [18] (1992). The condition of fully data sufficient statistics is expected to replace the iterative process of the desired step, which does not specifically discuss the parameter estimation process, and the following content is primarily a likelihood function of the layered linear model. Bringing the second layer model into the first layer model, y i has a linear mixed model form, known, first layer random error R i ~ N 0 , σ 2 and the second layer random error U ~ N ( 0 , the linear hybrid model

y 】 y ~ N ( W ,


) , corresponding, V = x T

τ + σ 2 i . The likelihood function of the layered linear model is: L ( y = π J 2 π )

– N 2 | V | Exp { –



Y – X W τ V ( y – X ) } = π J = 1 J

( 2 π

| X X J τ + N

| Exp { – ( y –

τ X J τ X J + σ 2 | – ( Y – X W } = ( 2 π – N | X X J τ + σ 2 i N N 2 { – 1 2


1 N ( y – X W τ | X τ J τ + i N | – ( y – wherein T g ( τ ⋯ , τ ) , = 1 , ⋯ , J . According to the above, a general hypothesis of a layered linear model having a nested structure is given: Under zero assault: First layer: Y 1 β 1 + ε 1 ~ ( 0 , σ ) Layer 2: β ~ N W R , T 1 Preparation hypothesis: First layer: y 1 β 1 2 β 2 + ε 2 ~ ( 0 , ) Second layer: β 1 β 2 ~ N ( ( W R ) , Assume θ | Y ) is a likelihood function of parameter θ , wherein y = ( y 1 ⋯ , y [12 ” is a sample capacity of n The sample, parameter space Ω , the test problem is h : : ∈ Ω 0 H 1 : θ ∉ Ω 0 , the statistic definition is likelihood ratio (Wu Mi Xia [8] 2013) is: L Y ) = SUP θ ∈ Ω L N ( θ ∈ Ω L | y In the multi-analytical process, the likelihood ratio test is a common test method, and the ratio is constructed using the classic likelihood function: ( 1 , ⋯ N = ( H 0 ) / L ( h 1 . Here we construct a likelihood ratio of the layered nested inspection, the original assumption is a restriction layered linear regression model, and the alternative assumption is a non-limiting hierarchical linear regression model, thus constructing a similarity ratio test statistical comparison of the hierarchical linear regression model. the amount. L ( h = π I =

([ ) – N | | | 1 Exp – 1


y i – X W 1 R τ V 1 ( Y 1 i [1 1 i W

) } wherein V = 1 T 1 1 τ +


N . ( H ) ) π i = 1 ( 2 – N 2 | V 2 1 2 Exp


1 2 ( Y 2 i – 1 i W R – X 2 i R ) V 2 – 1 Y I – X W 1 R – 2 i W

2 R V 2 = 1 1 X 1 + X 2 X 2 τ + i N . The constructor is: L ( h 0 ) H 1 ) π i = N

π ) – | 1 2 Exp – 1 2 1 i – – X 1 i R ) τ V 1 – ( Y 1 I – –

1 i R ) π = N ( – N 2 | V 2 1


Exp – – 1 2 ( Y i – i W 1 R – X W 2 R ) τ V 2 – ( Y – X 1 W 1 R – 2 i W R } = V 1 2 { – – 1 2 σ i = ( Y x W 1 R τ V 1

( Y 1 – 1 i R ) | V 2 |

Exp { σ i = 1 N Y 2 i 1 i R – X 2 W 2 R τ V 2 ( y 2 i 1 i R – X 2 W 2 R } Among them, the molecule indicates the maximum value of the original hypothesis, and the denominator indicates the maximum value of the alternative hypothesis. If the value of the statistic is very large, the possibility of the original hypothesis is compared with the possibility of the preparation assumption. It is small, so we have reason to think that the original hypothesis is not established. In the multi-layer linear model, the statistical diagnosis of the model single self-variable parameter estimation value can be obtained by very large likelihood estimation to obtain a standard error of the fixed partial parameter estimate, and the significance of the fixed portion Test, you can estimate parameterThe value is divided by the standard, that is, the corresponding γ / S E is carried out (Liu Hongyun [9] 2005). For multi-layer nested linear regression model, constructor is: λ ( , ⋯ , X N = ( h L (H 1) The statistical suits χ 2 Distribution, its freedom is equal to the number of alternative hypothetical parameters minus the number of parameters in the original hypothesis, for the given hierarchical linear regression zero counterfeit and alternative hypothesis, here χ The degree of freedom in which the distribution corresponds to 1. is remarkableSexual level , its reject domain is: W = { χ 1

C , if it falls into the reject domain, The statistical diagnosis is not significant, then the original hypothesis is rejected, accepts the presence assumptions of non-restricted hierarchical linear models. 4. Data Analysis The following data comes from 7185 student mathematics scores in 160 schools, and analyzes the data in a layered linear regression model. Here we select some of these data to nested Layered linear regression model analysis, for the first layer, that is, the students level, this is selected as the variable, ie y , FEMALE (student gender) (1 means female, 0 means male), SES (student social status ) The students’ parents are educated, occupational and income synthesis as an argument. For the second level, ie the school level, this is selected here, including the average social status of each school, the average social status of each school student, Size (the number of students enrollment) is used as the second layer of arguments . Here, constructing a restriction layered linear model is: Mathach as the third student variable in paragraph J, Y i J , SES As the first layer level of the secondary variable X , Meanses and Disclim as the index quantity of the school in the second layer level, ie W 0 1 , non-limiting hierarchical linear model construction As the number I of JISA, the number of variables i J , SES and FEMALE As the first layer of schools, the first level of the second variable X


X 2 , MeanSes and Disclim and Size as the index quantum of the school in the second layer level, ie W W 1 , the specific model construct is as follows: original It is assumed: i J J + J * S E S I + ε i j β 0 = γ 00 01 * M E s E S J J β 1 = γ 10 γ 11 * D I C i M J Bringing a second layer model into a mixing effect obtained by the first layer modelThe model is: Y i J + γ * M N N + γ 10 * S i + γ 11 * C L i M * E S i + U 0 j + i J Table 2 . Estimation of Fixed Effects in null hypothesis Hierarchical Linear Model . The fixed effect estimation in the layered linear model Table 2 After 6 iterations The variable value of the albarative function is minimized, gives the coefficient estimate of the fixed effect variable and the corresponding standard error in the case of the original, where the fixed effect is obtained by the least squares estimation, and the variable can be seen by p value. Both coefficient estimates passed the test, and the variance corresponding to the first layer of the linear regression model was 36.887. Alternative assumptions are: First layer: Y i J = β J + β 1 * E M A i j + * S E i J + ε Layer 2: J = + γ 01 N S E + U 0 J J = + γ 11 * S i z E J J = γ + γ 21 * C L i M Baby the second layer model into the first layer to obtain a mixed effect model of the alternative hypothesis: Y J = + γ 01 N S + γ 10 F A L J + γ 11 * Z E J * F A L + γ 20 * E i j + 21 * i C L i M J * i J U J + Table 3 . Estimation of Fixed Effects in Alternative Hypothesis HIRArchical linear model . Preparation assumptions of fixed effects in the layered linear model from Table 3 after 6 times The iterative likelihood function change value minimizes, it can be seen that the coefficient estimate corresponding to the fixed effect in the hierarchical linear regression model in the case of alternative hypothesis, and most of the significance test is detected at a significant level of 0.05, coefficient did not pass a significant test at a significant level of 0.01, while The variance corresponding to the first layer is 36.628. The likelihood ratio of the nested model is 1.001 by calculation, corresponding to the suggestive level = χ 1 2 The value is 0.0002, according to a given reject domain, the original hypothesis cannot be rejected, so that the original hypothesis is accepted, that is, the restriction hierarchical linear model, indicating that the school recovery number variable from the school, from the students level Introducing gender variables, there is no significant effect on college students’ mathematics. 5. Small junction This article passes multiple linear nesting modesThe hypothetical hypothesis process proposes a hypothetical test of a layered linear regression model with a nested structure. By the lack of linear models, it is determined to determine whether the layered linear model is retained for introducing new variables. The corresponding theoretical basis is given, and the feasibility and construct of the assumption of the tester have the practicality of the nested structure likelihood ratio statistic. Notes * Corresponding author. Reference [1] COrgeau, D. And Goldstein, H. (1997) Multilevel StatisTical Models. Population, 52, 1043-1046. [2] Laird, NM (1982) Random Effects Model For Longitudinal Data. Biometrics, 38, 963-974. https: inexposure Dempster, AP and Tsutakawa, DBRK (1981) Estimation In Covariance Components Models. Journal of the American Statistical Association, 76, 341-353. [4] Wu Xizhi. Application regression and classification: Based on R [M]. Beijing: Renmin University of China Press, 2016. [5 Xie Yu. Regression analysis [M]. 2nd edition. Beijing: Social Science Document Press, 2013. Smith, DVLFM (1972) Bayes Estimates for the linear model. Journal of The Royal Statistical Society, 34, 1-41. [7] Tian Mao again, Chen Ge Mai.Clastic linear regression model in conditional packet [J]. 中国 科学 A series, 2006, 36 (10): 1103-1118. Wu Mi Xia. Linear hybrid effect model introduction [M]. Beijing: Science Press, 2013.

Liu Hongyun. Tracking Data Analysis Method and Its Application [M]. Beijing: Education Science Press, 2005. [10] Mahahaqiang. Statistical diagnosis and impact analysis of variable coefficient model [D]: [Master’s thesis]. Changsha: Central South University, 2008. Derin delivery, Lin Jin official. Statistical diagnosis of generalized Poisson regression model [J]. Statistics and Decision, 2013 ( 21): 29-33. [12] LI, CS, Lee, SM AND YEH, MS (2019) A Test for lack-of-fit of zero-inflated negative binomial models. Journal of Statal Computation and Simulation, 89, 1301-1321. https: /

Feiyu Mixed model and statistical diagnosis [M]. Beijing: Science Press, 2013.

Zeng Wei, Hu Guizhi. Statistical Diagnosis of Logistic Regression Model [J]. Mathematical Statistics and Management, 2017, 36 (4 : 620-631. [15] Chown, J. and Müller, UU (2019) Corrigendum:. Detecting Heteroscedasticity in Non-Parametric Regression Using Weighted Empirical Processes Journal of the Royal Statistical Society Series B (Statistical Methodology), 81, 805-806. Zhen, Dai Xiaowen, Tian Mao again. Anomalies Diagnosis of Large Data Set Samples Based on Leverage [J]. Mathematical StatisticsManagement, 2016, 35 (5): 794-802. Liang Jin Wen, Tian Mao. Based on the big data Abnormal Diagnosis and Estimation of Volume Sampling [J]. Mathematical Statistics and Management, 2020, 39 (2): 223-235. [18] Bryk, AS and Raudenbush, SW (1992) Hierarchical Linear Models: Applications and Data Analysis Methods. Journal of the American Statistical Association, 98, 436-450.

Leave a Reply

Your email address will not be published. Required fields are marked *