Modern Stochastics: Theory and Applications

This paper continues studies of the jackknife (JK) technique application for statistical inference based on the model of mixture with varying concentrations (MVC). JK is a powerful tool of asymptotic covariance estimation of asymptotically normal statistics introduced by Quenouille (1949) and Tukey (1958). On its applications for homogeneous samples, see [11] and [1]. The JK technique was applied to heteroscedasitc nonlinear regression models in [9]. Applications to errors-in-variables models are considered in [12, 13].

In MVC models one deals with a nonhomogeneous sample which consists of subjects belonging to M different subpopulations (mixture components). One knows the probabilities with which a subject belongs to the mixture components and these probabilities are different for different subjects. So the considered observations are independent but not identically distributed. Modification of JK to such data analysis is a challenging problem.

On parametric inference in regression MVC models, see [2]. Estimation in nonparametric MVC models is discussed in [3]. In [5] a jackknife application to MVC of linear regressions models with errors in variables is considered. It is shown that the JK-estimators are consistent and allow to construct asymptotic confidence intervals for regression coefficients based on orthogonal regression estimators. In [7] a general result on asymptotic normality of generalized estimating equation (GEE) estimators for MVC is obtained which is applied to derive asymptotic normality of a modification of least squares (LS) estimators for MVC of nonlinear regressions models. A JK estimator for the asymptotic covariance was introduced in [7] also, but its properties were not investigated analytically.

In this paper we consider JK estimation of asymptotic covariance of GEE estimators in MVC models and show its consistency. The MVC model and the GEE estimator are discussed in Section 2. A version of JK for MVC is described in Section 3. Main results on consistency, asymptotic normality of GEE estimator and consistency of JK estimator of asymptotic covariance are presented in Section 4. Here we also consider an application to some nonlinear regression model. In Section 5 the developed statistical techniques are applied to a real life sociological data. Conclusive remarks are placed in Section 6. Section 7 contains technical proofs.

In the MVC model we assume that each observed subject O belongs to one of M different mixture components (subpopulations) ${\mathcal{P}_{k}}$, $k=1,\dots ,M$. The sample contains n subjects ${O_{1}}$,…, ${O_{n}}$. Let ${\kappa _{j}}=k$ iff ${O_{j}}\in {\mathcal{P}_{k}}$. The true ${\kappa _{j}}$ are unknown, but one knows the mixing probabilities These probabilities are also called the concentrations of the k-th component at j-th observation.

The D-dimensional vector of observed variables of O will be denoted by $\boldsymbol{\xi }(O)={({\xi ^{1}}(O),\dots ,{\xi ^{D}}(O))^{T}}\in {\mathbb{R}^{D}}$, ${\boldsymbol{\xi }_{j}}={\boldsymbol{\xi }_{j;n}}=\boldsymbol{\xi }({O_{j}})$.

Let ${F^{(k)}}$ be the distribution of $\boldsymbol{\xi }(O)$ for $O\in {\mathcal{P}_{k}}$, i.e. for all Borel sets $A\subseteq {\mathbb{R}^{D}}$. Then So, in the MVC model one observes independent ${\boldsymbol{\xi }_{j}}$, $j=1,\dots ,n$, with the distribution defined by (1). In this paper we adopt a semiparametric model of components’ distributions where F is some known function of its arguments, ${\boldsymbol{\vartheta }^{(k)}}\in \Theta \subseteq {\mathbb{R}^{d}}$ are unknown Euclidean parameters of interest, ${\boldsymbol{\nu }^{(k)}}$ are some nonparametric nuisance parameters.

In what follows we will denote by ${\boldsymbol{\xi }_{(k)}}$ a random vector with distribution ${F^{(k)}}$ which can be considered as the value of $\boldsymbol{\xi }(O)$ for a subject O selected at random from the component ${\mathcal{P}_{k}}$.

We assume that ${\varepsilon _{j}}$ are independent for different j and, for each j, ${\varepsilon _{j}}$ and ${\mathbf{X}_{j}}$ are conditionally independent given ${\kappa _{j}}$. Let ${F_{X}^{(k)}}$ and ${F_{\varepsilon }^{(k)}}$ be the conditional distributions of ${\mathbf{X}_{j}}$ and ${\varepsilon _{j}}$ given ${O_{j}}\in {\mathcal{P}_{k}}$. We assume that $\operatorname{\mathsf{E}}[{\varepsilon _{j}}\hspace{2.5pt}|\hspace{2.5pt}{\kappa _{j}}=k]=\textstyle\int x{F_{X}^{(k)}}(dx)=0$ for all $k=1,\dots ,M$.

Model (3) is a partial case of model (1)–(2) in which the nuisance parameters are the distributions of regressors and errors for all components, i.e., ${\boldsymbol{\nu }^{(k)}}=({F_{X}^{(k)}},{F_{\varepsilon }^{(k)}})$.

To estimate ${\boldsymbol{\vartheta }^{(k)}}$ in (1)–(2) we apply the technique of generalized estimating equations (GEE) considered in [7]. (On GEE estimation technique and its relations to least squares, maximum likelihood and M-estimators in context of i.i.d. observations, see Section 5.4 in [10].) Let us choose an elementary estimating function $\mathbf{s}:{\mathbb{R}^{D}}\times \Theta \to {\mathbb{R}^{d}}$ such that So, considering (4) as an equation of $\boldsymbol{\gamma }={({\gamma ^{1}},\dots {\gamma ^{d}})^{T}}\in \Theta \subseteq {\mathbb{R}^{d}}$, we observe that its unique solution is ${\boldsymbol{\vartheta }^{(k)}}$. To obtain an estimator for ${\boldsymbol{\vartheta }^{(k)}}$ we replace $\operatorname{\mathsf{E}}\mathbf{s}({\boldsymbol{\xi }_{(k)}},\boldsymbol{\gamma })$ by its estimator where ${a_{j;n}^{k}}$ are nonrandom weights satisfying the assumption Observe that ${\mathbf{S}_{n}^{(k)}}(\boldsymbol{\gamma })$ is an unbiased estimator for $\operatorname{\mathsf{E}}\mathbf{s}({\boldsymbol{\xi }_{(k)}},\boldsymbol{\gamma })$ under (6), i.e., $\operatorname{\mathsf{E}}{\mathbf{S}_{n}^{(k)}}(\boldsymbol{\gamma })=\operatorname{\mathsf{E}}\mathbf{s}({\boldsymbol{\xi }_{(k)}},\boldsymbol{\gamma })$. The GEE estimator to ${\boldsymbol{\vartheta }^{(k)}}$ is any statistics ${\hat{\boldsymbol{\vartheta }}_{n}^{(k)}}$ such that E.g., in the model (3) differentiation of the least squares functional yields the elementary estimating function where In this paper we consider only the minimax weights ${a_{j;n}^{k}}$, which can be defined as follows. Let ${\mathbf{p}_{;n}}$ be the matrix of all concentrations for all components of the mixture: Then the matrix of all weights ${\mathbf{a}_{;n}}={({a_{j;n}^{m}})_{j=1,\dots ,n,m=1,\dots ,M}}$ is defined as where ${\boldsymbol{\Gamma }_{;n}}={\mathbf{p}_{;n}^{T}}{\mathbf{p}_{;n}}$. (We assume that $\det {\boldsymbol{\Gamma }_{;n}}\ne 0$). Minimax properties of ${\mathbf{a}_{;n}}$ were discussed in [3]. For one alternative approach to weighting in GEE for MVC, see [4].

Consider the set of parameters ${\boldsymbol{\vartheta }^{(k)}}$, $k=1,\dots ,M$, for different components as one long vector parameter $\boldsymbol{\vartheta }={({({\boldsymbol{\vartheta }^{(1)}})^{T}},\dots ,{({\boldsymbol{\vartheta }^{(M)}})^{T}})^{T}}$ and similarly for the set of estimators ${\hat{\boldsymbol{\vartheta }}_{n}}={({({\hat{\boldsymbol{\vartheta }}_{n}^{(1)}})^{T}},\dots ,{({\hat{\boldsymbol{\vartheta }}_{n}^{(M)}})^{T}})^{T}}$. (Recall that ${\boldsymbol{\vartheta }^{(k)}},{\hat{\boldsymbol{\vartheta }}_{n}^{(k)}}\in {\mathbb{R}^{d}}$.) It was shown in [7] that under suitable assumptions (see Theorem 3 below) the estimator ${\hat{\boldsymbol{\vartheta }}_{;n}}$ is asymptotically normal, i.e. To apply asymptotic normality for hypotheses testing one needs an estimator for the dispersion matrix (asymptotic covariance) V given by (16). Jackknife (JK) is a powerful tool for constructing such estimators. We now consider its modification for the MVC models (cf. [5]). Let i.e. ${\mathbf{p}_{;-i,n}}$ is the matrix ${\mathbf{p}_{;n}}$ with the i-th row replaced by the zero row. Then ${\boldsymbol{\Gamma }_{;-i,n}}={\mathbf{p}_{;-i,n}^{T}}{\mathbf{p}_{;-i,n}}$ and Let and define ${\hat{\boldsymbol{\vartheta }}_{-in}^{(k)}}$ as a statistics which satisfy ${\hat{\boldsymbol{\vartheta }}_{-in}}={({({\hat{\boldsymbol{\vartheta }}_{-in}^{(1)}})^{T}},\dots ,{({\hat{\boldsymbol{\vartheta }}_{-in}^{(M)}})^{T}})^{T}}$. In fact, ${\hat{\boldsymbol{\vartheta }}_{-in}}$ is the GEE estimator for $\boldsymbol{\vartheta }$ calculated by the sample which contains all the observed subjects ${O_{j}}$, except the i-th one. Then the JK estimator for V is (On some efficient algorithms for calculation of ${\mathbf{a}_{;-i,n}}$ and ${\hat{\mathbf{V}}_{n}}$ see [5]).

In this section we consider asymptotic behavior of ${\hat{\boldsymbol{\vartheta }}_{;n}}$ and ${\hat{\mathbf{V}}_{n}}$ as $n\to \infty $. Note that we do not assume any relationship between the samples $\{{\boldsymbol{\xi }_{j;n}},\hspace{2.5pt}j=1,\dots ,n\}$ for different n. They can be independent or dependent, or a smaller sample can be a part of a larger one. The concentration arrays ${\mathbf{p}_{;n}}$ are also unrelated for different n.

To formulate the theorems we need some notations and assumptions. In what follows we assume that the limit exists and $\det {\boldsymbol{\Gamma }_{\infty }}\ne 0$.

For a vector x, the symbol $|\mathbf{x}|$ means the Euclidean norm. For a matrix A, $|\mathbf{A}|$ is the operator norm. Let $\boldsymbol{\psi }(\mathbf{x},\boldsymbol{\gamma })$ be any function of $\mathbf{x}\in {\mathbb{R}^{D}}$, $\boldsymbol{\gamma }\in \Theta $, maybe vector- or matrix-valued, $h:{\mathbb{R}^{D}}\to \mathbb{R}$, $\rho :\Theta \times \Theta \to \mathbb{R}$. We say that $\boldsymbol{\psi }$ satisfy Condition ${\boldsymbol{\Psi }_{h,\rho }}$ iff, for all ${\boldsymbol{\gamma }_{1}},{\boldsymbol{\gamma }_{2}}\in \Theta $ and all $\mathbf{x}\in {\mathbb{R}^{D}}$, A set of functions $\{{\boldsymbol{\psi }_{i}},i\in I\}$ satisfy Condition ${\boldsymbol{\Psi }_{h,\rho }}$ if ${\boldsymbol{\psi }_{i}}$ satisfy ${\boldsymbol{\Psi }_{h,\rho }}$ for each $i\in I$.

The following theorem states conditions of consistency of ${\hat{\boldsymbol{\vartheta }}_{n}}$ and consistency of ${\hat{\boldsymbol{\vartheta }}_{-in}}$ uniform by i.

Assumptions (C7) and (C7’) claim the existence of GEE solutions with probability tending to 1 as $n\to \infty $. They seem rather imperfect. The following theorem provides conditions under which they hold.

Let $\dot{\mathbf{S}}(\boldsymbol{\gamma })$ be the Jacobian of a vector-valued function S, ${\mathbf{S}_{\infty }}(\boldsymbol{\gamma })=\operatorname{\mathsf{E}}\mathbf{s}({\boldsymbol{\xi }_{(k)}},\boldsymbol{\gamma })$.

To formulate the asymptotic normality result we need some additional notations. Let (existence of these limits is a condition in the following Theorem 3). Now (here and below ${\mathbf{M}^{-T}}={({\mathbf{M}^{-1}})^{T}}$). Let’s pack all the matrices ${\mathbf{V}^{(m,l)}}$ into one $(Md)\times (Md)$ matrix Let $\mathbf{s}(\mathbf{x},\boldsymbol{\gamma })={({s^{1}}(\mathbf{x},\boldsymbol{\gamma }),\dots ,{s^{d}}(\mathbf{x},\boldsymbol{\gamma }))^{T}}$.

(Note that in Theorems 1–4 and the lemmas in Section 7 below the functions h can be different).

In fact, Theorem 3 is just Theorem 2 from [7] reformulated in terms of the present paper.

Now we are ready to formulate the theorem on consistency of the JK estimator of V.

Assume that $\boldsymbol{\gamma }\in \Theta $, where Θ is a compact set in ${\mathbb{R}^{2}}$. Then for the elementary estimating function s defined by (8) we obtain where $C<\infty $ is some constant. So Assumption (JK1) holds if $\operatorname{\mathsf{E}}[{({\varepsilon _{j}})^{4}}\hspace{2.5pt}|\hspace{2.5pt}{\kappa _{j}}=k]<\infty $ and $\operatorname{\mathsf{E}}[|{X_{j}}{|^{12}}\hspace{2.5pt}|\hspace{2.5pt}{\kappa _{j}}=k]<\infty $ for all $k=1,\dots ,M$. Assumption (AN5) holds if Assumption (C5) also holds under (18), see Theorem 2 in [8]. So, under rather mild assumptions Theorems 1–4 hold for generalized least squares estimator in this model. In [7] confidence sets for ${\boldsymbol{\vartheta }^{(k)}}$ are constructed based on the asymptotic normality of ${\hat{\boldsymbol{\vartheta }}_{n}^{(k)}}$ and consistency of ${\hat{\mathbf{V}}_{n}^{(k,k)}}$. Namely, the confidence ellipsoid for ${\boldsymbol{\vartheta }^{(k)}}$ is defined as where ${Q^{\eta }}(1-\alpha )$ is the quantile of level $1-\alpha $ of the ${\chi ^{2}}$-distribution with d degrees of freedom. Then under the assumptions of Theorems 1–4, if $\det {\mathbf{Z}^{(k,k)}}\ne 0$, In [7] results of simulations are presented which show that these ellipsoids can be used for samples large enough.

In this section we show how the considered technique can be applied to the statistical analysis of real life data. In many sociological problems one deals with two sets of data from two different sources. The first (I) set consists of individual records with values of variables which present personal information of investigated persons. The second (A) set contain some averaged information on large groups of a variable of investigated persons which is not presented in the I-set. The problem is how to merge information from A- and I-sets to infer on the model involving variables from both sets.

We consider as the I-set a data of results of Ukrainian External Independent Testing (EIT) in 2016 from the official site of Ukrainian Center for Educational Quality Assessment. EIT exams are to be passed by high school graduates for admission to universities. Information on scores in Ukrainian language and literature (Ukr) and on Mathematics (Math) for nearly 246 000 examinees of EIT-2016 is available.1 In [5] linear regression dependence between Ukr and Math is assumed. In this paper ve consider the model in which the coefficients ${\boldsymbol{\vartheta }_{0}^{(k)}}$ and ${\boldsymbol{\vartheta }_{1}^{(k)}}$ depend on the political attitudes of the adult environment in which the student was brought up. There can be a family of Ukrainian independence adherents or an environment critical to existence of Ukrainian state and culture.

EIT-2016 does not contain information on political issues. But for each examine the region of Ukraine is recorded where he/she graduated. So we used data on results of Ukrainian Parliament (Verhovna Rada) elections-2014 to get approximate proportions of adherents of different political choices in regions of Ukraine (A-set). All possible electoral choices at these elections (voting for one of parties, voting against all or not to take part in the voting) were divided into three groups (components): (1) pro-Ukrainian, (2) contra-Ukrainian and (3) neutral (see [5] for details). The concentrations of components are taken as frequencies of adherents of corresponding electoral choice at the region where j-th examinee attended high school. On Fig. 1 the fitted regression lines are presented. The dependence between Math and Ukr on this picture seems significantly different in the three components. Say, in the pro component it is increasing and seemingly nonlinear, in the contra component it is decreasing, in the neutral one it is increasing and quite near to linear dependence.

To verify significance of these differences we constructed the confidence ellipsoids for the parameters as described in Section 4. By the Bonferroni rule, to infer with the significance level ${\alpha _{0}}=0.05$, we took the levels of the ellipsoids $\alpha ={\alpha _{0}}/3=0.01666$. Obtained ellipsoids are presented on Fig. 2. Since they are not intersecting, we conclude that the differences between the parameters are significant for all the components.

So, we obtained conditions under which the JK estimator for asymptotic variance is valid for nonlinear GEE estimators in MVC models. The presented example of sociological data analysis demonstrates possibilities of practical applications of this estimator.

We start from some auxiliary lemmas.

For the proof, see [5], lemma 1.

Let ${\psi _{j;n}}$, $j=1,\dots ,n$, $n=1,2,\dots \hspace{0.1667em}$, be any set of functions with domain ${\mathbb{R}^{D}}$ and values in a set of scalars, or vectors, or matrices. We use the following notation In this notation ψ can be replaced by any other symbol, e.g., $\tilde{\psi }$, s or $\dot{\mathbf{s}}$.