Modern Stochastics: Theory and Applications

Theory of large deviations in mathematical statistics and statistics of stochastic processes deals with the asymptotic behaviour of tails of distribution functions of parametric and nonparametric statistical estimators. Concerning parametric estimators it is necessary to refer to the monograph of Ibragimov and Has’minskii [6] where the exponential convergence rate of probabilities of large deviations for maximum likelihood estimator was obtained. This result led to the appearance of a large number of publications on the subject of large deviations of statistical estimators.

Further we will speak about least squares estimators (l.s.e.’s) for parameters of a nonlinear regression model. In the paper of Ivanov [8] a statement was proved on the power decreasing rate for probabilities of large deviations of l.s.e. for a scalar parameter in the nonlinear regression model with i.i.d. observation errors having moments of finite order. Prakasa Rao [13] obtained a similar result with the exponential decreasing rate in the Gaussian nonlinear regression.

In the paper of Sieders and Dzhaparidze [15] a general Theorem 2.1 on probabilities of large deviations for M-estimators based on a data set of any structure was proved that generalizes the mentioned result in [6] with an application to l.s.e. for parameters of the nonlinear regression with pre-Gaussian and sub-Gaussian i.i.d. observation errors (Theorems 3.1 and 3.2 in [15]). Some results in this direction are obtained by Ivanov [9].

The results on probabilities of large deviations of an l.s.e. in a nonlinear regression model with correlated observations can be found in the works of Ivanov and Leonenko [11], Prakasa Rao [14], Hu [4], Yang and Hu [16], Huang et al. [5].

Upper exponential bounds for probabilities of large deviations of an l.s.e. for a parameter of the nonlinear regression in discrete-time models with a jointly strictly sub-Gaussian (j.s.s.-G.) random noise were obtained in Ivanov [10]. In the present paper we extend some results of [10] to continuous-time observation models.

Consider a regression model where $a(t,\hspace{0.1667em}\tau )$, $(t,\hspace{0.1667em}\tau )\in {\mathbb{R}_{+}}\times {\varTheta }^{c}$, is a continuous function, a true parameter value $\theta ={({\theta _{1}},\dots ,{\theta _{q}})}^{\prime }$ belongs to an open bounded convex set $\varTheta \subset {\mathbb{R}}^{q}$ and a random noise $\varepsilon =\{\varepsilon (t),\hspace{2.5pt}t\in \mathbb{R}\}$ satisfies the following condition.

N1. ε is a mean-square and almost sure (a.s.) continuous stochastic process defined on the probability space $(\varOmega ,\hspace{2.5pt}\mathfrak{F},\hspace{2.5pt}P)$, $E\varepsilon (t)=0$, $t\in \mathbb{R}$.

We shall write $\int ={\int _{0}^{T}}$.

Under assumptions introduced above there exists at least one such random vector ${\theta _{T}}$ [12].

In the asymptotic theory of nonlinear regression in the problem of normal approximation of the distribution of an l.s.e., the difference ${\theta _{T}}-\theta $ is normed by diagonal matrix [11] Further it is supposed that the function $a(t,\hspace{0.1667em}\cdot )\in {C}^{1}(\varTheta )$ for any $t\ge 0$.

The paper is organized in the following way. In Section 2 an upper exponential bound is obtained for large deviations of ${d_{T}}(\theta )({\theta _{T}}-\theta )$ in the regression model (1) with a j.s.s.-G. random noise ε. In Section 3 the results of Section 2 are applied to a stationary j.s.s.-G. noise ε. Section 4 contains examples of regression functions a and noise ε satisfying the conditions of our theorems.

Note that we obtain from Definition 2 the definition of an s.s.-G. random variable (r.v.) ξ taking $n=1$.

These definitions and a more detailed information on sub-Gaussian r.v.’s, vectors and stochastic processes can be found in the monograph [1] by Buldygin and Kozachenko.

Concerning the random noise ε in the model (1) we introduce the following assumption.

N2(i) ε is a j.s.s.-G. stochastic process with the covariance function $B(t,s)=E\varepsilon (t)\varepsilon (s)$, $t,s\in \mathbb{R}$.

(ii) For any $T>0$, $\varDelta (\cdot )\in {L_{2}}([0,T])$ for some constant ${d_{0}}>0$, $\| \varDelta {\| _{T}}={(\int \hspace{0.2778em}{\varDelta }^{2}(t)dt)}^{\frac{1}{2}}$.

For a fixed T the exact bound in (2) is where $\| B{\| _{T}}$ is the norm of a self-adjoint positive semidefinite operator B in ${L_{2}}([0,T])$. Note that $\| B{\| _{T}}$ is a nondecreasing function of $T>0$, so there exists if (2) is fulfilled.

Let $\varDelta (t),\hspace{2.5pt}t\in {\mathbb{R}_{+}}$, be a continuous function. Then condition N1 implies the existence of the integral determined for almost all paths of the process $\varepsilon (t),\hspace{2.5pt}t\in [0,T]$, as the Riemann integral. Consider partitions $r(n)$ of the interval $[0,T]$ such that ${\max _{1\le k\le n}}({t_{k}^{(n)}}-{t_{k-1}^{(n)}})\to 0$, as $n\to \infty $, and the corresponding integral sums Then

It is obvious also that

The following statement on the exponential bound for distribution tails of integrals (3) plays an important role in subsequent proofs.

We need some notation to formulate conditions on the regression function $a(t,\hspace{0.1667em}\theta )$ using the approach of the paper [15] (see also [9, 11]). Write $u={({u_{1}},\hspace{2.5pt}\dots ,\hspace{2.5pt}{u_{q}})}^{\prime }\in {\mathbb{R}}^{q}$. Denote by G the family of all functions $g={g_{T}}(R)$, $T>0$, $R>0$, having the following properties:

1) for fixed T, ${g_{T}}(R)\uparrow \infty $, as $R\to \infty $;

2) for any $r>0$, Let $\gamma (R)$ be polynomials of R (possibly different) with coefficients that do not depend on values $T,\hspace{2.5pt}\theta ,\hspace{2.5pt}u,\hspace{2.5pt}v$. Set also

Assume the existence of a function $g\in G$, constants $\delta \in (0,\frac{1}{2})$, $\varkappa >0$, $\rho \in (0,1]$ and polynomials $\gamma (R)$ such that for sufficiently large $T,\hspace{2.5pt}R$ (we will write $T>{T_{0}}$, $R>{R_{0}}$) the following conditions are fulfilled.

R1(i) For any $u,v\in {\varGamma _{T,\theta ,R}}$ such that $\| u-v\| \le \varkappa $

(ii) for any $u\in {\varGamma _{T,\theta ,R}}$ ${\varPhi _{T}}(u,0)\le \gamma (R)$.

R2. For any $u\in {\varGamma _{T,\theta ,R}}$

Suppose there exist a diagonal matrix ${s_{T}}=\operatorname{diag}({s_{iT}},\hspace{2.5pt}i=\overline{1,q})$ with elements that do not depend on $\tau \in \varTheta $, and constants $0<{\underline{c}_{i}}<{\overline{c}_{i}}<\infty $, $i=\overline{1,q}$, such that uniformly in $\tau \in \varTheta $ for $T>{T_{0}}$ Then instead of the matrix ${d_{T}}(\theta )$ (at least in the framework of the topic of this paper) it is possible to consider, without loss of generality, the normalizing matrix ${s_{T}}$.

The next condition is more restrictive than R1 and R2, however it is simpler due to requirement (19).

R3. There exist numbers $0<{c_{0}}<{c_{1}}<\infty $ such that for any $u,v\in {U_{T}}(\theta )={s_{T}}({\varTheta }^{c}-\theta )$ and $T>{T_{0}}$,

It goes without saying that in the expression for ${\varPhi _{T}}(u,v)$ in (20) we use the matrix ${s_{T}^{-1}}$ instead of ${d_{T}^{-1}}(\theta )$.

A condition of the type (20) has been introduced in [8] and used in [13, 15, 4] and other works. The next theorem generalizes Theorem 3.2 from [15].

We impose an additional restriction on the noise process ε.

N3. The stochastic process ε is stationary with the covariance function $B(t)=E\varepsilon (0)\varepsilon (t),\hspace{2.5pt}t\in \mathbb{R}$, and the bounded spectral density $f(\lambda ),\hspace{2.5pt}\lambda \in \mathbb{R}$:

Under assumption N3 the following corollaries of the theorems proved in Section 2 are true.

Our next assumption is a particularization of the requirements N2 and N3.

N4(i). The random noise ε is of the form where $\xi =\{\xi (t),\hspace{2.5pt}t\in \mathbb{R}\}$ is a mean-square continuous j.s.s.-G. stochastic process with orthogonal increments, $E\xi (t)=0$, $\psi (t)=0$ as $t<0$ and

The stochastic integral in (24) is understood as a mean-square Stieltjes integral [3]. The process ξ is an integrated white noise, ε can be considered as a stationary process at the output of a physically realizable filter with the covariance function (see ibid.) and the spectral density

N4(ii). ${f_{0}}={\sup _{\lambda \in \mathbb{R}}}|h(i\lambda ){|}^{2}<\infty $.

Obviously N4(ii) holds, if ${\int _{0}^{\infty }}\hspace{0.1667em}|\psi (t)|dt<\infty $.

Assume

For $\nu =0$ we arrive at a quite strong result on the weak consistency of l.s.e. Similarly, in the conditions of Corollary 5 the following result on probabilities of moderate deviations for l.s.e. holds: for any $h>0$, $T>{T_{0}}$

Obviously, Gaussian stochastic processes ε are j.s.s.-G. ones, and all the previous results are valid for them.

In this section, we consider an example of a regression function satisfying the condition R3 and an example of the j.s.s.-G. process ξ from expression (24) in condition N4(i).