Modern Stochastics: Theory and Applications

In this paper, we investigate the problem of estimating the missed observations of stochastic sequences with stationary increments. Kolmogorov [13], Wiener [27], and Yaglom [29, 30] developed effective methods of estimation of the unknown values of stationary sequences and processes. Later on Yaglom [28] and Pinsker [21] introduced and investigated stochastic processes with stationary increments of order n. Properties of these and other processes generalizing the concept of stationarity are described in the books by Yaglom [29, 30]. The stationary and related stochastic sequences are widely used in econometrics and in financial time series analysis. Examples of these sequences are autoregressive sequences (AR), moving-average sequences (MA), and autoregressive moving-average sequences (ARMA). Time series with trends are described by integrated ARMA sequences (ARIMA) and seasonal time series, which are examples of stochastic sequences with stationary increments. These models are properly described in the book by Box, Jenkins, and Reinsel [2]. Granger [8] introduced a concept of cointegrated sequences, namely, the integrated sequences such that some linear combination of them has a lower order of integration. Cointegrated sequences are described in more details in the paper by Engle and Granger [5]. We also refer to the papers [3, 4, 9, 12] for recent developments.

Traditional methods of finding solutions to extrapolation, interpolation, and filtering problems for stationary and related stochastic processes are developed under the basic assumption that the spectral densities of the considered stochastic processes are exactly known. However, in most practical situations, complete information on the spectral densities of the processes is not available. Investigators can apply the traditional methods considering the estimated spectral densities instead of the true ones. However, as it was shown by Vastola and Poor [26] with the help of some examples, this approach can result in significant increasing of the value of the error of estimate. Therefore, it is reasonable to derive estimates that are optimal for all densities from a certain class of spectral densities. These estimates are called minimax-robust since they minimize the maximum of the mean-square errors for all spectral densities from a set of admissible spectral densities simultaneously. This approach to study the problem of extrapolation of stationary stochastic processes was introduced by Grenander [10]. Franke [6] investigated the minimax extrapolation and interpolation problems for stationary sequences applying the convex optimization methods. In the book by Moklyachuk [20], the minimax-robust estimates of the linear functionals of stationary sequences and processes are presented. See also the survey paper [18], The classical and minimax-robust problems of interpolation, extrapolation, and filtering of the functional of stochastic sequences with stationary increments are investigated in the papers by Luz and Moklyachuk [14–17, 19]. Particularly, the cointegrated sequences are investigated in the papers [14, 15]. The classical extrapolation problem in the case where both the signal and the noise processes are not stationary was investigated by Bell [1].

In the present paper, we consider the problem of estimation of the linear functional which depends on the unknown values of the sequence $\xi (k)$ with stationary nth increments based on observations of the sequence $\xi (k)+\eta (k)$ at points $m\in \mathbb{Z}\setminus \{0,1,2,\dots ,N\}$. The sequence $\eta (k)$ is assumed to be stationary and uncorrelated with $\xi (k)$.

In this section, we present the main results of the spectral theory of stochastic sequences with nth stationary increments. For more details, we refer to the books by Yaglom [29, 30].

Representation (3) and the Karhunen theorem [7] give us a spectral representation of the stationary nth increment sequence ${\xi }^{(n)}(m,\mu )$: where $Z_{{\xi }^{(n)}}(\lambda )$ is a random process with uncorrelated increments on $[-\pi ,\pi )$ with respect to the spectral function $F(\lambda )$:

We will use the spectral representation (4) for deriving the optimal linear estimates of unknown values of stochastic sequences with stationary increments.

Consider a stochastic sequence $\{\xi (m),m\in \mathbb{Z}\}$ with stationary nth increments ${\xi }^{(n)}(m,\mu )$ and uncorrelated with $\xi (m)$ stationary stochastic sequence $\{\eta (m),m\in \mathbb{Z}\}$. Suppose that these sequences have absolutely continuous spectral functions $F(\lambda )$ and $G(\lambda )$ with spectral densities $f(\lambda )$ and $g(\lambda )$, respectively. We will suppose that the stationary increment ${\xi }^{(n)}(m,\mu )$ and the stationary sequence $\eta (m)$ have zero mean values and $\mu >0$.

Interpolation problem for the sequences $\xi (m)$ and $\eta (m)$ is considered as the problem of the mean-square optimal estimation of the linear functional which depends on the unknown values of the stochastic sequence $\xi (m)$ at points $m=0,1,\dots ,N$ based on observations of the sequence $\zeta (m)=\xi (m)+\eta (m)$ at points of the set $\mathbb{Z}\setminus \{0,1,2\dots ,N\}$.

Suppose that the spectral densities $f(\lambda )$ and $g(\lambda )$ satisfy the minimality condition Under this condition, the mean-square error of the estimate of the functional $A_{N}\xi $ is not equal to zero [24].

The functional $A_{N}\xi $ admits the representation where The coefficients $v_{\mu ,N}(k)$, $k=-\mu n,-\mu n+1,\dots ,-1$, and $b_{\mu ,N}(k)$, $k=0,1,2,\dots ,N$, are calculated by the formulas (see [15]) where by ${[x]^{\prime }}$ we denote the least integer number among the numbers that are greater than or equal to x, the coefficients $\{d_{\mu }(k):k\ge 0\}$ are determined by the relationship the matrix ${D_{N}^{\mu }}$ of dimension $(N+1)\times (N+1)$ is defined by the coefficients $({D_{N}^{\mu }})_{k,j}=d_{\mu }(j-k)$ if $0\le k\le j\le N$, and $({D_{N}^{\mu }})_{k,j}=0$ if $0\le j<k\le N$; and $\mathbf{a}_{N}={(a(0),a(1),a(2),\dots ,a(N))^{\prime }}$ is a vector of dimension $(N+1)$.

The functional $H_{N}\xi $ from representation (6) has finite variance, and the functional $V_{N}\zeta $ depends on the known observations of the stochastic sequence $\zeta (k)$ at the points $k=-\mu n,-\mu n+1,\dots ,-1$. Therefore, optimal estimates $\widehat{A}_{N}\xi $ and $\widehat{H}_{N}\xi $ of the functionals $A_{N}\xi $ and $H_{N}\xi $ and the mean-square errors $\varDelta (f,g;\widehat{A}_{N}\xi )=\mathsf{E}|A_{N}\xi -\widehat{A}_{N}\xi {|}^{2}$ and $\varDelta (f,g;\widehat{H}_{N}\xi )=\mathsf{E}|H_{N}\xi -\widehat{H}_{N}\xi {|}^{2}$ of the estimates $\widehat{A}_{N}\xi $ and $\widehat{H}_{N}\xi $ satisfy the following relations: Thus, the interpolation problem for the functional $A_{N}\xi $ is equivalent to the interpolation problem for the functional $H_{N}\xi $. This problem can be solved by applying the Hilbert space projection method proposed by Kolmogorov [13]. The optimal linear estimate $\widehat{A}_{N}\xi $ of the functional $A_{N}\xi $ can be represented in the form where $h_{\mu }(\lambda )$ is the spectral characteristic of the optimal estimate $\widehat{H}_{N}\xi $.

Let ${H}^{0-}({\xi _{\mu }^{(n)}}+{\eta _{\mu }^{(n)}})$ be the closed linear subspace generated by elements $\{{\xi }^{(n)}(k,\mu )+{\eta }^{(n)}(k,\mu ):k\le -1\}$ of the Hilbert space $H=L_{2}(\varOmega ,\mathcal{F},\mathsf{P})$ of random variables γ with zero mean value and finite variance, $\mathsf{E}\gamma =0$, $\mathsf{E}|\gamma {|}^{2}<\infty $, with the inner product $(\gamma _{1};\gamma _{2})=\mathsf{E}\gamma _{1}\overline{\gamma _{2}}$. Let ${H}^{N+}({\xi _{-\mu }^{(n)}}+{\eta _{-\mu }^{(n)}})$ be the closed linear subspace of the Hilbert space $H=L_{2}(\varOmega ,\mathcal{F},\mathsf{P})$ generated by elements $\{{\xi }^{(n)}(k,-\mu )+{\eta }^{(n)}(k,-\mu ):k\ge N+1\}$. The equality ${\xi }^{(n)}(k,-\mu )={(-1)}^{n}{\xi }^{(n)}(k+\mu n,\mu )$ implies Let us also define the subspaces ${L_{2}^{0-}}(p)$ and ${L_{2}^{N+}}(p)$ of the Hilbert space $L_{2}(p)$ with the inner product $(x_{1};x_{2})={\int _{-\pi }^{\pi }}x_{1}(\lambda )\overline{x_{2}(\lambda )}p(\lambda )d\lambda $ that are generated by the functions $\{{e}^{i\lambda k}{(1-{e}^{-i\lambda \mu })}^{n}{(i\lambda )}^{-n}:k\le -1\}$ and $\{{e}^{i\lambda k}{(1-{e}^{-i\lambda \mu })}^{n}{(i\lambda )}^{-n}:k\ge N+1\}$, respectively, where the function is the spectral density of the sequence $\zeta (m)$, $m\in \mathbb{Z}$ [15]. The optimal estimate $\widehat{H}_{N}\xi $ of the functional $H_{N}\xi $ is the projection of the element $H_{N}\xi $ of the Hilbert space $H=L_{2}(\varOmega ,\mathcal{F},\mathsf{P})$ onto the subspace The following conditions characterize the estimate $\widehat{H}_{N}\xi $: The functional $H_{N}\xi $ in the space H admits the spectral representation Making use of the described representation and condition 2), we derive the following equation for determining the spectral characteristic $h_{\mu }(\lambda )$: Thus, the spectral characteristic $h_{\mu }(\lambda )$ can be represented as follows: where $c_{\mu }(k)$, $k=0,1,2,\dots ,N+\mu n$, are unknown coefficients we have to determine. Condition 1) implies that the spectral characteristic $h_{\mu }(\lambda )$ satisfies the following equations: The derived equations are represented as a system of $N+\mu n+1$ linear equations: where the coefficients $\{a_{\mu ,N}(m):0\le m\le N+\mu n\}$ are calculated by the formula and the Fourier coefficients $\{{T_{k,j}^{\mu }},{P_{k,j}^{\mu }}:0\le k,j\le N+\mu n\}$ are calculated by the formulas

Denote by $[{D_{N}^{\mu }}\mathbf{a}_{N}]_{+\mu n}$ the vector of dimension $(N+\mu n+1)$ constructed by adding $\mu n$ zeros to the vector ${D_{N}^{\mu }}\mathbf{a}_{N}$ of dimension $(N+1)$. Using these definitions, system (12)–(13) can be represented in the matrix form where and are vectors of dimension $(N+\mu n+1)$; and ${\mathbf{P}_{N}^{\mu }}$ and ${\mathbf{T}_{N}^{\mu }}$ are matrices of dimension $(N+\mu n+1)\times (N+\mu n+1)$ with elements $({\mathbf{P}_{N}^{\mu }})_{l,k}={P_{l,k}^{\mu }}$ and $({\mathbf{T}_{N}^{\mu }})_{l,k}={T_{l,k}^{\mu }}$, $0\le l,k\le N+\mu n$. Thus, the coefficients $c_{\mu }(k)$, $0\le k\le N+\mu n$, are determined by the formula where $({({\mathbf{P}_{N}^{\mu }})}^{-1}[{D_{N}^{\mu }}\mathbf{a}_{N}]_{+\mu n}-{({\mathbf{P}_{N}^{\mu }})}^{-1}{\mathbf{T}_{N}^{\mu }}{\mathbf{a}_{N}^{\mu }})_{k}$, $0\le k\le N+\mu n$, is the kth element of the vector ${({\mathbf{P}_{N}^{\mu }})}^{-1}[{D_{N}^{\mu }}\mathbf{a}_{N}]_{+\mu n}-{({\mathbf{P}_{N}^{\mu }})}^{-1}{\mathbf{T}_{N}^{\mu }}{\mathbf{a}_{N}^{\mu }}$. The existence of the invertible matrix ${({\mathbf{P}_{N}^{\mu }})}^{-1}$ was shown in [25] under condition (5). The spectral characteristic $h_{\mu }(\lambda )$ of the estimate $\widehat{H}_{N}\xi $ of the functional $H_{N}\xi $ is calculated by formula (11), where The value of the mean-square errors of the estimates $\widehat{A}_{N}\xi $ and $\widehat{H}_{N}\xi $ can be calculated by the formula where $\mathbf{Q}_{N}$ is the matrix of dimension $(N+1)\times (N+1)$ with the coefficients $(\mathbf{Q}_{N})_{l,k}=Q_{l,k}$, $0\le l,k\le N$, calculated by the formula

We can summarize the derived results in the form of the following theorem.

In the case of estimation of an unobserved value $\xi (p)$, $0\le p\le N$, the following statement holds true.

Consider two integrated sequences $\{\xi (m),m\in \mathbb{Z}\}$ and $\{\zeta (m),m\in \mathbb{Z}\}$ with absolutely continuous spectral functions $F(\lambda )$ and $P(\lambda )$ and the corresponding spectral densities $f(\lambda )$ and $p(\lambda )$.

The interpolation problem for cointegrated sequences consists in mean-square optimal linear estimation of the functional of unknown values of the stochastic sequence $\xi (m)$ based on observations of the stochastic sequence $\zeta (m)$ at the points $m\in \mathbb{Z}\setminus \{0,1,2,\dots ,N\}$. To solve the problem, we can use the results obtained in the previous sections.

Suppose that the spectral density $p(\lambda )$ of the sequence $\zeta (m)$ satisfies the minimality condition Let the matrices ${\mathbf{P}_{N}^{\mu ,\beta }}$, ${\mathbf{T}_{N}^{\mu ,\beta }}$, ${\mathbf{Q}_{N}^{\beta }}$ be defined by the Fourier coefficients of the functions in the same way as the matrices ${\mathbf{P}_{N}^{\mu }}$, ${\mathbf{T}_{N}^{\mu }}$, $\mathbf{Q}_{N}$ were defined. Theorem 2 implies the following formula for calculating the spectral characteristic ${h_{\mu ,N}^{\beta }}(\lambda )$ of the optimal estimate of the functional $A_{N}\xi $: where

The value of the mean-square error of the estimate $\widehat{A}_{N}\xi $ is calculated by the formula

The described results are presented as the following theorem.

Formulas for calculating values of the mean-square error $\varDelta (h(f,g);f,g)=\varDelta (f,g;\widehat{A}_{N}\xi )=\mathsf{E}|A_{N}\xi -\widehat{A}_{N}\xi {|}^{2}$ and the spectral characteristics of the optimal estimates of the functional $A_{N}\xi $ based on observations of the sequence $\xi (m)+\eta (m)$ can be applied under the condition that the spectral densities $f(\lambda )$ and $g(\lambda )$ of the stochastic sequences $\xi (m)$ and $\eta (m)$ are known. However, these formulas often cannot be used in many practical situations since the exact values of the densities are not available. In this situation, the minimax-robust method can be applied. It consists in finding the estimate that provides a minimum of the mean-square errors for all spectral densities from a given set $\mathcal{D}=\mathcal{D}_{f}\times \mathcal{D}_{g}$ of admissible spectral densities simultaneously.

The presented statements follow from the introduced definitions and Theorem 2.

The minimax-robust spectral characteristic ${h}^{0}$ and the least favorable spectral densities $({f}^{0},{g}^{0})$ form a saddle point of the function $\varDelta (h;f,g)$ on the set $H_{\mathcal{D}}\times \mathcal{D}$. The saddle-point inequalities hold if ${h}^{0}=h_{\mu }({f}^{0},{g}^{0})$, $h_{\mu }({f}^{0},{g}^{0})\in H_{\mathcal{D}}$, and $({f}^{0},{g}^{0})$ is a solution to the constrained optimization problem This constrained optimization problem is equivalent to the unconstrained optimization problem where $\varDelta (f,g|\mathcal{D}_{f}\times \mathcal{D}_{g})$ is the indicator function of the set $\mathcal{D}_{f}\times \mathcal{D}_{g}$: $\varDelta (f,g|\mathcal{D}_{f}\times \mathcal{D}_{g})=0$ if $(f;g)\in \mathcal{D}_{f}\times \mathcal{D}_{g}$ and $\varDelta (f,g|\mathcal{D}_{f}\times \mathcal{D}_{g})=+\infty $ if $(f;g)\notin \mathcal{D}_{f}\times \mathcal{D}_{g}$.

A solution $({f}^{0},{g}^{0})$ to the unconstrained optimization problem is determined by the condition $0\in \partial \varDelta _{\mathcal{D}}({f}^{0},{g}^{0})$, which is a necessary and sufficient condition that the pair $({f}^{0},{g}^{0})$ belongs to the set of minimums of the convex functional $\varDelta _{\mathcal{D}}(f,g)$ [11, 22, 23]. By $\partial \varDelta _{\mathcal{D}}(f,g)$ we denote the subdifferential of the functional $\varDelta _{\mathcal{D}}(f,g)$ at the point $(f,g)=({f}^{0},{g}^{0})$, that is, the set of all linear continuous functionals Λ on the space $L_{1}\times L_{1}$ that satisfy the inequality

In the case of estimating the cointegrated sequences, we have the following optimization problem of finding the least favorable spectral densities: A solution $({f}^{0},{p}^{0})$ to this optimization problem is characterized by the condition $0\in \partial \varDelta _{\mathcal{D}}({f}^{0},{p}^{0})$.

The derived representations of the linear functionals $\varDelta (h_{\mu }({f}^{0},{g}^{0});f,g)$ and $\varDelta ({h_{\mu }^{\beta }}({f}^{0},{p}^{0});f,p)$ allow us to calculate derivatives and subdifferentials in the space $L_{1}\times L_{1}$. Therefore, the complexity of the optimization problems (26) and (27) is determined by the complexity of calculation of the subdifferentials of the indicator functions $\varDelta (f,g|\mathcal{D}_{f}\times \mathcal{D}_{g})$ and $\varDelta (f,p|\mathcal{D}_{f}\times \mathcal{D}_{p})$ of the sets $\mathcal{D}_{f}\times \mathcal{D}_{g}$ and $\mathcal{D}_{f}\times \mathcal{D}_{p}$.

Consider the problem of minimax-robust estimation of the functional $A_{N}\xi $ of unknown values of the sequence with stationary increments $\xi (m)$ based on observations of the sequence $\xi (m)+\eta (m)$ at the points $m\in \mathbb{Z}\setminus \{0,1,2,\dots ,N\}$ for the set of admissible spectral densities $\mathcal{D}={\mathcal{D}_{0,f}^{-}}\times {\mathcal{D}_{0,g}^{-}}$, where If the spectral densities ${f}^{0}\in {\mathcal{D}_{0,f}^{-}}$, ${g}^{0}\in {\mathcal{D}_{0,g}^{-}}$ and the functions where ${p}^{0}(\lambda )={f}^{0}(\lambda )+{\lambda }^{2n}{g}^{0}(\lambda )$, are bounded, then the linear functional $\varDelta (h_{\mu }({f}^{0},{g}^{0});f,g)$ is continuous and bounded in the space $L_{1}\times L_{1}$. The condition $0\in \partial \varDelta _{\mathcal{D}}({f}^{0},{g}^{0})$ implies that the spectral densities ${f}^{0}\in {\mathcal{D}_{0,f}^{-}}$ and ${g}^{0}\in {\mathcal{D}_{0,g}^{-}}$ are determined by the relations where the constants $\alpha _{1}\ge 0$, $\alpha _{2}\ge 0$ with $\alpha _{1}\ne 0$ if ${\int _{-\pi }^{\pi }}{({f}^{0}(\lambda ))}^{-1}d\lambda =2\pi P_{1}$ and $\alpha _{2}\ne 0$ if ${\int _{-\pi }^{\pi }}{({g}^{0}(\lambda ))}^{-1}d\lambda =2\pi P_{2}$.

The derived statements allow us to formulate the following theorems.

Consider the problem of minimax-robust estimation of the functional $A_{N}\xi $ of unknown values of the sequence $\xi (m)$, cointegrated with the sequence $\zeta (m)$, based on observations of the sequence $\zeta (m)$ at the points $m\in \mathbb{Z}\setminus \{0,1,2,\dots ,N\}$. Suppose that the stochastic sequences $\xi (m)$ and $\zeta (m)-\beta \xi (m)$ are uncorrelated. The least favorable spectral densities in the class ${\mathcal{D}_{f}^{0}}\times {\mathcal{D}_{p}^{0}}$, where are determined by the condition $0\in \partial \varDelta _{\mathcal{D}}({f}^{0},{p}^{0})$, which implies the following relations for determining the least favorable spectral densities ${f}^{0}\in {\mathcal{D}_{f}^{0}}$ and ${p}^{0}\in {\mathcal{D}_{p}^{0}}$: where the constants $\alpha _{1}\ge 0$, $\alpha _{2}\ge 0$ with $\alpha _{1}\ne 0$ if ${\int _{-\pi }^{\pi }}{({f}^{0}(\lambda ))}^{-1}d\lambda =2\pi P_{1}$ and $\alpha _{2}\ne 0$ if ${\int _{-\pi }^{\pi }}{({p}^{0}(\lambda ))}^{-1}d\lambda =2\pi P_{2}$.

Consider the problem of minimax-robust interpolation of the functional $A_{N}\xi $ based on observations of the sequence $\xi (m)+\eta (m)$ at the points of $m\in \mathbb{Z}\setminus \{0,1,2,\dots ,N\}$ in the case where the spectral densities $f(\lambda )$ and $g(\lambda )$ belong to the set $\mathcal{D}=\mathcal{D}_{2\varepsilon _{1}}\times \mathcal{D}_{1\varepsilon _{2}}$, where are ε-neighborhoods of the given spectral densities $f_{1}(\lambda )$ and $g_{1}(\lambda )$ in the spaces $L_{2}$ and $L_{1}$, respectively.

Suppose that the spectral densities $f_{1}(\lambda )$ and $g_{1}(\lambda )$ are bounded and the functions $h_{\mu ,f}({f}^{0},{g}^{0})$ and $h_{\mu ,g}({f}^{0},{g}^{0})$ calculated by formulas (28) and (29) with spectral densities ${f}^{0}\in \mathcal{D}_{2\varepsilon _{1}}$ and ${g}^{0}\in \mathcal{D}_{1\varepsilon _{2}}$ are bounded as well. The condition $0\in \partial \varDelta _{\mathcal{D}}({f}^{0},{g}^{0})$ implies the following relations for determining the least favorable spectral densities: where the function $|\gamma (\lambda )|\le 1$ and $\gamma (\lambda )=\text{sign}(g(\lambda )-g_{1}(\lambda ))$ if $g(\lambda )\ne g_{1}(\lambda )$; $\alpha _{1}$, $\alpha _{2}$ are two constants to be found using the equations

Now we can present the following theorems, which describe the least favorable spectral densities in the class $\mathcal{D}=\mathcal{D}_{2\varepsilon _{1}}\times \mathcal{D}_{1\varepsilon _{2}}$.

Consider the problem of minimax-robust interpolation of the functional $A_{N}\xi $ in the case of cointegrated sequences $\xi (m)$ and $\zeta (m)$ on the set of admissible spectral densities $\mathcal{D}=\mathcal{D}_{2\varepsilon _{1}}\times \mathcal{D}_{1\varepsilon _{2}}$, where From the condition $0\in \partial \varDelta _{\mathcal{D}}({f}^{0},{g}^{0})$ we obtain the following relations that determine the least favorable spectral densities: where the function $|\gamma (\lambda )|\le 1$ and $\gamma (\lambda )=\text{sign}(p(\lambda )-p_{1}(\lambda ))$ if $p(\lambda )\ne p_{1}(\lambda )$; $\alpha _{1}$, $\alpha _{2}$ are two constants that can be found from the equations

Thus, we have the following theorem.

In the article, the problem of the mean-square optimal linear estimation of the functional $A_{N}\xi ={\sum _{k=0}^{N}}a(k)\xi (k)$, which depends of unknown values of the sequence $\xi (m)$ with nth stationary increments based on observations of the sequence $\xi (m)+\eta (m)$ at the points $m\in \mathbb{Z}\setminus \{0,1,2,\dots ,N\}$, is considered in the case of observations with the stationary noise $\eta (m)$ uncorrelated with $\xi (m)$. The classical and minimax-robust methods of interpolation are applied in the case of spectral certainty and in the case spectral uncertainty. Particularly, in the case of spectral certainty, formulas for calculating the spectral characteristics and the value of the mean-square error of the optimal estimate are found. The derived results are applied to interpolation problem for a class of cointegrated sequences. In the case spectral uncertainty, where spectral densities are not known exactly, whereas some sets of admissible spectral densities are given, formulas that determine the least favorable spectral densities and the minimax-robust spectral characteristics are derived for some special sets of admissible spectral densities.