Modern Stochastics: Theory and Applications

The classical autoregressive model ${X_{n+1}}=\alpha {X_{n}}+{\varepsilon _{n}}$, where α is a constant and ${\varepsilon _{n}}$ are i.i.d. standard normal random variables, is well studied, and it is known, in particular, that for $\alpha \in (0,1)$ the corresponding Markov chain is geometrically recurrent, positive and ergodic.

However, in real applications, we cannot always guarantee that all ${\varepsilon _{n}}$ are standard normal i.i.d. random variables. In addition, parameter α may not always be the same for all ${X_{n}}$. From the theoretical standpoint, it is interesting to study the behavior of such a model when α is “oscillating” around unity and ${\varepsilon _{n}}$ are independent but not necessarily normal and identically distributed.

So, we come to the model ${X_{n+1}}={\alpha _{n}}{X_{n}}+{\varepsilon _{n}}$, where ${\alpha _{n}}$ are some numbers “around” unity and ${\varepsilon _{n}}$ are independent random variables. Our goal is to find the conditions which guarantee the geometric recurrence of such a chain and study its stability in terms of the stability of n-steps transition probability functions.

Usually, when studying the recurrence of Markov chains, we use techniques developed in the classical theory based on a drift condition (see, for example, [5, 24]). However, the process of interest is time-inhomogeneous, so the aforementioned techniques are not applicable. That’s why we use a particular time-inhomogeneous drift condition developed in the paper [13]. In order to establish geometric recurrence, we use results from [12].

The general theory for inhomogeneous Markov chains is much more involved than its homogeneous counterpart. One of the most popular instruments in research is the coupling method (see classical books by T. Lindvall [22] and H. Thorisson [25]). An interested reader may find an example of applying the coupling method to the stability of Markov chains on a countable phase space in both homogeneous and inhomogeneous cases in the papers [15, 16, 18, 20]. The papers [1, 3, 7] are devoted to the calculation of convergence rates for subgeometrically ergodic, homogeneous Markov chains, and [4] addresses the inhomogeneous case. All these works use the coupling method. We heavily rely on the coupling method in the present paper. We present a modified coupling construction that enables us to couple two different inhomogeneous chains. Such modified coupling generates an inhomogeneous renewal process which we use in our analysis. We can refer to the papers [14, 17] for more details about inhomogeneous renewal processes. Generally speaking, renewal processes have been used for a long time to study Markov chains. See, for example, [2]. The papers [6, 8, 9, 19, 21] are also related to the inhomogeneous case. Another interesting example of using the renewal theory for the analysis of Markov chains with applications in statistics is [23]. Since general inhomogeneous renewal theory is not well-developed, we use some techniques that are specially adapted to the problem under consideration. One of the key techniques that helps to handle inhomogeneity is stochastic domination. Building a dominating sequence is a key aspect of our development. See more details about this subject in [11] and [10].

Summarizing the methods described above, we can outline the plan for this paper. First, we use the modified drift condition from [13] to establish geometric recurrence of the autoregressive model. Then, we obtain an estimate for ${\mathbb{E}_{x}}[{\beta ^{{\sigma _{C}}}}]$ for some $\beta >1$, where ${\sigma _{C}}$ is the first return time to some “recurrence” set C. Second, we use the result from [12] to establish a similar estimate for a pair of autoregressive processes, i.e. ${\mathbb{E}_{x,y}}[{\beta ^{{\sigma _{C\times C}}}}]$. Third, we introduce the coupling schema for two inhomogeneous processes and employ the renewal theory for the stability analysis. Finally, we construct a domination sequence (see Lemmas 1–4) and use it to obtain the stability estimate. We will show that n-steps transition probabilities for inhomogeneous processes will be close, assuming that one-step transition probabilities are also close.

This paper is organized as follows. Section 2 contains main definitions and notations. In Section 3, we present the geometric recurrence result. Section 4 contains the main result, the stability estimate. Section 5 includes technical auxiliary lemmas. Finally, the appendix contains the example of calculating all the necessary constants in a practical application.

Throughout this paper, we assume that all random variables are defined on a common probability space $(\Omega ,ℱ,\mathbb{P})$. In Section 3 we deal with an inhomogeneous Markov chain ${({X_{n}})_{n\ge 0}}$ with values in a general phase space $(E,ℰ)$. We denote the chain’s transition probability by When it is convenient, we will understand ${P_{n}}$ as an operator acting on an appropriate functional space as

By ${\mathbb{P}_{x}}\{X\in \cdot \}$ we denote the probability measure generated by the process ${({X_{n}})_{n\ge 0}}$ which starts at $x\in E$. Thus, for example We denote by ${\mathbb{E}_{x}}$ the corresponding expectation. Strictly speaking, ${\mathbb{P}_{x}}$ is defined on a measurable space $({E^{\infty }},{ℰ^{\infty }})$, and for every $w=({w_{0}},{w_{1}},\dots )\in {E^{\infty }}$ we have a canonical variable ${X_{n}}(w)={w_{n}}$. Thus, rigorously speaking ${X_{n}}$, in the context of $\mathbb{P}$ and ${\mathbb{P}_{x}}$, represents two different random variables defined on different probability spaces. However, such notation is very convenient and helps to develop the proper intuition.

Next, we need a notation for a process conditioned on ${X_{n}}=x$, i.e. Note that in the homogeneous case, this would be the same ${\mathbb{P}_{x}}$ introduced above. But in our inhomogeneous case, we must maintain the index n in the notation since, generally speaking, probabilities are not equal if $k\ne n$. At the same time, we would like to stick to the homogeneous notation as closely as possible. Thus, with some abuse of notation, we introduce probabilities ${\mathbb{P}_{x}^{n}}\{{({X_{k}})_{k\ge 0}}\in \cdot \}$, which mean We will denote the corresponding expectation by ${\mathbb{E}_{x}^{n}}$. It is important to note that, unlike the homogeneous case, ${\mathbb{P}_{x}^{n}}$ and ${\mathbb{P}_{x}^{m}}$ are probabilities defined on a different probability spaces. Thus, we can have a “homogeneous-style” derivation like

In Section 4 and later we deal with a pair of inhomogeneous Markov chains ${\left({X_{n}^{(1)}},{X_{n}^{(2)}}\right)_{n\ge 0}}$. In this case, we will add a subscript i to probabilities and expectations related to the chain ${X^{(i)}}$, $i\in \{1,2\}$, thus having ${P_{i,n}}$, the transition probability, ${\mathbb{P}_{i,x}}$ and ${\mathbb{P}_{i,x}^{n}}$, the canonical probabilities generated by the chain ${X_{n}^{(i)}}$. In case we need to operate with both chains simultaneously, we will use the notation As usual, we denote by ${\mathbb{E}_{x,y}^{n}}$ the corresponding expectation.

In this section we deal with a single time-inhomogeneous Markov chain ${({X_{n}})_{n\ge 1}}$ with values in $(\mathbb{R},ℬ)$ where ℬ is a Borel σ-field.

The main result of this section is presented in the following theorem.

The following immediate corollary could be useful in practical applications.

In this section we consider a set of independent random variables ${W_{n}^{(i)}}:\Omega \to \mathbb{R}$ with distributions ${\Gamma _{n}^{(i)}}(A)=\mathbb{P}\{{W_{n}^{(i)}}\in A\}$ and a pair of Markov chains $n\ge 1$, $i\in \{1,2\}$, ${x_{0}^{(i)}}\in \mathbb{R}$.

Our goal is to demonstrate that where ε defines how “close” are families $\left\{{\Gamma _{n}^{(1)}},n\ge 0\right\}$ and $\left\{{\Gamma _{n}^{(2)}},n\ge 1\right\}$.

We define ε by

It is important to emphasize that one-step transition probabilities for two chains do not coincide when $n\to \infty $ and remain different. Strictly speaking, ε is an internal characteristic of a pair of processes and “changing” ε means shifting to another pair of processes. This can be illustrated by the example that we will use across this paper which emphasizes the dependence of ε and assumes the following representation: where $\tilde{\varepsilon }$ is a small constant. Here we can think of ${\Gamma _{n}}$ as a “common part” of ${\Gamma _{n}^{(1)}}$ and ${\Gamma _{n}^{(2)}}$, and ${R_{n}^{(i)}}$ states for a residual part. This particular example can also be considered as a two-steps model, in which we flip a coin with a probability of “success” $1-\tilde{\varepsilon }$, and if the flip was “successful” we render the next ${W_{n}^{(i)}}$ from a common distribution ${\Gamma _{n}}$, otherwise from a residual distribution ${R_{n}^{(i)}}$. The connection between $\tilde{\varepsilon }$ and ε in (6) is immediate:

Our goal is to find conditions which ensure proximity of n-steps transition probabilities for all n, given only one-step proximity. Additionally, we want to demonstrate that such proximity is approximately of order ε, and all involved constants can be calculated in practical applications.

For this purpose we construct a coupling for chains ${X^{(1)}}$ and ${X^{(2)}}$. First, we assume that conditions of Theorem 3 hold, let $C=[-c,c]$ to be a corresponding set and introduce the following condition.

Second, we introduce substochastic kernels where min should be understood as a minimum of two measures (see [5], Proposition D.2.8 as an example of a formal definition).

Note that by definition of ε and elementary equality $a\wedge b=\frac{a+b-|a-b|}{2}$

We denote the residual substochastic kernels by so that

Here, ${Q_{n}}$ is a general analogue of a “common part” in (7).

To prove the main result, we will need some regularity conditions on ${Q_{n}}$.

The motivation for this condition is as follows. We may think of ${\tilde{Q}^{t,k}}(x,\cdot )=\frac{{Q^{t,k}}(x,\cdot )}{{Q^{t,k}}(x,\mathbb{R})}$ as a proper k-steps transition probability, which is true in the case of schema (7). Generally speaking, this is not true, and we need additional regularity conditions to ensure that function $x\mapsto {Q_{t}}(x,\mathbb{R})$ does not vary too much (in schema (7) ${Q_{t}}(x,\mathbb{R})=1-\tilde{\varepsilon }$ and does not depend on x at all). We do not need ${\tilde{Q}_{t}}={\tilde{Q}^{t,1}}$ to be a proper Markov kernel in our derivations, so we do not impose any additional conditions. But we can think of ${\tilde{Q}_{t}}$ as a Markov kernel and ${\tilde{Q}^{t,k}}$ as k-steps transition probability function to develop the intuition behind Condition T. First, imagine ${\tilde{Q}_{t}}(x,\cdot )$ is a homogeneous Markov kernel that is driving “coupling chain”, so that ${\tilde{Q}_{t}}(x,\cdot )=\tilde{Q}(x,\cdot )$. Assume the kernel is irreducible and geometrically recurrent (and thus positive), so there exists an invariant probability measure, say π. Denote ${\bar{\pi }_{m}}=\pi ({A_{m}})$. We can write Assuming that C is a geometrically recurrent set, we may conclude This demonstrates that condition $\frac{{Q^{t,k}}(x,{A_{m}})}{{Q^{t,k}}(x,\mathbb{R})}\le {\hat{S}_{m}}$, $x\in C$, is reasonable.

Let us now discuss the condition on ${R_{k}^{(i)}}(x,dy)$. This condition requires that the “shifted” residual process does not go far away from its starting position. For example, if ${R_{k}^{(i)}}$ are also centred and have the “autoregressive” form, ${R_{k}^{(i)}}(x,dy)={F_{k}^{(i)}}(dy-x)$, the derivation of Section 3 demonstrates that it is reasonable to expect ${\hat{r}_{n}}=O(n)$. In this case, condition $\Delta <\infty $ is reduced to and which are also reasonable conditions that are satisfied in the case of the geometrically ergodic autoregressive model. Moreover, the homogeneous theory tells us that existing of a small set C (in the sense of Condition M) with a finite second moment of the return time is sufficient for the above conditions to hold.

It is easy to show that a similar motivation remains valid for inhomogeneous autoregressive models. Assume ${\tilde{Q}_{n}}(x,\cdot )={\tilde{\Gamma }_{n}}(\cdot -{\alpha _{n}}x)$, so that we can construct the chain ${({Y_{n}})_{n\ge 0}}$ of the form ${Y_{n+1}}={\alpha _{n+1}}{Y_{n}}+{U_{n+1}}$, $n\ge 0$, where ${U_{n}}\sim {\tilde{\Gamma }_{n}}$ are independent and centred random variables. The transition kernels of the chain $({Y_{n}})$ are ${\tilde{Q}_{n}}$. Then the Chebyshev inequality turns into Next, we write here ${U_{0}}\in [-c,c]$ is a fixed (nonrandom) initial state and ${\textstyle\prod _{n+1}^{n}}=1$. Assume that ${U_{n}},n\ge 1$, have the uniformly bounded second moment. Denote its bound by ${\mu _{2}}$. We can then set So, we can conclude that the existence of the uniformly bounded second moment (or any higher moments) for distributions ${\tilde{\Gamma }_{n}}$ along with the condition are the sufficient conditions for building $\{{\hat{S}_{n}},n\ge 1\}$. Note that Condition 1 of Theorem 1 requires a set $\{n\ge 1:{\alpha _{n}}>1-\delta \}$ (for some $\delta <1$) to be very sparse, so that it is reasonable to expect for some constants M and $\gamma >0$. Of course, in order to ensure $\textstyle\sum \limits_{m}{\hat{r}_{m}}{\hat{S}_{m}}<\infty $ we may need to impose a stronger condition on ${U_{n}}$, for example, to have a uniformly bounded fourth, or even exponential, moment.

Generally speaking, Condition T requires that the “residual mean” grows slower than the ${\tilde{Q}_{n}}$-chain “advances in space”, in other words, the probability of hitting distant regions of the phase space is small enough to compensate for the “residual mean’s” growth as the starting point x is moving away from the origin. It should be clear now that the most typical autoregression, the Gaussian one, satisfies this condition as long as all variances are uniformly bounded.

Finally, we would like to mention that Δ is in fact $\Delta (\varepsilon )$ (we have shown that ${R_{k}^{(i)}}(x,\mathbb{R})\le \varepsilon $), which means it should be small as $\varepsilon \to 0$. In schema (7) we clearly have $\Delta =O(\varepsilon )$.

Assuming Condition M holds true, we define “noncoupling” operators

We define the Markov chain ${\bar{Z}_{n}}=\left({Z_{n}^{(1)}},{Z_{n}^{(2)}},{d_{n}}\right)$ with values in $(\mathbb{R},\mathbb{R},\{0,1,2\})$ by setting its transition probabilities we assume the latter probability is equal to zero if ${Q_{n}}(x,\mathbb{R})=1$, All other probabilities are equal to zero.

It is straightforward that marginal distributions of the process ${\bar{Z}_{n}}$ equal to those of ${X_{n}^{(i)}}$.

We will use the canonical probability ${\bar{\mathbb{P}}_{x,y,d}^{t}}$ and the expectation ${\bar{\mathbb{E}}_{x,y,d}^{t}}$, $x,y\in \mathbb{R}$, $d\in \{0,1,2\}$ in the same sense as before.

Let us denote by the first and m-th return times to $C\times C$ by the pair $\left({Z_{n}^{(1)}},{Z_{n}^{(2)}}\right)$.

We will also need a special notation for the sets and for the values

Everywhere in this section we assume that conditions of Theorem 1 are valid as well as Condition C and will use the corresponding notation.

We start by introducing an important result that allows us to connect the expectation of the exponential moment of ${\sigma _{C\times C}}$ with individual exponential moments of ${\sigma _{C}^{(i)}}$.

Next we introduce and

Going forward we will use the notation from Theorem 1, Corollary 1, Theorem 3 and Condition M. We now present a series of lemmas that are necessary to prove the main result.

Now, we consider a situation when the pair $\left({Z_{k}^{(1)}},{Z_{k}^{(2)}}\right)$ hits $C\times C$ exactly once.

Finally, we look at the situation when the pair $\left({Z_{k}^{(1)}},{Z_{k}^{(2)}}\right)$ hits $C\times C$ exactly $k\ge 2$ times.

It is critical that the probability from Lemma 4 is double summable over n, k. One may notice that since ${\rho _{jk}}=0$ if $j < k$ (because it is impossible that ${\bar{\sigma }_{C\times C}}(k)=j < k$), the double sum in the right hand side of the inequality from Lemma 4 is a convolution of two geometric sequences (${\psi _{1}^{-n}}$) with ${\rho _{jk}}$ (as function of j with fixed k) evaluated at $n-k+1$. Thus in order to establish finitness of the desired sum it is required to demonstrate that the double sequence $\{{\rho _{nk}},n\ge 1,k=\overline{1,n}\}$ is summable. For doing so we will need the next three lemmas.

Finally we can show that $\{{\rho _{nk}}\}$ is summable.