The need to model a Markov renewal on-off process with multiple off-states arise in many applications such as economics, physics, and engineering. Characterization of the occupation time of one specific off-state marginally or two off-states jointly is crucial to understand such processes. The exact marginal and joint distributions of the off-state occupation times are derived. The theoretical results are confirmed numerically in a simulation study. A special case when all holding times have Lévy distribution is considered for the possibility of simplification of the formulas.
Markov renewal on-off processes, also known as alternating renewal processes or telegraph processes, arise in applications in a variety of fields. Classic alternating renewal processes were first studied in [2, 12] and [11] with applications to animal ethology, maintenance of electronic equipment, and communication engineering, respectively. Recently, the telegraph process and their variations are employed in various fields of applications such as pricing in mathematical finance and insurance (see, e.g., [9, 6–8, 16]), modelling the propagation of a damped wave in physics (see, e.g., [4, 10]), and inventory and storage models in engineering [18]. In certain applications, the off-state of an on-off process has multiple types, which raises new questions about its properties such as the distribution of the occupation time in specific off-state type marginally or multiple off-state types jointly. Our interest in on-off processes with multiple off-states is motivated by our recent work in animal movement modeling, where a predator can have different nonmoving states such as resting or handling a kill [15, 14]. The occupation time in the handling state is important for ecologists to understand the behavior of predators.
To fix ideas, consider a server that has two different types of failures, each requiring a different time and cost to repair. One basic question is: if the server is on at time 0, what is the distribution of the time spent on fixing the type one failures by time t>00$]]>? We model the server state process with the following Markov renewal process. The process starts in state 0 (the on-state) and spends there a random holding time according to a given absolutely continuous distribution. When the first holding time is over, we flip an asymmetric coin to decide which type of failure (off-state 1 or off-state 2) comes next. The holding times in the off-states are also absolutely continuous, generally with different distributions. Once the second holding time is over the process returns to the on-state (state 0) and then we repeat the construction process.
The occupation time of a specific type of the off-state is our focus. For the on-state, the distribution of the occupation time is known from the results on the telegraph process [13, 5, 17, 19]. Indeed, if we collapse the two off-states into one, then the resulting process is a regular on-off process (or alternating renewal process) whose off-state holding time distribution is a mixture of the two original off-state holding time distributions. If the different types of repair require different resources, however, we want to know the distribution of the occupation time in a particular type of off-state. For that task, the results on classical telegraph processes are not directly applicable. Moreover, assume that the cost for being in an off-state is proportional to its occupation time and different for different off-states. If one wants to know the distribution of the total cost for repairs of all kinds by a fixed time, then we need the joint distribution of the two types of off-state occupation times.
Our problem is related to but different from some recent works. An extension of the telegraph process to a process with three states is studied in [1], where within a renewal cycle all three states are visited in a given deterministic order. In our case, however, we have an on-off process with two off-states; only two states (the on-state and an off-state) are visited in each renewal cycle, but the off-states is chosen randomly. Another related work is [3], where there are two states, but at each random epoch, the new state is determined by the outcome of a random trial, and, as a result, the process can stay in the same state.
The key idea to study the occupation time of a specific off-state is to exploit a certain periodicity of our Markov renewal process. In fact, when analyzing any Markov renewal process, it is always convenient to do the conditioning on returning to a certain state. For the on-off process with two off-states, state 0 has an additional nice property. The numbers of steps between two consecutive visits of state 0 is not random, and it is always equal to 2. We derive the marginal distribution of the occupation time of a specific off-state first, and then we modify our derivation to obtain the joint distribution of the two off-state occupation times. Of course, one can get the marginal distribution by integrating the joint one. Our approach, however, is easier to follow.
This article is arranged as follows. In Section 2, the formal construction of on-off process with two off-states and the definition of its occupation time are provided. The marginal and joint distribution of off-states occupation time are derived in Section 3 and 4. In Section 5, a special case with holding time modeled by Lévy distribution are discussed alongside with an application of server maintenance cost. Additional technical discussion is provided in Section 6.
Algorithmic construction of on-off process with two off-states
Suppose that we are given the following collection of independent sequences of nonnegative random variables:
{Uk}k≥1 are independent identically distributed (iid) positive random variables with absolutely continuous cumulative distribution function (cdf) FU and probability density function (pdf) fU (these random variables will be used as the holding times when the server is up, state 0);
{Sk}k≥1 are iid positive random variables with absolutely continuous cdf FS and pdf fS (holding times when the server is down for short repairs, state 1);
{Lk}k≥1 are iid positive random variables with absolutely continuous cdf FL and pdf fL (holding times when the server is down for long repairs, state 2);
{ξk}k≥1 are iid random variables with Pr(ξk=1)=p1>00$]]> and Pr(ξk=0)=p2=1−p1.
Now, we present our construction of the on-off process with two off-states, {X(t),t≥0}, with state-space {0,1,2}.
Initialize with X(0)=0 and T0=0.
For cycles i=1,2,…:
Let T2i−1=Ui+T2i−2, and X(t)=0 for all t∈[T2i−2,T2i−1).
If ξi=1 then T2i=T2i−1+Si, and X(t)=1 for all t∈[T2i−1,T2i); otherwise, T2i=T2i−1+Li, and X(t)=2 for all t∈[T2i−1,T2i).
That is, the process starts in state 0 and stays in this state U1 time units. Then it switches to state 1 or 2, which is decided by random variable ξ1. Depending on the value of ξ1, it stays in state 1 or 2 for S1 or L1 time units, respectively. Then the process switches back to state 0. The first renewal cycle is finished, and then the procedure is repeated again. A realization of the on-off process is given in Figure 1.
A realization (first three circles) of on-off process with ξ1=1, ξ2=0, ξ3=0
Note also that Tn→∞ with probability 1, because it is bounded from below by a sum of n iid positive random variables. This means that process X(t) is well-defined for all t>00$]]>.
If all the holding times are exponentially distributed (with rates λi, i=0,1,2), then X(t) is a continuous time Markov chain with the state space {0,1,2} and initial distribution Pr(X(0)=0)=1. Its transition rate matrix is given by
−λ0λ0p1λ0p2λ1−λ10λ20−λ2
where again p1,p2,λ0,λ1,λ2>00$]]> and p1+p2=1.
The occupation times in state 0, state 1, and state 2 are, respectively,
U(t)=∫0t1{X(s)=0}ds,S(t)=∫0t1{X(s)=1}ds,L(t)=∫0t1{X(s)=2}ds.
The corresponding defective marginal densities of S(t) and L(t) are denoted as
pSj(s,t)=Pr(S(t)∈ds,X(t)=j)/ds,pLj(s,t)=Pr(L(t)∈ds,X(t)=j)/ds,
where t≥0, 0<s<t, j=0,1,2. The defective joint two-dimensional density of S(t) and L(t) for u,v>00$]]>, u+v<t is denoted as
pSLj(u,v,t)=1dudvPr(S(t)∈du,L(t)∈dv,X(t)=j),
where j=0,1,2. The densities are defective in the following sense: since both occupation times S(t) and L(t) have an atom at 0, all three sums
∑j=02∫0tpSj(s,t)ds=Pr(S(t)>0)=1−Pr(S(t)=0),∑j=02∫0tpLj(s,t)ds=Pr(L(t)>0)=1−Pr(L(t)=0),0)=1-\Pr (S(t)=0),\\ {} \displaystyle {\sum \limits_{j=0}^{2}}{\int _{0}^{t}}{p_{Lj}}(s,t)\mathrm{d}s=\Pr (L(t)>0)=1-\Pr (L(t)=0),\end{array}\]]]>
and
∑j=02∬u,v>0,u+v<tpSLj(u,v,t)dudv=Pr(S(t)>0,L(t)>0)=1−Pr(S(t)=0orL(t)=0)0,\hspace{0.1667em}u+v0,L(t)>0)\\ {} =& 1-\Pr (S(t)=0\hspace{2.5pt}\text{or}\hspace{2.5pt}L(t)=0)\end{aligned}\]]]>
are less than 1.
Marginal distribution of occupation time of an off-state
To derive the distribution of occupation time we will need some auxiliary random variables. Let N(t) be the number of cycles (or returns to the on-state) by time t. Formally, for n≥0,
N(t)=niffT2n≤t<T2n+2.
Finally, let Dk=ξkSk+(1−ξk)Lk, k≥1. These random variables are associated with the off-state holding time of the regular on-off process when two off-states are combined.
Our formulas include convolutions of different distributions. We will use the following notation. If we are given a cdf G(·) and its pdf g(·), then G(n)(·) denotes n-fold convolution of G(·), and g(n)(·) denotes the n-fold convolution of g(·). If we are given two cdfs G(·) and H(·) with pdfs g(·) and h(·), then G∗H(·) denotes the convolution of cdfs G(·) and H(·), and g∗h(·) denotes the convolution of pdfs g(·) and h(·).
We also use the following conventions. Any summation over the empty set is 0. Zero-fold convolution G(0)(·) is the cdf of a random variable that is equal to 0 with probability 1. Finally, g(k)∗h(0)(·)=g(k)(·) for any k≥1.
Recall that {ξk}k≥1 are independent as defined in Section 2. First, note the distribution of S(t) has an atom. Indeed, if U1>tt$]]> or all the failures that occur before t are of the second type, then S(t)=0. More specifically, by conditioning on the number of returns to the on-state, N(t), we obtain that
Pr(S(t)=0,X(t)=0)=∑n=0∞PrS(t)=0,X(t)=0,N(t)=n=Pr(U1>t)+∑n=1∞Pr∑j=1n(Uj+Lj)≤t,∑j=1n(Uj+Lj)+Un+1>t,∑j=1nξj=0=Pr(U1>t)+∑n=1∞Pr∑j=1n(Uj+Lj)≤t,∑j=1n(Uj+Lj)+Un+1>tp2n=Pr(U1>t)+∑n=1∞Pr∑j=1nUj+∑j=1nLj≤t−Pr∑j=1n+1Uj+∑j=1nLj≤tp2n=(1−FU(t))+∑n=1∞FU(n)∗FL(n)(t)−FU(n+1)∗FL(n)(t)p2n=∑n=0∞FU(n)∗FL(n)(t)−FU(n+1)∗FL(n)(t)p2n.t)+{\sum \limits_{n=1}^{\infty }}\Pr \left({\sum \limits_{j=1}^{n}}({U_{j}}+{L_{j}})\le t,\right.\\ {} & \hspace{2em}\left.{\sum \limits_{j=1}^{n}}({U_{j}}+{L_{j}})+{U_{n+1}}>t,{\sum \limits_{j=1}^{n}}{\xi _{j}}=0\right)\\ {} & =\Pr ({U_{1}}>t)+{\sum \limits_{n=1}^{\infty }}\Pr \left({\sum \limits_{j=1}^{n}}({U_{j}}+{L_{j}})\le t,\right.\\ {} & \hspace{2em}\left.{\sum \limits_{j=1}^{n}}({U_{j}}+{L_{j}})+{U_{n+1}}>t\right){p_{2}^{n}}\\ {} & =\Pr ({U_{1}}>t)+{\sum \limits_{n=1}^{\infty }}\left[\Pr \left({\sum \limits_{j=1}^{n}}{U_{j}}+{\sum \limits_{j=1}^{n}}{L_{j}}\le t\right)\right.\\ {} & \hspace{2em}\left.-\Pr \left({\sum \limits_{j=1}^{n+1}}{U_{j}}+{\sum \limits_{j=1}^{n}}{L_{j}}\le t\right)\right]{p_{2}^{n}}\\ {} & =(1-{F_{U}}(t))\\ {} & +{\sum \limits_{n=1}^{\infty }}\left[{F_{U}^{(n)}}\ast {F_{L}^{(n)}}(t)-{F_{U}^{(n+1)}}\ast {F_{L}^{(n)}}(t)\right]{p_{2}^{n}}\\ {} & ={\sum \limits_{n=0}^{\infty }}\left[{F_{U}^{(n)}}\ast {F_{L}^{(n)}}(t)-{F_{U}^{(n+1)}}\ast {F_{L}^{(n)}}(t)\right]{p_{2}^{n}}.\end{aligned}\]]]>
Next, again by conditioning on N(t), we get that, for 0<s<t,
Pr(S(t)∈ds,X(t)=0)=∑n=1∞PrS(t)∈ds,X(t)=0,N(t)=n.
Note that the summation starts from 1, because X(t)=0 and N(t)=0 implies that U1>tt$]]>, and, therefore, S(t)=0.
The next step is to fix the number of switches to failures of type 1. Since the total numbers of switches is n and there is at least one failure of type 1, we have
Pr(S(t)∈ds,X(t)=0,N(t)=n)=∑k=1nPrS(t)∈ds,X(t)=0,N(t)=n,∑j=1nξj=k.
Because
X(t)=0,N(t)=niff∑j=1n(Uj+Dj)≤t,∑j=1n(Uj+Dj)+Un+1>t,t,\end{aligned}\]]]>
and it really does not matter during which cycles the switches to failures of type 1 occur, we find that
Pr(S(t)∈ds,X(t)=0,N(t)=n,∑j=1nξj=k)=PrS(t)∈ds,∑j=1n(Uj+Dj)≤t,∑j=1n(Uj+Dj)+Un+1>t,∑j=1nξj=k=nkPrS(t)∈ds,∑j=1n(Uj+Dj)≤t,∑j=1n(Uj+Dj)+Un+1>t,∑j=1kξj=k,∑j=k+1nξj=0.t,{\sum \limits_{j=1}^{n}}{\xi _{j}}=k\right)\\ {} & =\left(\genfrac{}{}{0.0pt}{}{n}{k}\right)\Pr \left(S(t)\in \mathrm{d}s,{\sum \limits_{j=1}^{n}}({U_{j}}+{D_{j}})\le t,\right.\\ {} & \left.\hspace{2em}{\sum \limits_{j=1}^{n}}({U_{j}}+{D_{j}})+{U_{n+1}}>t,{\sum \limits_{j=1}^{k}}{\xi _{j}}=k,{\sum \limits_{j=k+1}^{n}}{\xi _{j}}=0\right).\end{aligned}\]]]>
Next, observe that in this case S(t)=∑j=1kSj. Using independence between {ξk}k≥1 and the holding time sequences we have
Pr(S(t)∈ds,X(t)=0,N(t)=0,∑j=1nξj=k)=nkp1kp2n−k×Pr∑j=1kSj∈ds,∑j=1nUj+∑j=1kSj+∑j=k+1nLj≤t,∑j=1n+1Uj+∑j=1kSj+∑j=k+1nLj>t=nkp1kp2n−k×Pr∑j=1kSj∈ds,∑j=1nUj+∑j=k+1nLj≤t−s,∑j=1n+1Uj+∑j=k+1nLj>t−s.t\right)\\ {} & =\left(\genfrac{}{}{0.0pt}{}{n}{k}\right){p_{1}^{k}}{p_{2}^{n-k}}\\ {} & \times \Pr \left({\sum \limits_{j=1}^{k}}{S_{j}}\in \mathrm{d}s,{\sum \limits_{j=1}^{n}}{U_{j}}+{\sum \limits_{j=k+1}^{n}}{L_{j}}\le t-s,\right.\\ {} & \left.\hspace{2em}\hspace{2em}\hspace{1em}{\sum \limits_{j=1}^{n+1}}{U_{j}}+{\sum \limits_{j=k+1}^{n}}{L_{j}}>t-s\right).\end{aligned}\]]]>
Finally, independence of holding time sequences gives us that
Pr∑j=1kSj∈ds,∑j=1nUj+∑j=k+1nLj≤t−s,∑j=1n+1Uj+∑j=k+1nLj>t−s=Pr∑j=1kSj∈dsPr∑j=1nUj+∑j=k+1nLj≤t−s−Pr∑j=1n+1Uj+∑j=k+1nLj≤t−s=fS(k)(s)×FU(n)∗FL(n−k)(t−s)−FU(n+1)∗FL(n−k)(t−s)ds.t-s\right)\\ {} & =\Pr \left({\sum \limits_{j=1}^{k}}{S_{j}}\in \mathrm{d}s\right)\left[\Pr \left({\sum \limits_{j=1}^{n}}{U_{j}}+{\sum \limits_{j=k+1}^{n}}{L_{j}}\le t-s\right)\right.\\ {} & \hspace{2em}\hspace{2em}\hspace{2em}\left.-\Pr \left({\sum \limits_{j=1}^{n+1}}{U_{j}}+{\sum \limits_{j=k+1}^{n}}{L_{j}}\le t-s\right)\right]\\ {} & ={f_{S}^{(k)}}(s)\\ {} & \times \left[{F_{U}^{(n)}}\ast {F_{L}^{(n-k)}}(t-s)-{F_{U}^{(n+1)}}\ast {F_{L}^{(n-k)}}(t-s)\right]\mathrm{d}s.\end{aligned}\]]]>
Now let us consider the case when X(t)=1 (at time t the server is down for a short repair). One difference is that there is no atom in this case. As before, for 0<s<t we have
=Pr(S(t)∈ds,X(t)=1)=∑n=0∞PrS(t)∈ds,X(t)=1,N(t)=n=∑n=0∞∑k=0nPrS(t)∈ds,X(t)=1,N(t)=n,∑j=1n∑j=1nξj=k,ξn+1=1=∑n=0∞∑k=0nnkPrS(t)∈ds,X(t)=1,N(t)=n,∑j=1k∑j=1kξj=k,∑j=k+1nξj=0,ξn+1=1.
Note that event
{S(t)∈ds,X(t)=1,N(t)=n,∑j=1kξj=k,∑j=k+1nξj=0,ξn+1=1}
can be rewritten as
{S(t)∈ds,∑j=1n+1Uj+∑j=1kSj+∑j=k+1nLj≤t,∑j=1n+1Uj+∑j=1kSj+∑j=k+1nLj+Sn+1>t,∑j=1kξj=k,∑j=k+1nξj=0,ξn+1=1}.t,\\ {} & \hspace{0.2778em}{\sum \limits_{j=1}^{k}}{\xi _{j}}=k,\hspace{0.2778em}{\sum \limits_{j=k+1}^{n}}{\xi _{j}}=0,\hspace{0.2778em}{\xi _{n+1}}=1\Big\}.\end{aligned}\]]]>
Since S(t)=t−∑j=1n+1Uj−∑j=k+1nLj, we finally obtain
Pr(S(t)∈ds,X(t)=1)=∑n=0∞∑k=0nnkp1k+1p2n−kPr∑j=1n+1Uj+∑j=k+1nLj∈t−ds,∑j=1kSj≤s,∑j=1kSj+Sn+1>s=∑n=0∞∑k=0nnkp1k+1p2n−kfU(n+1)∗fL(n−k)(t−s)×FS(k)(s)−FS(k+1)(s)ds.s\right)\\ {} & ={\sum \limits_{n=0}^{\infty }}{\sum \limits_{k=0}^{n}}\left(\genfrac{}{}{0.0pt}{}{n}{k}\right){p_{1}^{k+1}}{p_{2}^{n-k}}{f_{U}^{(n+1)}}\ast {f_{L}^{(n-k)}}(t-s)\\ {} & \times \left[{F_{S}^{(k)}}(s)-{F_{S}^{(k+1)}}(s)\right]\mathrm{d}s.\end{aligned}\]]]>
The other formulas can be derived in a similar way. □
Theorem 1 gives us the distribution of occupation time in state 1. Since both off-states enter our story in a completely symmetric way, the formulas for the occupation time in state 2 can be obtained by interchanging state 1 and 2 in Theorem 1.
Joint distribution of off-state occupation times
As mentioned in the introduction, if we are interested in the distribution of the total cost, then we need the joint distribution of off-state occupation times. More specifically, assume that the total cost C(t) is a linear function of occupation times, that is,
C(t)=α0U(t)+α1S(t)+α2L(t),
where α0, α1, and α2 are positive constants representing the maintenance cost per unit of time for state 0, 1, and 2, respctively. Then the distribution of C(t) is fully determined by the joint distribution of S(t) and L(t). Note that we do not need the joint distribution of all three occupation times, because U(t)=t−S(t)−L(t).
Let us also note here that the cost function process {C(t),t≥0} is similar to what is know in the literature as integrated telegraph process. Indeed, C(t) is a linearly increasing process. It goes up with velocity α0, if X(t) is in state 0. When the process X(t) switches to one of two off-states, it goes up either with velocity α0 or α2 (depending on an off-state).
Since both S(t) and L(t) have atoms at 0, for every value of X(t) we have four cases: (1) both occupation times are 0; (2) S(t)=0, L(t)>00$]]>; (3) S(t)>00$]]>, L(t)=0; and (4) both S(t) and L(t) are strictly greater than 0. In total, we have 12 formulas. Some of them are trivial. For instance, event {S(t)=0,X(t)=1} has probability 0, therefore, the corresponding defective one-dimensional density of L(t) is also 0. Moreover, since there is a certain symmetry between S(t) and L(t), some formulas can be found by interchanging state 1 and 2. That is why in the following theorem we have only 5 formulas.
We will only derive the formula for pSL1(u,v,t). The remaining formulas can be obtained by similar modifications of the proof of Theorem 1.
As before, we start with partitioning with respect to events {N(t)=n}. More specifically, for u,v>00$]]> and 0<u+v<t we have
pSL1(u,v,t)dudv=∑n=0∞PrS(t)∈du,L(t)∈dv,X(t)=1,N(t)=n=∑n=0∞∑k=0n−1PrS(t)∈du,L(t)∈dv,X(t)=1,∑j=1nξj=kN(t)=n,∑j=1nξj=k,ξn+1=1=∑n=0∞∑k=0n−1nkPrS(t)∈du,L(t)∈dv,X(t)=1,∑j=1nξj=kN(t)=n,∑j=1kξj=k,∑j=k+1nξj=0,ξn+1=1.
Note that now the upper limit of the inner summation is n−1, because ∑j=1nξj=n and ξn+1=1 implies that L(t)=0.
Next, observe that event
{S(t)∈du,L(t)∈dv,X(t)=1,N(t)=n,∑j=1kξj=k,∑j=k+1nξj=0,ξn+1=1}
can be rewritten as
{S(t)∈du,L(t)∈dv,∑j=1n+1Uj+∑j=1kSj+∑j=k+1nLj≤t,∑j=1n+1Uj+∑j=1kSj+∑j=k+1nLj+Sn+1>t,∑j=1kξj=k,∑j=k+1nξj=0,ξn+1=1}.t,\\ {} & \hspace{2em}\hspace{2em}{\sum \limits_{j=1}^{k}}{\xi _{j}}=k,{\sum \limits_{j=k+1}^{n}}{\xi _{j}}=0,{\xi _{n+1}}=1\Big\}.\end{aligned}\]]]>
Finally, taking into account that in the case when {X(t)=1} occupation time S(t)=t−∑j=1n+1Uj−∑j=k+1nLj and occupation time L(t)=∑j=k+1nLj, we get that
pSL1(u,v,t)dudv=∑n=0∞∑k=0n−1nkp1k+1p2n−k×Pr∑j=1n+1Uj∈t−du−dv,∑j=k+1nLj∈dv,∑j=1kSj≤u,∑j=1kSj+Sn+1>u=∑n=0∞∑k=0n−1nkp1k+1p2n−kfU(n+1)(t−u−v)fL(n−k)(v)×FS(k)(u)−FS(k+1)(u)dudv.u\right)\\ {} & ={\sum \limits_{n=0}^{\infty }}{\sum \limits_{k=0}^{n-1}}\left(\genfrac{}{}{0.0pt}{}{n}{k}\right){p_{1}^{k+1}}{p_{2}^{n-k}}{f_{U}^{(n+1)}}(t-u-v){f_{L}^{(n-k)}}(v)\\ {} & \times \left[{F_{S}^{(k)}}(u)-{F_{S}^{(k+1)}}(u)\right]\mathrm{d}u\mathrm{d}v.\end{aligned}\]]]>
□
One can verify now that, for instance,
pS1(u,t)=∑n=0∞p1n+1fU(n+1)(t−u)FS(n)(u)−FS(n+1)(u)+∫0t−upSL1(u,v,t)dv.
Using symmetry between S(t) and L(t), we also can get that
pSL2(u,v,t)=∑n=0∞∑k=0n−1nkp2k+1p1n−k×fU(n+1)(t−u−v)fS(n−k)(u)FL(k)(v)−FL(k+1)(v).
Theorem 1 and Theorem 2 can be generalized to the multiple off-states case. We give an example for three off-states here. Define iid random variables {Mk}k≥1 with cumulative distribution FM and probability density function fM, as the holding times for the 3rd off-state (medium repair). Then, {ξk}k≥1 will change to iid random variables with Pr(ξk=S)=p1, Pr(ξk=L)=p2, Pr(ξk=M)=p3, and p1+p2+p3=1. Accordingly, the occupation time in the 3rd off-state is M(t)=∫0t1{X(s)=3}ds. By the same technique, we can get the marginal and joint distribution of the occupation times. The only difference is that binomial probabilities are replaced with multinomial ones. For example,
pS0(s,t)=Pr(S(t)∈ds,X(t)=0)/ds=∑n=1∞∑k=1n∑k∗=0n−knkn−kk∗p1kp2k∗p3n−k−k∗fS(k)(s)×FU(n)∗FL(k∗)∗FM(n−k−k∗)(t−s)−FU(n+1)∗FL(k∗)∗FM(n−k−k∗)(t−s),
and
pSLM1(u,v,w;t)=Pr(S(t)∈du,L(t)∈dv,M(t)∈dw,X(t)=1)/dudvdw=∑n=0∞∑k=0n−1∑k∗=1n−k−1nkn−kk∗p1kp2k∗p3n−k−k∗×fU(n+1)(t−v−w−u)fL(k∗)(v)fM(n−k−k∗)(w)×FS(k)(u)−FS(k+1)(u).
All other formulas can be derived similarly.
Special case: Lévy distribution
In this section we consider a special case when all the holding times have the Lévy distribution (with location parameter 0 and possibly different scale parameters). The Lévy distribution with scale parameter c2 has pdf
gc(x)=c2πe−c22xx3/2,x>0,0,\]]]>
and cdf
Gc(x)=2π∫c2x∞e−t2dt,x>0.0.\]]]>
The Lévy distribution is heavy-tailed with infinite expectation. The median is given by 0.5c2erfc−1(0.5)−2≈2.198112c2, where erfc is the complementary error function:
erfc(x)=2π∫x∞e−t2dt.
The parametrization that we use is a bit unusual, but it allows us to shorten our notation for convolutions. More specifically, as a member of the family of stable distributions with mobility parameter 1/2, the Lévy distribution is closed under the operation of convolution in the following way:
Gc1∗Gc2(x)=Gc1+c2(x).
Let cU2, cS2, and cL2 be the scale parameters for states 0, 1 and 2, respectively. Then the formulas from Theorem 1 and Theorem 2do not involve any convolutions. For instance, in this case we have that
pS2(s,t)=∑n=1∞∑k=1nnkp1kp2n−k+1gkcS(s)×G(n+1)cU+(n−k)cLt−s−G(n+1)cU+(n−k+1)cLt−s.
The expectations and variances of the off-times is given by
E(S(t))=∑j=02∫0tspSj(s,t)ds,Var(S(t))=∑j=02∫0ts2pSj(s,t)ds−[E(S(t))]2,E(L(t))=∑j=02∫0tspLj(s,t)ds,Var(L(t))=∑j=02∫0ts2pLj(s,t)ds−[E(L(t))]2.
Note that the discrete component of the occupation times is not used for these calculations, because the atoms are at 0. The covariance of S(t) and L(t) is given by
Cov(S(t),L(t))=∑j=02∬u,v>0,u+v<tuvpSLj(u,v,t)dudv−E(S(t))E(L(t)).0,\hspace{0.1667em}u+v
As an example we consider a server with median holding time 7 days, 0.5 hour, and 4 hours for the on-state, the short off-state, and the long off-state, respectively. Using one day as the time unit, the model parameters are: cU≈1.785, cS≈0.097, and cL≈0.275. Assume also that the total repair cost is given by
C(t)=S(t)+2L(t),
that is, long repairs are twice costlier than the short ones.
Defective densities pSj(s,t) and pLj(s,t), j∈{0,1,2}, with t=30, cU≈1.785, cS≈0.097, cL≈0.275, and p1=0.9
Contour plots of the joint densities pSLj(u,v,t), j∈{0,1,2}, with t=30, cU≈1.785, cS≈0.097, cL≈0.275, and p1=0.9
Figure 2 presents defective marginal densities pSj(s,t) and pLj(s,t) when t=30 days and p1=.9 (that is, the less serious breakdowns occur 9 times more often). The marginal densities for both occupation times are severely defected when X(30)=0 (the process is in the on-state). As we mentioned above S(30) and L(30) have atoms at s=0:
Pr(S(30)=0,X(30)=0)≈0.280;Pr(L(30)=0,X(30)=0)≈0.798;Pr(S(30)=0,X(30)=1)≈0.000;Pr(L(30)=0,X(30)=1)≈0.034;Pr(S(30)=0,X(30)=2)≈0.004;Pr(L(30)=0,X(30)=2)≈0.000.
Figure 3 shows the contour plot of defective joint densities pSLj(u,v,t) with the same parameter setup, where the negative association of the two occupation times is obvious. We also ran a simulation study to numerically confirm the correctness of Theorem 1 and Theorem 2. A total 1,000,000 realizations of the above on-off process were generated and the empirical results are consistent with the theoretical densities (not shown).
Table 1 summarizes the expectation and variance of the two types of off-state occupation times and the associated repair cost for t∈{30,60} days and p1∈{0.70,0.75,…,0.95,0.99}. As p1 increases, the expectation and the variance goes up for the less serious breakdown time but goes down for the more serious breakdown time. The total repair cost has lower expectation and variance for smaller values of p1. The correlation of the two off-state occupation times is negative, with a magnitude decreasing as p1 increases. When t is doubled, the expectation of the occupation times and the total repair cost is slightly more than doubled, which is because the process always starts from the on-state with a random holding time.
Summaries of off-state occupation times and total repair cost with cU≈1.785, cS≈0.097, and cL≈0.275 as p1 increases
t
p1
E(S(t))
Var(S(t))
E(L(t))
Var(L(t))
E(C(t))
Var(C(t))
ρ(S(t),L(t))
30
0.70
0.81
10.82
0.95
13.01
2.71
60.99
−0.040
0.75
0.87
11.60
0.79
10.95
2.46
53.71
−0.037
0.80
0.93
12.38
0.64
8.85
2.20
46.32
−0.035
0.85
0.99
13.15
0.48
6.70
1.95
38.80
−0.031
0.90
1.05
13.93
0.32
4.51
1.69
31.16
−0.026
0.95
1.11
14.70
0.16
2.28
1.43
23.38
−0.019
0.99
1.16
15.32
0.03
0.46
1.23
17.07
−0.009
60
0.70
1.75
48.20
2.08
57.96
5.92
271.58
−0.040
0.75
1.88
51.68
1.74
48.81
5.37
239.32
−0.038
0.80
2.02
55.16
1.40
39.46
4.82
206.48
−0.035
0.85
2.15
58.64
1.05
29.91
4.26
173.04
−0.031
0.90
2.29
62.13
0.70
20.16
3.70
139.01
−0.026
0.95
2.42
65.62
0.35
10.19
3.13
104.37
−0.019
0.99
2.53
68.41
0.07
2.05
2.67
76.21
−0.009
Concluding remarks
The marginal distribution of an off-state occupation time could be obtained by collapsing the on-state with the other off-state and using results on the telegraph process. Nonetheless, this approach would lead to more complicated formulas because the holding time of the new collapsed state is distributed as an infinite mixture of convolutions. Further, this technique cannot be employed if we need the joint distribution of the two off-state occupation times.
The total repair cost process C(t) also can be viewed as a telegraph process governed by the on-off process with multiple off-states. That is, C(t) is the time t position of a particle that moves with speed αj whenever X(t)=j, j=0,1,2. The formulas for the joint distribution of X(t) and occupation times can be useful for parameter estimation when process C(t) is discretely observed. In particular, if X(t) is also Markov (that is, all the holding times are exponential), then the efficient likelihood estimation is possible with help of tools for hidden Markov model [14].
ReferencesBshouty, D., Di Crescenzo, A., Martinucci, B., Zacks, S.: Generalized telegraph process with random delays. J. Appl. Probab.49, 850–865 (2012). MR3012104. https://doi.org/10.1017/s002190020000958xCane, V.R.: Behavior sequences as semi-Markov chains. J. R. Stat. Soc., Ser. B21, 36–58 (1959). MR0109095Crimaldi, I., Di Crescenzo, A., Iuliano, A., Martinucci, B.: A generalized telegraph process with velocity driven by random trials. Adv. Appl. Probab.45(4), 1111–1136 (2013). MR3161299. https://doi.org/10.1239/aap/1386857860De Gregorio, A., Macci, C.: Large deviation principles for telegraph processes. Stat. Probab. Lett.82(11), 1874–1882 (2012). MR2970286Di Crescenzo, A.: On random motions with velocities alternating at Erlang-distributed random times. Adv. Appl. Probab.33, 690–701 (2001). MR1860096. https://doi.org/10.1239/aap/1005091360Di Crescenzo, A., Iuliano, A., Martinucci, B., Zacks, S.: Generalized telegraph process with random jumps. J. Appl. Probab.50(2), 450–463 (2013). MR3102492Di Crescenzo, A., Martinucci, B., Zacks, S.: On the geometric brownian motion with alternating trend. Mathematical and statistical methods for actuarial sciences and finance, Eds. Perna, C. and Sibillo, M., 81–85 (2014).Di Crescenzo, A., Zacks, S.: Probability law and flow function of Brownian motion driven by a generalized telegraph process. Methodol. Comput. Appl. Probab.17(3), 761–780 (2015). MR3377859. https://doi.org/10.1007/s11009-013-9392-1Kolesnik, A.D., Ratanov, N.: Telegraph Processes and Option Pricing. Springer Briefs in Statistics. Springer (2013). MR3115087. https://doi.org/10.1007/978-3-642-40526-6Macci, C.: Large deviations for some non-standard telegraph processes. Stat. Probab. Lett.110, 119–127 (2016). MR3474745Newman, D.S.: On the probability distribution of a filtered random telegraph signal. Ann. Math. Stat.39(3), 890–896 (1968). MR0230564. https://doi.org/10.1214/aoms/1177698321Page, E.S.: Theoretical considerations of routine maintenance. Comput. J.2(4), 199–204 (1960).Perry, D., Stadje, W., Zacks, S.: First-exit times for increasing compound processes. Commun. Stat., Stoch. Models15(5), 977–992 (1999). MR1721237. https://doi.org/10.1080/15326349908807571Pozdnyakov, V., Elbroch, L.M., Hu, C., Meyer, T., Yan, J.: On estimation for brownian motion governed by telegraph process with multiple off states. Methodol. Comput. Appl. Probab.22, 1275–1291 (2020). MR4129134. https://doi.org/10.1007/s11009-020-09774-1Pozdnyakov, V., Elbroch, L.M., Labarga, A., Meyer, T., Yan, J.: Discretely observed Brownian motion governed by telegraph process: estimation. Methodol. Comput. Appl. Probab.21(3), 907–920 (2019). MR4001858. https://doi.org/10.1007/s11009-017-9547-6Ratanov, N.: Piecewise linear process with renewal starting points. Stat. Probab. Lett.131, 78–86 (2017). MR3706699. https://doi.org/10.1016/j.spl.2017.08.010Stadje, W., Zacks, S.: Telegraph processes with random velocities. J. Appl. Probab.41, 665–678 (2004). MR2074815. https://doi.org/10.1239/jap/1091543417Xu, Y., De, S.K., Zacks, S.: Exact distribution of intermittently changing positive and negative compound Poisson process driven by an alternating renewal process and related functions. Probab. Eng. Inf. Sci.29(3), 385–397 (2015). MR3355610Zacks, S.: Generalized integrated telegraph processes and the distribution of related stopping times. J. Appl. Probab.41(2), 497–507 (2004). MR2052587. https://doi.org/10.1239/jap/1082999081