Modern Stochastics: Theory and Applications

For usual optimal control problems, by the dynamic principle of optimality [40] one may check that an optimal control remains optimal when it is restricted to a later time interval, meaning that optimal controls are time-consistent. The time-consistency feature provides a powerful advance to deal with optimal control problems. The dynamic principle of optimality consists in establishing relationships among a family of time-consistent optimal control problems parameterized by initial pairs (of time and state) through the so-called Hamilton–Jacobi–Bellman equation (HJB), which is a nonlinear partial differential equation. If the HJB equation is solvable, then one can find an optimal feedback control by taking the optimizer of the general Hamiltonian involved in the HJB equation.

However, in reality, the time-consistency can be lost in various ways, meaning that, as time goes, an optimal control might not remain optimal. Among several possible reasons causing the time-inconsistency, there are three ones playing some important roles:

The portfolio optimization problem with a hyperbolic discount function [11] and the risk aversion attitude in mean-variance models [17, 43] and [44] are two well-known cases of time-inconsistency in mathematical finance. Motivated by the second example, the present paper studies a general linear-quadratic optimal control problem for jump diffusions, which is time-inconsistent in the sense that it does not satisfy the Bellman optimality principle due to the existence of some quadratic terms in the expected controlled state process as well as a state-dependent risk aversion term in the running and the terminal cost functionals. The fundamental challenge when dealing with a time-inconsistent optimal control models is that we can’t employ the dynamic programming approach and the standard HJB equation, in general. One way to get around the time-inconsistency issue is to consider only precommitted strategies, see, e.g., [45] and [26].

However, the main method of dealing with time-inconsistency is to consider the time-inconsistent problems as noncooperative games, in which decisions at every moment of time are taken by multiple players at each moment of time and are intended to maximize or minimize their own objective functions. As a result, Nash equilibriums are considered rather than optimal solutions, see, e.g., [3, 8, 11, 15, 16, 23, 24, 28, 37, 38] and [39]. Strotz [28] was the first who applied this game perspective for dealing with the dynamic time-inconsistent decision problem posed by the deterministic Ramsay problem. He then proposed a rudimentary notion of Nash equilibrium strategy by capturing the concept of noncommitment and allowing the commitment period to be infinitesimally small. Further references which extend [28] are [16, 24] and [13]. Ekeland and Pirvu [11] gave a formal definition of feedback Nash equilibrium controls in a continuous-time setting in order to investigate the optimal investment–consumption problem under general discount functions in both deterministic and stochastic frameworks. Björk & Murguci [3] and Ekeland et al. [10] are two further expansions of Ekeland and Pirvu’s work. Yong [39] proposed an alternative method for analyzing general discounting time-inconsistent optimal control problem in continuous-time setting by taking into account a discrete time counterpart. Zhao et al. [42] investigated the consumption–investment problem under a general discount function and a logarithmic utility function using Yong’s method. Wang and Wu investigated a partially observed time-inconsistent recursive optimization issue in [33]. Basak and Chabakauri [1] touched upon the continuous-time Markowitz’s mean-variance portfolio selection problem, while Björk et al. [4] addressed the mean-variance portfolio selection with state-dependent risk aversion. Hu et al. [15], followed by Czichowski [8], found a time-consistent strategy for mean-variance portfolio selection in a non-Markovian framework.

The linear-quadratic optimal control problems are well known as a fundamental category of optimal control problems, since they may cover a wide range of problems in applications, such as the mean-variance portfolio selection model in financial applications. Furthermore, the LQ model may be used to approximate many nonlinear control problems. In recent years, time-inconsistent LQ control problems have gotten a lot of attention. Yong worked on a general discounted time-inconsistent deterministic LQ model in [37] and he consider a forward ordinary differential equation coupled with a backward Riccati–Volterra integral equation to obtain closed-loop equilibrium strategies. Hu et al. [15] presented a specific definition of open-loop Nash equilibrium controls in a continuous-time setting, which is distinct from that for the feedback controls provided in [11], in order to analyze a time-inconsistent stochastic linear-quadratic optimal control problem with stochastic coefficients. Yong [39] studied a time-inconsistent stochastic LQ problem for mean-field type stochastic differential equation. Finally, Hu et al. [14] looked into the uniqueness of the equilibrium solution found in [15]. They are the first who give a positive result regarding the uniqueness of the solution to a time-inconsistent problem.

There is little work in the literature concerning equilibrium strategies for optimal investment and reinsurance problems under the mean-variance criterion. Zeng and Li [43] are the first who study Nash equilibrium strategies for mean-variance insurers with constant risk aversion, where the surplus process of insurers is described by the diffusion model and the price processes of the risky stocks are driven by geometric Brownian motions. They have obtained equilibrium reinsurance and investment strategies explicitly using the technique described in [3]. Li and Li [17] obtained equilibrium strategies in the case of state-dependent risk aversion through a set of well-posed integral equations. Zeng et al. [44] investigate time-consistent investment and reinsurance strategies for mean-variance insurers under constant risk aversion, in which the surplus process and the price process of the risky stock are both jump-diffusion processes.

Markov regime-switching models have recently gotten a lot of interest in financial applications; see, for example, [46, 5, 6, 34] and [18]. Markov regime-switching models permit the market to face financial crises at any moment. The market is supposed to be governed by some kind of regime at any given moment. A bull market, in which stock prices are generally increasing, is a standard illustration of such a regime. The market’s behavior radically alters after a financial crisis. A switch in the regime symbolizes the crisis. The problem of mean-variance optimization under a continuous-time Markov regime-switching financial market was first studied by Zhou and Yin [46]. By applying stochastic linear-quadratic control methods, they obtained mean-variance efficient portfolios and efficient frontiers via solving two systems of ordinary linear differential equations. In the context of continuous and multiperiod time models, Chen et al. [5] and Chen and Yang [6] studied the mean-variance asset-liability management problem, respectively. Mean-variance asset-liability management problems with a continuous-time Markov regime-switching setup have been studied by Wei et al. [34]. They explicitly deduced a time-consistent investment strategy using the method described in [3]. Liang and Song [18] investigated optimal investment and reinsurance problems for insurers with mean-variance utility under partial information, where the stock’s drift rate and the risk aversion of the insurer are both Markov-modulated.

In this work, we present a general time-inconsistent stochastic conditional LQ control problem. Differently from most current studies [15, 39, 2, 42], where the noise is driven by a Brownian motion, in our LQ system the state develops according to a SDE, in which the noise is driven by a multidimensional Brownian motion and an independent multidimensional Poisson point process under a Markov regime-switching setup. Cases of continuous-time mean-variance criteria with state-dependent risk aversion are included in the objective function. We establish a stochastic system that describes open-loop Nash equilibrium controls, using the variational technique proposed by Hu et al. [14]. We emphasize that our model generalizes the ones investigated by Zeng and Li [43], Li et al. [17], Sun and Guo [30] and Zeng et al. [44], in addition to some classes of time-inconsistent stochastic LQ optimal control problems introduced in [15].

The paper is organized as follows: in the second section, we formulate the problem and provide essential notations and preliminaries. Section 3 is dedicated to presenting the necessary and sufficient conditions for equilibrium, which is our main result, and we get the unique equilibrium control in state feedback representation through a specific category of ordinary differential equations. In the last section, we apply the results of Section 3 to find the unique equilibrium reinsurance, investment and consumption strategies for the mean-variance-utility portfolio problem, as well as discuss some special cases. The paper concludes with an Appendix that includes some proofs.

Let $(\Omega ,\mathcal{F},\mathbb{F},\mathbb{P})$ be a filtered probability space where $\mathbb{F}:=\left\{\left.{\mathcal{F}_{t}}\right|t\in [0,T]\right\}$ is a right-continuous, $\mathbb{P}$-completed filtration to which all of the processes outlined below are adapted, such as the Markov chain, the Brownian motions, and the Poisson random measures.

During the present paper, we assume that the Markov chain $\alpha \left(\cdot \right)$ takes values in finite state space $\chi =\left\{{e_{1}},{e_{2}},\dots ,{e_{d}}\right\}$ where $d\in \mathbb{N}$, ${e_{i}}\in {\mathbb{R}^{d}}$ and the j-th component of ${e_{i}}$ is the Kronecker delta ${\delta _{ij}}$ for each $\left(i,j\right)\in {\left\{1,\dots ,d\right\}^{2}}$. $\mathcal{H}:={\left({\lambda _{ij}}\right)_{1\le i,j\le d}}$ represents the rate matrix of the Markov chain under $\mathbb{P}$. Note that ${\lambda _{ij}}$ is the constant transition intensity of the chain from state ${e_{i}}$ to state ${e_{j}}$ at time t, for each $\left(i,j\right)\in {\left\{1,\dots ,d\right\}^{2}}$. As a result, for $i\ne j$, we have ${\lambda _{ij}}\ge 0$ and ${\textstyle\sum \limits_{j=1}^{d}}{\lambda _{ij}}=0$, thus ${\lambda _{ii}}\le 0$. In the sequel, for each $i,j=1,2,\dots ,d$ with $i\ne j$, we assume that ${\lambda _{ij}}>0$ consequently, ${\lambda _{ii}}<0$. We have the following semimartingale representation of the Markov chain $\alpha \left(\cdot \right)$ obtained from Elliott et al. [12] where $\{\mathcal{M}(t)|t\in [0,T]\}$ is an ${\mathbb{R}^{d}}$-valued $(\mathbb{F},\mathbb{P})$-martingale.

First, we provide a set of Markov jump martingales linked with the chain $\alpha \left(\cdot \right)$, which will be used to model the controlled state process. For each $\left(i,j\right)\in {\left\{1,\dots ,d\right\}^{2}}$, with $i\ne j$, and $t\in \left[0,T\right]$, denote by ${J^{ij}}\left(t\right):={\lambda _{ij}}{\textstyle\int _{0}^{t}}\left\langle \alpha \left(\tau -\right),{e_{i}}\right\rangle d\tau +{m_{ij}}(t)$ the number of jumps from state ${e_{i}}$ to state ${e_{j}}$ up to time t, where ${m_{ij}}(t):={\textstyle\int _{0}^{t}}\left\langle \alpha \left(\tau -\right),{e_{i}}\right\rangle \left\langle d\mathcal{M}\left(\tau \right),{e_{j}}\right\rangle d\tau $ is an $(\mathbb{F},\mathbb{P})$-martingale. ${\Phi _{j}}(t)$ denotes the number of jumps into state ${e_{j}}$ up to time t, for each fixed $j=1,2,\dots ,d$, then where ${\tilde{\Phi }_{j}}(t):={\textstyle\sum \limits_{i=1,i\ne j}^{d}}{m_{ij}}(t)$ is an $\left(\mathbb{F},\mathbb{P}\right)$-martingale for each $j=1,2,\dots ,d$. For each $j=1,2,\dots ,d$ set

Note that the process ${\tilde{\Phi }_{j}}(t)={\Phi _{j}}(t)-{\lambda _{j}}(t)$ is an $\left(\mathbb{F},\mathbb{P}\right)$-martingale, for each $j=1,2,\dots ,d$.

Now, we present the Markov regime-switching Poisson random measures. Assume that ${N_{i}}(dt,dz)$, $i=1,2,\dots ,l$, are independent Poisson random measures on $\left(\left[0,T\right]\times {\mathbb{R}_{0}},\mathcal{B}\left(\left[0,T\right]\right)\otimes {\mathcal{B}_{0}}\right)$ under $\mathbb{P}$. Assume that the compensator for the Poisson random measure ${N_{i}}(dt,dz)$ is defined by where ${\theta ^{i}}(dz):={\left({\theta _{{e_{1}}}^{i}}(dz),{\theta _{{e_{2}}}^{i}}(dz),\dots ,{\theta _{{e_{d}}}^{i}}(dz)\right)^{\top }}\in {\mathbb{R}^{d}}$. The subscript α in ${n_{\alpha }^{i}}$, for $i=1,2,\dots ,l$, represents the dependence of the probability law of the Poisson random measure on the Markov chain $\alpha \left(\cdot \right)$. In fact ${\theta _{{e_{j}}}^{i}}(dz)$ is the conditional Lévy density of jump sizes of the random measure ${N_{i}}(dt,dz)$ at time t when $\alpha \left(t-\right)={e_{j}}$, for each $j=1,2,\dots ,d$. Furthermore, the compensated Poisson random measure ${\tilde{N}_{\alpha }}(dt,dz)$ is given by

In view of the fact that Problem (N) is time-inconsistent, the aim of this paper is to characterize open-loop Nash equilibriums as an alternative of optimal strategies. We employ the game theoretic approach to handle the time-inconsistency in the identical viewpoint as Ekeland et al. [11] and Björk and Murgoci [3]. Let us briefly explain the game perspective that we will consider as follows.

We explain the concept of a “Nash equilibrium strategy” for the game described as above: This is an admissible control process $\hat{u}\left(\cdot \right)$ fulfilling the following criteria. Assume that every player s, with $s>t$, will apply the strategy $\hat{u}\left(s\right)$. Then the optimal decision for player t is that he/she also uses the strategy $\hat{u}\left(t\right)$. However, the difficulty with this “definition” is that the individual player t does not have any effect on the game’s result. He/she just selects the control at one point t. Furthermore, because this is a time set of Lebesgue measure zero, the control dynamics will be unaffected.

As a result, to identify open-loop Nash equilibrium controls, we follow [15], where a formal definition (Definition 4 below), inspired by [11], is proposed.

In this section, we provide the main results about the necessary and sufficient conditions for equilibrium of the control problem formulated in the preceding section. To make the presentation of the paper more clear, the proofs will be relegated to Appendix A. To proceed towards the definition of an equilibrium, we first introduce the local spike variation for a given admissible control $\hat{u}\left(\cdot \right)\in {\mathcal{L}_{\mathcal{F},p}^{2}}\left(t,T;{\mathbb{R}^{m}}\right)$: for any $t\in \left[0,T\right)$, $v\in {\mathbb{L}^{2}}\left(\Omega ,{\mathcal{F}_{t-}^{\alpha }},\mathbb{P};{\mathbb{R}^{m}}\right)$ and $\varepsilon \in \left(0,T-t\right)$, define We have the following definition.

In this section, we discuss an extension of a new class of optimization problems [36], in which the investor manages her/his wealth by consuming and investing in a financial market subject to a mean variance criterion controlling the final risk of the portfolio. This problem can be eventually formulated as a time-inconsistent stochastic LQ problem and solved by the results presented in the preceding sections.

In this paper, we have considered a class of dynamic decision models of conditional time-inconsistent LQ type, under the effect of a Markovian regime-switching. We have employed the game theoretic approach to handle the time inconsistency. Throughout this study open-loop Nash equilibrium strategies are established as an alternative to optimal strategies. This was achieved using a stochastic system that includes a flow of forward-backward stochastic differential equations under equilibrium conditions. The inclusion of concrete examples in mathematical finance confirms the validity of our proposed study. The work may be developed in different ways: