Backward stochastic differential equations (BSDEs, for short) were initially investigated as adjoint processes under the maximum stochastic control principle by Bismut [3]. Pardoux and Peng [13] obtained the first result dealing with nonlinear BSDEs. They showed the existence and uniqueness of the solution in a Brownian setting under specific conditions, most notably the Lipschitz continuity of the generator via the martingale representation theorem. The case of nonlinear discontinuous BSDEs has been carried out by Tang and Li [18] and Situ [17] in a more general filtration generated by a Brownian motion and an independent Poisson random measure.

In other context, Pardoux and Zhang [15] provided a probabilistic representation for a solution of a system of parabolic and elliptic semilinear partial differential equations (PDEs, for short) with the Neumann boundary condition using a new class of BSDEs which involves the integral with respect to a continuous increasing process interpreted as the local time of a diffusion process on the boundary. This kind of BSDEs is called generalized BSDEs (GBSDEs, for short). A solution of this equation, associated with a terminal value ξ and generators or drivers f ( ω , t , y , z ) and g ( ω , t , y ), is a couple of stochastic processes ( Y t , Z t ) t ≤ T such that (1) Y t = ξ + ∫ t T f ( s , Y s , Z s ) d s + ∫ t T g ( s , Y s ) d A s − ∫ t T Z s d B s , 0 ≤ t ≤ T , where ( B t ) t ≤ T is a standard Brownian motion and the process ( Y t , Z t ) t ≤ T is adapted to the natural filtration of ( B t ) t ≤ T .

Following this work, Pardoux [12] has considered the case of GBSDEs with jumps where the discontinuity stems from the Poisson random measure. The same problem has been treated by El Otmani [6] in the case of GBSDEs driven by the Lévy process, where the author provides the link between those equations and a class of PDEs with the Neumann boundary condition.

In this paper, we are interested in exploring, on a complete probability space ( Ω , F , P ), equipped with a complete, quasi-left continuous, right continuous filtration ( F t ) t ≤ T , a generalized backward stochastic differential equation of the form (2) Y t = ξ + ∫ t T f ( s , Y s , Z s ) d ⟨ M ⟩ s + ∫ t T g ( s , Y s ) d A s − ∫ t T Z s d M s − ∫ t T d N s , driven by the square integrable martingale ( M t ) t ≤ T , where ⟨ M ⟩ denotes the predictable projection of the quadratic variation [ M ], T is a fixed time horizon and F T = F.

As opposed to (1), the solution is now a triplet ( Y t , Z t , N t ) t ≤ T , where ( Z t ) t ≤ T is predictable and ( N t ) t ≤ T is a square integrable martingale orthogonal to M, fulfilling a few integrability requirements that we will specify more explicitly in the following section. We do not impose any constraints on the filtration ( F t ) t ≤ T other than the usual conditions and the quasi-left continuity.

In comparison to existing literature, our work reexamines and generalizes papers by Barles et al. [1], Bender and Kohlmann [2], El Karoui and Huang [5], Carbone et al. [4], Nie and Rutkowski [10] for classical BSDEs, the book by [14] and the aforementioned works [6, 12, 15, 17, 18] since: on the one hand, we allow for a more general filtration instead of dealing with the one generated by a Brownian motion or a Poisson random measure; on the other hand, the generators f and g are stochastic monotone with respect to the y-variable, and the coefficient f is stochastic Lipschitz in the z-variable – this includes all Lipschitz and monotonic constraints placed on drivers that have been published previously. Moreover, under some suitable integrability assumptions on the terminal value ξ and linear growth condition on f ( t , y , 0 ) and g ( t , y ), we provide existence and uniqueness results in an appropriate L 2 space using Yosida’s approximation method. We note that Yosida’s approximation technique has been employed by several authors in this context. Notably, Hu [7] proved the existence and uniqueness results for a class of equations known as forward-backward stochastic differential equations, defined over an arbitrarily prescribed time duration, under specific monotonicity conditions on the coefficients (for other related works on FBSDEs, readers can refer to [8]).

The main difficulties of our problem lies in the fact that, firstly, the GBSDE (2) is considered on an arbitrary filtered probability space, which allows for a more general semimartingale setting, and secondly, the drivers of our GBSDE satisfy weaker assumptions than the ones considered in the literature. To the best of our knowledge, there is no existence and uniqueness result for GBSDEs driven by a right continuous with left limits (RCLL, for short) martingale in a general arbitrary probability space with stochastic monotone coefficients. Finally, note that our proofs can be easily extended to the multidimensional case.

The article is structured as follows. The notations, assumptions, definitions and other properties are covered in Section 1. Section 2 is devoted to giving a priori estimates of the solutions of the GBSDE (2) under stochastic monotonicity of the drivers f and g in the y-variable and the stochastic Lipschitz condition of f with respect to z, which specifically produces the solution’s uniqueness. Finally, in Section 3 we establish the article’s main result. Namely, we prove the existence of the solution using the Yosida approximation method by dividing the proof into several steps in which we deal first with a generator f independent of z, then approximate the resulting equation using a family of GBSDE with a deterministic Lipschitz constant; using this, the general results may be obtained via a fixed point argument in an appropriate Banach space.

Let T > 0 be a fixed time and ( Ω , F , F , P ) be a complete filtered probability space where the filtration F : = ( F t ) t ∈ [ 0 , T ] is quasi-left continuous and satisfies the usual conditions of right continuity and completeness. The initial σ-field F 0 is assumed to be trivial and F T = F. The equality X = Y between any two processes ( X t ) t and ( Y t ) t must be understood in the indistinguishably sens i.e. P ( ω : X t ( ω ) = Y t ( ω ) , ∀ t ∈ [ 0 , T ] ) = 1. For a given RCLL process ( Y t ) t , Y t − = lim s ↗ t Y s is the left limits of Y at t, we set Y 0 − = Y 0 by convention. Y − = ( Y t − ) t the left limited process, Δ Y t = Y t − Y t − the jump of Y at time t and E F t [ Y ] : = E [ Y ∣ F t ]. Next, for given two locally square integrable F-martingales M and N, we denote by ⟨ M , N ⟩ the predictable F-dual projection of the quadratic co-variation process [ M , N ], by M c the continuous part of M. Finally, the integral ∫ s t Y u d X u is interpreted as ∫ ] s , t ] Y u d X u where ] s , t ] = { u ∈ [ 0 , T ] : s < u ≤ t }.

We assume given an R-valued, square-integrable F-martingale M : = ( M ) t ≤ T on ( Ω , F , F , P ). Since the filtration F is right-continuous, there exists a modification of M with RCLL paths, hence, we may assume throughout this paper that M is an RCLL process.

Now, we present the conditions imposed on the data of the generalized BSDE (2) that enable us to prove the main result of the paper.

The basic assumptions on the data ( ξ , f , g , A ). 

Measurability of the data and trajectory properties of the process ( A t ) t ≤ T . 

Stochastic monotonicity of f and g in y. There exists two F t -progressively measurable processes α : Ω × [ 0 , T ] → R and β : Ω × [ 0 , T ] → R − such that:

Stochastic Lipschitz condition on f in z. There exists an F t -progressively measurable process γ : Ω × [ 0 , T ] → R + such that

Linear growth of f and g. For some constant κ > 0 and some [ 1 , ∞ )-valued adapted processes { φ t , ψ t ; 0 ≤ t ≤ T } and all ( t , y ) ∈ [ 0 , T ] × R, we have

Integrability condition. Let ( V t ) t ≤ T and ( Q t ) t ≤ T be the two F t -progressively measurable continuous increasing stochastic processes defined by V t : = ∫ 0 t ( | α s | + α s 2 + γ s 2 ) d ⟨ M ⟩ s + ∫ 0 t β s 2 d A s , Q t : = ⟨ M ⟩ t + A t . Let us set ϕ t λ , θ , μ : = λ V t + θ ⟨ M ⟩ t + μ A t and Φ λ , θ , μ : = e ϕ λ , θ , μ for λ , θ , μ > 0 . We assume that, for any λ, θ, μ > 0, (v)

E [ Φ T λ , θ , μ | ξ | 2 ] < ∞;

Spaces. For λ, θ, μ ≥ 0, we introduce the following spaces to describe the parameters and the solution of the equation (2).

We adopt the following definition of a solution to the GBSDE (2).

First, we give some remarks that will be used subsequently. Remark 2.

Recall that (i)

For any process Z that belongs to H 2 , the process ( ∫ 0 · Z s d M s ) 2 − ∫ 0 · | Z s | 2 d [ M ] s is an F-martingale (Theorem 27 in [16, p. 71]) and we have (3) E F t [ ( ∫ t T Z s d M s ) 2 ] = E F t [ ∫ t T | Z s | 2 d [ M ] s ] = E F t [ ∫ t T | Z s | 2 d ⟨ M ⟩ s ] .

Let ( ξ , f , g , A ) and ( ξ ′ , f ′ , g ′ , A ′ ) be two sets of data, each satisfying the above assumption (H-M). Let ( Y , Z , N ) (resp. ( Y ′ , Z ′ , N ′ )) denote a solution of the generalized BSDE (2) with data ( ξ , f , g , A ) (resp. ( ξ ′ , f ′ , g ′ , A ′ )) in the sense of Definition 1. The following result is useful for further applications. Proposition 1.

Define ℜ ¯ = ℜ − ℜ ′ for ℜ = Y , Z , N , A and ξ. Then, for any λ > 2 and θ, μ > 0, there exists a constant C = C ( λ , θ , μ ) such that, for all s ∈ [ 0 , T ], E F s [ sup s ≤ t ≤ T Φ ¯ t λ , θ , μ | Y ¯ t | 2 ] + E F s [ ∫ s T Φ ¯ r λ , θ , μ | Y ¯ r | 2 d V r ] + E F s [ ∫ s T Φ ¯ r λ , θ , μ | Y ¯ r | 2 d ⟨ M ⟩ r ] + E F s [ ∫ s T Φ ¯ r λ , θ , μ | Y ¯ r | 2 d ‖ A ¯ ‖ r ] + E F s [ ∫ s T Φ ¯ r λ , θ , μ | Z ¯ r | 2 d ⟨ M ⟩ r ] + E F s [ ∫ s T Φ ¯ r λ , θ , μ d [ N ¯ ] r ] ≤ C E F s [ Φ ¯ T λ , θ , μ | ξ ¯ | 2 + ∫ s T Φ ¯ r λ , θ , μ | f ( r , Y r ′ , Z r ′ ) − f ′ ( r , Y r ′ , Z r ′ ) | 2 d ⟨ M ⟩ r ] + E F s [ ∫ s T Φ ¯ r λ , θ , μ ( | g ( r , Y r ′ ) | 2 d ‖ A ¯ ‖ r + | g ( r , Y r ′ ) − g ′ ( s , Y r ′ ) | 2 d A r ′ ) ] . Here ‖ A ¯ ‖ denote the total variation of the process A ¯ = A − A ′ , and Φ ¯ λ , θ , μ : = e ϕ ¯ λ , θ , μ where ϕ ¯ t λ , θ , μ = λ V t + θ ⟨ M ⟩ t + μ ( ‖ A ¯ ‖ t + A t ′ ).