The aim of the present paper is to study an infinite horizon optimal control problem in which the controlled state dynamics is governed by a stochastic delay evolution equation in Hilbert spaces. The existence and uniqueness of the optimal control are obtained by means of associated infinite horizon backward stochastic differential equations without assuming the Gâteaux differentiability of the drift coefficient and the diffusion coefficient. An optimal control problem of stochastic delay partial differential equations is also given as an example to illustrate our results.

1. Introduction

In this paper, we consider a controlled stochastic evolution equation of the following form:

()

where

()

u is the control process in a measurable space (U, 𝒰),and W is a cylindrical Wiener process in a Hilbert space Ξ. A is the generator of a strongly continuous semigroup of bounded linear operator in another Hilbert space H, and the coefficients F and G, defined on [0, ∞) × C ([−τ, 0], H), are assumed to satisfy Lipschitz conditions with respect to appropriate norms. We introduce the cost function

()

Here, g is a given real function, λ is large enough, and the control problem is understood in the weak sense. We wish to minimize the cost function over all admissible controls.

The particular form of the control system is essential for our results, but it covers numerous interesting cases. For example, in the particular cases U = H and R(t, x, u) = u, the term u(t)dt + dW(t) in the state equation can be considered as a control affected by noise.

The stochastic optimal control problem was considered in 1977 by Bismut [1]. The optimal control problem for stochastic partial differential equations in the framework of a compact control state space has been studied in [2–5]. Buckdahn and Raşcanu [6] considered an optimal control problem for a semilinear parabolic stochastic differential equation with a nonlinear diffusion coefficient, and the existence of a quasioptimal (nonrelaxed) control is showed without assuming convexity of the coefficients. In [7–11], the authors provided a direct (classical or mild) solution of the Hamilton-Jacobi-Bellman equation for the value function, which is then used to prove that the optimal control is related to the corresponding optimal trajectory by a feedback law. In Gozzi [10, 11], the existence and uniqueness of a mild solution of the associated Hamilton-Jacobi-Bellman equation are proved, when the diffusion term only satisfies weak nondegeneracy conditions. The proofs are based on the corresponding regularity properties of the transition semigroup of the associated Ornstein-Uhlenbeck process.

The main tools for the control problem are techniques from the theory of backward stochastic differential equations (BSDEs) in the sense of Pardoux and Peng, first considered in the nonlinear case in [12]; see [13, 14] as general references. BSDEs have been successfully applied to control problems; see, for example, [15, 16] and we also refer the reader to [17–20]. Fuhrman and Tessiture [19] considered the optimal control problem for stochastic differential equation in the strong form, assuming Lipschitz conditions and allowing degeneracy of the diffusion coefficient, under some structural constraint on the state equation. Existence of an optimal control for stochastic systems in infinite dimensional spaces also has been obtained in [21–27]. In [21], Fuhrman and Tessitore showed the regularity with respect to parameters and the regularity in the Malliavin spaces for the solution of the backward-forward system and defined the feedback law by Malliavin calculus. Finally, the optimal control is obtained by the feedback. Appealing to the Malliavin calculus, compared with Fuhrman et al. [23], the existence of optimal control for stochastic differential equations with delay is proved by the feedback law. Fuhrman and Tessiture [24] dealt with an infinite horizon optimal control problem for the stochastic evolution equation in Hilbert space, and the optimal control is showed by means of infinite horizon backward stochastic differential equation in infinite dimensional spaces and Malliavin calculus. In Masiero [25], the infinite horizon optimal control problem for stochastic evolution equation is also studied by means of the Hamilton-Jacobi-Bellman equation. In Fuhrman [26], a class of optimal control problems governed by stochastic evolution equations in Hilbert spaces which includes state constraints is considered, and the optimal control is obtained by the Fleming logarithmic transformation. Masiero [27] studied stochastic evolution equations evolving in a Banach space where G is a constant and characterized the optimal control via a feedback law by avoiding use of Malliavin calculus. Since there is a lack of regularity of F and G, Malliavin calculus is not available in this case; the method in [27] also cannot be used as G is not a constant, but we can prove a theorem similar to [26, Proposition 3.2], which will be used to define the feedback law.

In the present paper, we study the infinite horizon optimal control problem for stochastic delay evolution equations in Hilbert spaces, and by using Theorem 10, the optimal control is obtained. Since we do not relate the optimal feedback law with the gradient of the value function and do not consider the associated Hamilton-Jacobi-Bellman equation, we can drop the Gâteaux differentiability of the drift term and the diffusion term.

The plan of the paper is as follows. In the next section, some notations are fixed, and the stochastic delay evolution equations are considered with an infinite horizon; in particular, continuous dependence on initial value (t, x) is proved. In Section 3, we give the proof of Theorem 10, which is the key of many subsequent results. The addressed optimal control problem is considered, and the fundamental relation between the optimal control problem and BSDEs is established in Section 4. Section 5 is devoted to proving the existence and uniqueness of optimal control in the weak sense. Finally, an application is given in Section 6.

2. Preliminaries

We list some notations that are used in this paper. We use the symbol |·| to denote the norm in a Banach space F, with a subscript if necessary. Let Ξ, H, and K denote real separable Hilbert spaces, with scalar products (·, ·) _Ξ, (·, ·) _H, and (·, ·) _K, respectively. For fixed τ > 0, 𝒞 = C([−τ, 0], H) denotes the space of continuous functions from [−τ, 0] to H, endowed with the usual norm |f|_C = sup _{θ∈[−τ,0]} | f(θ)|_H. Let Ξ^* denote the dual space of Ξ, with scalar product , and let L(Ξ, H) denote the space of all bounded linear operators from Ξ into H; the subspace of Hilbert-Schmidt operators, with the Hilbert-Schmidt norm, is denoted by L₂(Ξ, H).

Let (Ω, ℱ, P) be a complete space with a filtration {ℱ_t} _t≥0 which satisfies the usual condition. By a cylindrical Wiener process with values in a Hilbert space Ξ, defined on (Ω, ℱ, P), we mean a family {W(t), t ≥ 0} of linear mappings Ξ → L²(Ω) such that for every ξ, η ∈ Ξ, {W(t)ξ, t ≥ 0} is a real Wiener process and E(W(t)ξ · W(t)η) = (ξ, η) _Ξ. In the following, {W(t), t ≥ 0} is a cylindrical Wiener process adapted to the filtration {ℱ_t} _t≥0.

In this section and the next section, {ℱ_t} _t≥0 will denote the natural filtration of W, augmented with the family 𝒩 of P-null of ℱ. The filtration {ℱ_t, t ≥ 0} satisfies the usual conditions. For [a, b], [a, ∞) ⊂ [0, ∞), we also use the following notations:

()

By 𝒫 we denote the predictable σ-algebra, and by ℬ(Λ) we denote the Borel σ-algebra of any topological space Λ.

Similar to [24], we define several classes of stochastic processes with values in a Banach space F as follows.

(i)
denotes the space of equivalence classes of processes Y ∈ L²(Ω × [t, ∞); F), admitting a predictable version. is endowed with the norm
()
(ii)
, defined for β ∈ R and p, q ∈ [1, ∞), denotes the space of equivalence classes of processes {Y(s), s ≥ t}, with values in F, such that the norm
()
is finite and Y admits a predictable version.
(iii)
denotes the space . The norm of an element is |(Y, Z)| = |Y | +|Z|. Here, F is a Hilbert space.
(iv)
, defined for T > t ≥ 0 and p ∈ [1, ∞), denotes the space of predictable processes {Y(s), s ∈ [t, T]} with continuous paths in F, such that the norm
()

is finite. Elements of

are identified up to indistinguishability.

(v)
, defined for η ∈ R and q ∈ [1, ∞), denotes the space of predictable processes {Y(s), s ≥ t} with continuous paths in F, such that the norm
()
is finite. Elements of are identified up to indistinguishability.
(vi)
Finally, for η ∈ R and q ∈ [1, ∞), we defined as the space , endowed with the norm
()

For simplicity, we denote

, and

, respectively.

Now, for every fixed t ≥ 0, we consider the following stochastic delay evolution equation:

()

We make the following assumptions.

Hypothesis 1. (i) The operator A is the generator of a strongly continuous semigroup {e^tA, t ≥ 0} of bounded linear operators in the Hilbert space H. We denote by M and ω two constants such that |e^tA | ≤ Me^ωt, for t ≥ 0.

(ii) The mapping F: [0, ∞) × 𝒞 → H is measurable and satisfies, for some constant L > 0 and 0 ≤ θ < 1,

()

(iii) G is a mapping [0, ∞) × 𝒞 → L(Ξ, H) such that for every v ∈ Ξ, the map Gv: [0, ∞) × 𝒞 → H is measurable, e^sAG(t, x) ∈ L₂(Ξ, H) for every s > 0, t ∈ [0, ∞) and x ∈ 𝒞, and

()

for some constants L > 0 and γ ∈ [0, 1/2).

We say that X is a mild solution of (10) if it is a continuous, {ℱ_t} _t≥0-predictable process with values in H, and it satisfies P-a.s.,

()

To stress dependence on initial data, we denote the solution by X(s, t, x). Note that X(s, t, x) is ℱ_[t,s] measurable, hence, independent of ℱ_t.

We first recall a well-known result on solvability of (10) on bounded interval.

Theorem 1. Assume that Hypothesis 1 holds. Then, for all q ∈ [2, ∞) and T > 0, there exists a unique process as mild solution of (10). Moreover,

()

for some constant C depending only on q, γ, θ, T, τ, L, ω, and M.

By Theorem 1 and the arbitrariness of T in its statement, the solution is defined for every s ≥ t. We have the following result.

Theorem 2. Assume that Hypothesis 1 holds and the process X(·, t, x) is mild solution of (10) with initial value (t, x)∈[0, ∞) × 𝒞. Then, for every q ∈ [1, ∞), there exists a constant η(q) such that the process . Moreover, for a suitable constant C > 0, one has

()

with the constant η(q) depending only on q, γ, θ, τ, L, ω, and M.

Proof. We define a mapping Φ from to by the formula

()

We are going to show that, provided η is suitably chosen, Φ(·, t, x) is well defined and that it is a contraction in

; that is, there exists c < 1 such that

()

For simplicity, we set t = 0, and we treat only the case F = 0, the general case being handled in a similar way. We will use the so called factorization method; see [28, Theorem 5.2.5]. Let us take q > 1 and α ∈ (0,1) such that

By the stochastic Fubini theorem,

()

where

()

Since sup _{−τ≤l≤0} | e^(s+l)Ax(0)| ≤ Me^ωs | x|_C, the process e^(s+·)Ax(0), s ≥ 0, belongs to

provided ω + η < 0. Next, we estimate Φ^′(X_·), where

()

setting q^′ = q/(q − 1), so that

()

Applying the Young inequality for convolutions, we have

()

and we conclude that

()

If we start again from (20) and apply the Hölder inequality, we obtain

()

So, we conclude that

()

On the other hand, by the Burkholder-Davis-Gundy inequalities, for some constant c_q depending only on q, we have

()

which implies that

()

so that

()

for suitable constants C₁, C₂. Applying the Young inequality for convolutions, we obtain

()

This shows that

is finite provided we assume that η < 0 and ω + η < 0; so, the map is well defined.

If are processes belonging to and Y¹, Y² are defined accordingly, the entirely analogous passages show that

()

Recalling the inequalities (23) and (25) and noting that the map Y → Φ^′(X_·) is linear, we obtain an explicit expression for the constant c in (17), and it is immediate to verify that c < 1 provided η < 0 is chosen sufficiently large. We fix such a value of η(q). The first result is a consequence of the contraction principle. The estimate (15) also follows from the contraction property of Φ(·, t, x).

For investigating the dependence of the solution X(s, t, x) on the initial data x and t, we reformulate (13) as an equation on [0, ∞). We set

()

and we consider the equation

()

Under the assumptions of Hypothesis 1, by Theorem 2, it is easy to prove that equation (32) has a unique solution X and

for every q ∈ [2, ∞). It clearly satisfies X(s) = x((s − t)∨(−τ)) for s ∈ [−τ, t), and its restriction to the time internal [t, ∞) is the unique mild solution of (10). From now on, we denote by X(s, t, x), s ∈ [0, ∞), the solution of (32).

We need the following parameter-depending contraction principle, which is stated in the following lemma and proved in [29, Theorems 10.1 and 10.2].

Lemma 3 (Parameter Depending Contraction Principle). Let B, D denote Banach spaces. Let h : B × D → B be a continuous mapping satisfying

()

for some α ∈ [0,1) and every x₁, x₂ ∈ B, y ∈ D. Let ϕ(y) denote the unique fixed point of the mapping h(·, y) : B → B. Then, ϕ : D → B is continuous.

Theorem 4. Assume that Hypothesis 1 holds true. Then, for every q ∈ [1, ∞), the map (t, x) → X_·(t, x) is continuous from [0, ∞) × 𝒞 to .

Proof. Clearly, it is enough to prove the claim for q large. Let us consider the map Φ defined in the proof of Theorem 2. In our present notation, Φ can be seen as a mapping from to as follows:

()

By the arguments of the proof of Theorem 2, Φ(·, t, x) is a contraction in

uniformly with respect to t, x. The process X_·(t, x) is the unique fixed point of Φ(·, t, x). So, by the parameter-depending contraction principle (Lemma 3), it suffices to show that Φ is continuous from

. From the contraction property of Φ(·, t, x) mentioned earlier, we have that Φ(·, t, x) is continuous, uniformly in t, x. Moreover, for fixed X_·, it is easy to verify that Φ(X_·, ·, ·) is continuous from [0, ∞) × 𝒞 to

. The proof is finished.

Remark 5. By similar passages, we can show that, for fixed t, Theorem 4 still holds true for q large enough if the spaces [0, ∞) × 𝒞 and are replaced by the spaces L^q(Ω, 𝒞, ℱ_t) and respectively, where L^q(Ω, 𝒞, ℱ_t) denotes that the space of ℱ_t-measurable function with value in 𝒞, such that the norm

()

is finite.

3. The Backward-Forward System

In this section, we consider the system of stochastic differential equations, P-a.s.,

()

for s varying on the time interval [t, ∞)⊂[0, ∞). As in Section 2, we extend the domain of the solution setting X(s, t, x) = x((s − t)∨(−τ)) for s ∈ [−τ, t).

We make the following assumptions.

Hypothesis 2. The mapping ψ : [0, ∞) × 𝒞 × K × L₂(Ξ, K) → K is Borel measurable such that, for all t ∈ [0, ∞), ψ(t, ·) : 𝒞 × K × L₂(Ξ, K) → K is continuous, and for some L_y, L_z > 0, μ ∈ R, and m ≥ 1,

()

for every s ∈ [0, ∞), x ∈ 𝒞, y, y₁, y₂ ∈ K, z, z₁, and z₂ ∈ L₂(Ξ, K).

We note that the third inequality in (37) follows from the first one taking μ = −L_y but that the third inequality may also hold for different values of μ.

Firstly, we consider the backward stochastic differential equation

()

K is a Hilbert space, the mapping ψ : [0, ∞) × 𝒞 × K × L₂(Ξ, K) → K is a given measurable function, X_· is a predictable process with values in another Banach space 𝒞, and λ is a real number.

Theorem 6. Assume that Hypothesis 2 holds. Let p > 2 and δ < 0 be given, and choose

()

Then, the following hold.

(i)
For and , (38) has a unique solution in that will be denoted by (Y(X_·)(s), Z(X_·)(s)), s ≥ 0.
(ii)
The estimate
()
holds for a suitable constant c. In particular, .
(iii)
The map X_· → (Y(X_·), Z(X_·)) is continuous from to , and X_· → Y(X_·) is continuous from to .
(iv)
The statements of points (i), (ii), and (iii) still hold true if the space is replaced by the space .

Proof. The theorem is very similar to Proposition 3.11 in [24]. The only minor difference is that the mapping ψ : [0, ∞) × 𝒞 × K × L₂(Ξ, K) → K is a given measurable function, while in [24], the measurable function ψ is from H × K × L₂(Ξ, K) to K; however, the same arguments apply.

Theorem 7. Assume that Hypothesis 1 holds and that Hypothesis 2 holds true in the particular case K = R. Then, for every p > 2, q, δ < 0 satisfying (39) with η = η(q), and for every , there exists a unique solution in of (36) that will be denoted by (X(·, t, x), Y(·, t, x), Z(·, t, x)). Moreover, . The map (t, x)→(Y(·, t, x), Z(·, t, x)) is continuous from [0, ∞) × 𝒞 to , and the map (t, x) → Y(·, t, x) is continuous from [0, ∞) × 𝒞 to .

Proof. We first notice that the system is decoupled; the first does not contain the solution (Y, Z) of the second one. Therefore, under the assumption of Hypothesis 1 by Theorem 2, there exists a unique solution X(·, t, x) and of the first equation. Moreover, from Theorem 4, it follows that the map (t, x) → X_·(t, x) is continuous from [0, ∞) × 𝒞 to .

Let K = R; from Theorem 6, we have that there exists a unique solution of the second equation, and the map X_· → (Y(X_·), Z(X_·)) is continuous from to while X_· → (Y(X_·)) is continuous from to . We have proved that is the unique solution of (36), and the other assertions follow from composition.

Remark 8. From Remark 5, by similar passages, we can show that for fixed t and for q large enough, under the assumptions of Theorem 7, the map x → (Y(·, t, x), Z(·, t, x)) is continuous from L^q(Ω, 𝒞, ℱ_t) to .

We also remark that the process X(·, t, x) is ℱ_[t,∞) measurable, since 𝒞 is separable Banach space, we have that X_·(t, x) is ℱ_[t,∞) measurable; So that Y(t) is measurable with respect to both ℱ_[t,∞) and ℱ_t, it follows that Y(t) is deterministic.

For later use, we notice three useful identities; for t ≤ s < ∞, the equality, P-a.s.,

()

is a consequence of the uniqueness of the solution of (13). Since the solution of the backward equation is uniquely determined on an interval [s, ∞) by the values of the process X_· on the same interval, for t ≤ s < ∞, we have, P-a.s.,

()

Lemma 9 (see [30].)Let E be a metric space with metric d, and let f : Ω → E be strongly measurable. Then, there exists a sequence f_n, n ∈ N, of simple E-valued functions (i.e., f_n is ℱ/ℬ(E) measurable and takes only a finite number of values) such that for arbitrary ω ∈ Ω, the sequence d(f_n(ω), f(ω)), n ∈ N, is monotonically decreasing to zero.

Let now f ∈ L^q(Ω, 𝒞). By Lemma 9 we get the existence of a sequence of simple function f_n, n ∈ N, such that

()

Hence, f_n → f in

by Lebesgue′s dominated convergence theorem.

We are now in a position of showing the main result in this section.

Theorem 10. Assume that Hypothesis 1 holds true and that Hypothesis 2 holds in the particular case K = R. Then, there exist two Borel measurable deterministic functions υ : [t, ∞) × 𝒞 → R and ζ : [t, ∞) × 𝒞 → Ξ^* = L(Ξ, R) = L₂(Ξ, R), such that for t ∈ [0, ∞) and x ∈ 𝒞, the solution (X(t, x), Y(t, x), Z(t, x)) of (36) satisfies

()

Proof. We apply the techniques introduced in [26, Proposition 3.2]. Let {e_i} be a basis of Ξ^*, and let us define . Then, for every 0 ≤ t₁ < t₂ < ∞, Δ > 0, and x₁, x₂ ∈ 𝒞, we have that

()

From Theorem 7, we have that the map

is continuous from [0, ∞) × 𝒞 to R. By Remark 8, we also have that, for fixed t, the map

is continuous from L^q(Ω, 𝒞, ℱ_t) to R for q large enough. Let us define

()

It is clear that ζ^i,N : [0, ∞) × 𝒞 → R is a Borel function.

We fix x and 0 ≤ t ≤ s < ∞. For l ∈ [s, ∞), we denote , the random variable obtained by composing X_s(t, x) with the map y → E[Z^i,N(l, s, y)].

By Lemma 9, there exists a sequence of 𝒞-valued ℱ_s-measurable simple functions

()

where

are pairwise distinct and

, such that

()

For any B ∈ ℱ_s, we have

()

and we get that

()

Fix t and x. Recalling that |Z^i,N | ≤ N, by the Lebesgue theorem on differentiation, it follows that P-a.s.

()

By the boundedness of Z^i,N, applying the dominated convergence theorem, we get that

()

Now, we have proved that for every t, x,

()

for every i, N. Let C ⊂ [0, ∞) × 𝒞 denote the set of pairs (t, x) such that lim _N→∞ζ^i,N(t, x) exists and the series

converges in Ξ^*. We define

()

Since Z satisfies

()

for every ω, s, t, x. From (53), it follows that for every t, x, we have (s, X_s(ω, t, x)) ∈ C, P-a.s., for almost all s ∈ [t, ∞), and Z(s, t, x) = ζ(s, X_s(t, x)) P-a.s., for a.a. s ∈ [t, ∞).

We define υ(t, x) = Y(t, t, x); since Y(t, t, x) is deterministic, so the map (t, x) → υ(t, x) can be written as a composition υ(t, x) = Γ₃(Γ₂(t, Γ₁(t, x))) with

()

From Theorem 7, it follows that Γ₁ is continuous. By

()

we have that Γ₂ is continuous. It is clear that Γ₃ is continuous. Then, the map (t, x) → υ(t, x) is continuous from [0, ∞) × 𝒞 to R; therefore, υ(t, x)is a Borel measurable function. From uniqueness of the solution of (36), it follows that Y(s, t, x) = υ(s, X_s(t, x)), P-a.s., for a.a. s ∈ [t, ∞).

4. The Fundamental Relation

Let (Ω, ℱ, P) be a given complete probability space with a filtration {ℱ_t} _t≥0 satisfying the usual conditions. {W(t), t ≥ 0} is a cylindrical Wiener process in Ξ with respect to {ℱ_t} _t≥0. We will say that an {ℱ} _t≥0-predictable process u with values in a given measurable space (U, 𝒰) is an admissible control. The function R : [0, ∞) × 𝒞 × U → Ξ is measurable and bounded. We consider the following controlled state equation:

()

Here, we assume that there exists a mild solution of (58) which will be denoted by X^u(s, t, x) or simply by X^u(s). We consider a cost function of the form:

()

Here, g is function on [0, ∞) × 𝒞 × U with real values. Our purpose is to minimize the function J over all admissible controls.

We define in a classical way the Hamiltonian function relative to the previous problem; for all t ∈ [0, ∞), x ∈ 𝒞, and z ∈ Ξ^*,

()

and the corresponding, possibly empty, set of minimizers

()

We are now ready to formulate the assumptions we need.

Hypothesis 3. (i) A, F, and G verify Hypothesis 1.

(ii) (U, 𝒰) is a measurable space. The map g : [0, ∞) × 𝒞 × U → R is continuous and satisfies for suitable constants K_g > 0, m_g > 0 and all x ∈ 𝒞,u ∈ U. The map R : [0, ∞) × 𝒞 × U → Ξ is measurable, and |R(t, s, u)| ≤ L_R for a suitable constant K_R > 0 and all x ∈ 𝒞,u ∈ U, andz ∈ Ξ^*.

(iii) The Hamiltonian ψ defined in (60) satisfies the requirements of Hypothesis 2 (with K = R).

(iv) We fix here p > 2, q and δ < 0 satisfying (39) with η = η(q) and such that q > m_g.

We are in a position to prove the main result of this section.

Theorem 11. Assume that Hypothesis 3 holds, and suppose that λ verifies

()

Let υ, ζ denote the function in the statement of Theorem 10. Then, for every admissible control u and for the corresponding trajectory X starting at (t, x), one has

()

Proof. Consider (58) in the probability space (Ω, ℱ, P) with filtration {ℱ_t} _t≥0 and with an {ℱ_t} _t≥0-cylindrical Wiener process {W(t), t ≥ 0}. Let us define

()

Let P^u be the unique probability on ℱ_[0,∞) such that

()

We notice that under P^u, the process W^u is a Wiener process. Let us denote by

the filtration generated by W^u, and completed in the usual way. Relatively to W^u (58) can be rewritten as

()

In the space

, we consider the following system of forward-backward equations:

()

Applying the Itô formula to e^−λsY^u(s) and writing the backward equation in (67) with respect to the process W, we get

()

Recalling that R is bounded, we get, for all r ≥ 1 and some constant C,

()

It follows that

()

We conclude that the stochastic integral in (68) has zero expectation. If we set s = t in (68) and we take expectation with respect to P, we obtain

()

By Theorem 7,

, so that

()

By the Hölder inequality, we have that for suitable constant C > 0,

()

From Theorem 2, we obtain

; by the similar process, we get that

()

for suitable constant C > 0 and

()

Since Y^u(t, t, x) = υ(t, x) and

-a.s., for a.a. s ∈ [t, ∞), we have that

()

Thus, adding and subtracting

and letting T → ∞, we conclude that

()

The proof is finished.

We immediately deduce the following consequences.

Theorem 12. Let t ∈ [0, ∞) and x ∈ 𝒞 be fixed, assume that the set-valued map Γ has nonempty values and it admits a measurable selection Γ₀ : [0, ∞) × 𝒞 × Ξ^* → U, and assume that a control u(·) satisfies

()

Then, J(t, x, u) = υ(t, x), and the pair (u(·), X) is optimal for the control problem starting from x at time t.

Such a control can be shown to exist if there exists a solution for the so-called closed-loop equation as follows:

()

since in this case, we can define an optimal control setting

()

However, under the present assumptions, we cannot guarantee that the closed-loop equation has a solution in the mild sense. To circumvent this difficulty, we will revert to a weak formulation of the optimal control problem.

5. Existence of Optimal Control

We formulate the optimal control problem in the weak sense following the approach of [31]. The main advantage is that we will be able to solve the closed-loop equation in a weak sense, and, hence, to find an optimal control, even if the feedback law is nonsmooth.

We call (Ω, ℱ, {ℱ_t} _t≥0, P, W) an admissible setup, if (Ω, ℱ, {ℱ_t} _t≥0, P) is a filtered probability space satisfying the usual conditions, and W is a cylindrical P-Wiener process with values in Ξ, with respect to the filtration {ℱ_t} _t≥0.

By an admissible control system, we mean (Ω, ℱ, {ℱ_t} _t≥0, P, W, u, X^u), where (Ω, ℱ, {ℱ_t} _t≥0, P, W) is an admissible setup, u is an ℱ_t-predictable process with values in U, and X^u is a mild solution of (58). An admissible control system will be briefly denoted by (W, u, X^u) in the following. Our purpose is to minimize the cost functional

()

over all the admissible control system.

Our main result in this section is based on the solvability of the closed-loop equation

()

In the following sense, we say that X is a weak solution of (82) if there exists an admissible setup (Ω, ℱ, {ℱ_t} _t≥0, P, W) and an ℱ_t-adapted continuous process X(t) with values in H, which solves the equation in the mild sense; namely, P-a.s.,

()

Theorem 13. Assume that Hypothesis 3 holds. Then, there exists a weak solution of the closed-loop equation (82) which is unique in law.

Proof (uniqueness). Let X be a weak solution of (82) in an admissible setup (Ω, ℱ, {ℱ_t} _t≥0, P, W). We define

()

Since R is bounded, the Girsanov theorem ensures that there exists a probability measure P⁰ such that the process

()

is a P⁰-Wiener process and

()

Let us denote by

the filtration generated by W⁰ and completed in the usual way. In

, X is a mild solution of

()

By Hypothesis 3, the joint law of X and W⁰ is uniquely determined by A, F, G, and x. Taking into account the last displayed formula, we conclude that the joint law of X and ρ(T) under P⁰ is also uniquely determined, and consequently so is the law of X under P. This completes the proof of the uniqueness part.

Proof (existence). Let (Ω, ℱ, P) be a given complete probability space. {W(t), t ≥ 0} is a cylindrical Wiener process on (Ω, ℱ, P) with values in Ξ, and {ℱ_t} _t≥0 is the natural filtration of {W(t), t ≥ 0}, augmented with the family of P-null sets. Let X(·) be the mild solution of

()

and by the Girsanov theorem, let P¹ be the probability on Ω under which

()

is a Wiener process (notice that R is bounded). Then, X is the weak solution of (82) relatively to the probability P¹ and the Wiener process W¹.

Now, we can state the main result of this section.

Corollary 14. Assume that Hypothesis 3 holds true and λ verifies (62) Also, assume that the set-valued map Γ has nonempty values and it admits a measurable selection Γ₀ : [0, ∞) × 𝒞 × Ξ^* → U. Then, for every t ∈ [0, ∞) and x ∈ 𝒞 and for all admissible control system (W, u, X^u), one has

()

and the equality holds if

()

Moreover, from Theorem 13, it follows that the closed-loop equation (82) admits a weak solution (Ω, ℱ, {ℱ_t} _t≥0, P, W, X) which is unique in law, and setting

()

we obtain an optimal admissible control system (W, u, X).

6. Applications

In this section, we present a simple application of the previous results. We consider the stochastic delay partial differential equation in the bounded domain B ⊂ Rⁿ with smooth boundary ∂B as follows:

()

Here, W = (W¹, W², …, W^d) is a standard Wiener process in R^d, and the functions f : [0, +∞) × C([−1,0], R) → R and g_i : [0, +∞) × C([−1,0], R) → R are Lipschitz continuous and bounded. Setting U as a bounded subset of R^d, Ξ = R^d, H = L²(B), and x ∈ C([−1,0], H). We define F and G as following:

()

and let A denote the Laplace operator Δ in L²(B); with domain

then, (94) has the form (58) and Hypothesis 1 holds.

Let us consider the optimal control problem associated with the cost

()

where λ verifies (62) and σ : C([−1,0], R) × U → [0, ∞) is a bounded measurable function. Define g : C([−1,0], H) × U → [0, ∞) and R : C([−1,0], H) × U → Ξ by g(y, u) = ∫_B σ(t, y(ξ), u)dξ + u² and R(y, u) = (∫_B r¹(ξ)u¹dξ, ∫_B r²(ξ)u²dξ, …, ∫_B r^d(ξ)u^ddξ) for y ∈ C([−1,0], H), u = (u¹, u², …, u^d) ∈ U, respectively. It can be easily verified that Hypothesis 3 holds true, and the set-valued map Γ has nonempty values and it admits a measurable selection Γ₀ : [0, ∞) × 𝒞 × Ξ^* → U. Then, the closed-loop equation (82) admits a weak solution (Ω, ℱ, {ℱ_t} _t≥0, P, W, u, z_·(·)), and setting

()

we obtain an optimal admissible control system (W, u, z(·)).

References

1 Bismut J., On optimal control of linear stochastic equations with a linear-quadratic criterion, SIAM Journal on Control and Optimization. (1977) 15, no. 3, 1–4, 2-s2.0-0016949806, https://doi.org/10.1137/0314028, ZBL0331.93086.
10.1137/0315001
Web of Science® Google Scholar
2 Nagase N., On the existence of optimal control for controlled stochastic partial differential equations, Nagoya Mathematics Journal. (1989) 115, 73–85.
10.1017/S0027763000001549
Web of Science® Google Scholar
3 El Karoui N., Huu Nguyen D., and Jeanblanc-Piqué M., Compactification methods in the control of degenerate diffusions, Stochastics. (1987) 20, 169–219.
10.1080/17442508708833443
Google Scholar
4 Nisio M., Optimal control for stochastic partial differential equations and viscosity solutions of Bellman equations, Nagoya Mathematics Journal. (1991) 123, 13–37.
10.1017/S0027763000003639
Web of Science® Google Scholar
5 Nisio M., H. Kunita et al., On sensitive control for stochastic partial differential equations, 310, Stochastic Analysis on Infinite Dimensional Spaces Proceedings of the U.S. Japan Bilateral Seminar, January 1994, Baton Rouge, La, USA, Longman Scientific and Technical, 231–241, Pitman Research Notes Mathematical Series.
Google Scholar
6 Buckdahn R. and Raşcanu A., On the existence of stochastic optimal control of distributed state system, Nonlinear Analysis, Theory, Methods and Applications. (2003) 52, no. 4, 1153–1184, 2-s2.0-0037290471, https://doi.org/10.1016/S0362-546X(02)00158-X, ZBL1030.93057.
10.1016/S0362-546X(02)00158-X
Web of Science® Google Scholar
7 Barbu V. and Da Prato G., Equations in Hilbert Spaces, 1983, 86, Pitman, Pitman Research Notes in Mathematics.
Google Scholar
8 Cannarsa P. and Da Prato G., Second-order Hamilton-Jacobi equations in infinite dimensions, SIAM Journal on Control and Optimization. (1991) 29, no. 2, 474–492, 2-s2.0-0026118706, https://doi.org/10.1137/0329026, ZBL0737.49020.
10.1137/0329026
Web of Science® Google Scholar
9 Cannarsa P. and Da Prato G., G. Da Prato and L. Tubaro, Direct solution of a second-order Hamilton-Jacobi equations in Hilbert spaces, Stochastic Partial Differential Equations and Applications, 1992, 268, Pitman, Pitman Research Notes in Mathematics.
Google Scholar
10 Gozzi F., Regularity of solutions of second order Hamilton-Jacobi equations and application to a control problem, Communications in Partial Differential Equations. (1995) 20, 775–826.
10.1080/03605309508821115
Web of Science® Google Scholar
11 Gozzi F., Global regular solutions of second order Hamilton-Jacobi equations in Hilbert spaces with locally Lipschitz nonlinearities, Journal of Mathematical Analysis and Applications. (1996) 198, no. 2, 399–443, 2-s2.0-0030094496, https://doi.org/10.1006/jmaa.1996.0090, ZBL0858.35129.
10.1006/jmaa.1996.0090
Web of Science® Google Scholar
12 Pardoux E. and Peng S. G., Adapted solution of a backward stochastic differential equation, Systems and Control Letters. (1990) 14, no. 1, 55–61, 2-s2.0-0025262967, https://doi.org/10.1016/0167-6911(90)90082-6, ZBL0692.93064.
10.1016/0167-6911(90)90082-6
Web of Science® Google Scholar
13 N. El Karoui and L. Mazliak, Backward Stochastic Differential Equations, 1997, 364, Longman, Pitman Research Notes in Mathematics Series.
Google Scholar
14 Pardoux E. and BSDEs, F. H. Clarke and R. J. Stern, weak convergence and homogeneization of semilinear PDEs, Non- Linear Analysis, Differential Equations and Control, 1999, Kluwer, Dordrecht, The Netherlands, 503–549.
10.1007/978-94-011-4560-2_9
Google Scholar
15 Peng S., A generalized dynamic programming principle and Hamilton-Jacobi-Bellman equation, Stochastics and Stochastics Reports. (1992) 38, 119–134.
10.1080/17442509208833749
Google Scholar
16 Karoui N. E., Peng S., and Quenez M. C., Backward stochastic differential equations in finance, Mathematical Finance. (1997) 7, no. 1, 1–71, 2-s2.0-0031542653, https://doi.org/10.1111/1467-9965.00022, ZBL0884.90035.
10.1111/1467-9965.00022
Web of Science® Google Scholar
17 Hamadµene S. and Lepeltier J. P., Backward equations, stochastic control and zero-sum stochastic differential games, Stochastics and Stochastics Reports. (1995) 54, 221–231, ZBL0877.93125.
10.1080/17442509508834006
Google Scholar
18 El-Karoui N. and Hamadène S., BSDEs and risk-sensitive control, zero-sum and nonzero-sum game problems of stochastic functional differential equations, Stochastic Processes and their Applications. (2003) 107, no. 1, 145–169, 2-s2.0-0042843669, https://doi.org/10.1016/S0304-4149(03)00059-0, ZBL1075.60534.
10.1016/S0304-4149(03)00059-0
Web of Science® Google Scholar
19 Fuhrman M. and Tessiture G., Existence of optimal stochastic controls and global solutions of forward-backward stochastic differential equations, SIAM Journal on Control and Optimization. (2005) 43, no. 3, 813–830, 2-s2.0-19944417285, https://doi.org/10.1137/S0363012903428664.
10.1137/S0363012903428664
Web of Science® Google Scholar
20 Fuhrman M., Hu Y., and Tessitore G., On a class of stochastic optimal control problems related to bsdes with quadratic growth, SIAM Journal on Control and Optimization. (2006) 45, no. 4, 1279–1296, 2-s2.0-34547211757, https://doi.org/10.1137/050633548, ZBL1125.93069.
10.1137/050633548
Web of Science® Google Scholar
21 Fuhrman M. and Tessitore G., Nonlinear kolmogorov equations in infinite dimensional spaces: the backward stochastic differential equations approach and applications to optimal control, Annals of Probability. (2002) 30, no. 3, 1397–1465, 2-s2.0-0036630776, https://doi.org/10.1214/aop/1029867132, ZBL1017.60076.
10.1214/aop/1029867132
Web of Science® Google Scholar
22 Masiero F., Semilinear kolmogorov equations and applications to stochastic optimal control, Applied Mathematics and Optimization. (2005) 51, no. 1, 201–250, 2-s2.0-17444395616, https://doi.org/10.1137/0315001, ZBL1083.35143.
10.1007/s00245-004-0810-6
Web of Science® Google Scholar
23 Fuhrman M., Masiero F., and Tessitore G., Stochastic equations with delay: optimal control via BSDEs and regular solutions of Hamilton-jacobi-bellman equations, SIAM Journal on Control and Optimization. (2010) 48, no. 7, 4624–4651, 2-s2.0-77958602849, https://doi.org/10.1137/080730354, ZBL1213.60109.
10.1137/080730354
Web of Science® Google Scholar
24 Fuhrman M. and Tessiture G., Infinite horizon backward stochastic differential equations and elliptic equations in hilbert spaces, Annals of Probability. (2004) 32, no. 1, 607–660, 2-s2.0-2142752711, https://doi.org/10.1214/aop/1079021459, ZBL1046.60061.
10.1214/aop/1079021459
Web of Science® Google Scholar
25 Masiero F., Infinite horizon stochastic optimal control problems with degenerate noise and elliptic equations in Hilbert spaces, Applied Mathematics and Optimization. (2007) 55, no. 3, 285–326, 2-s2.0-34547355166, https://doi.org/10.1007/s00245-006-0864-3, ZBL1128.93057.
10.1007/s00245-006-0864-3
Web of Science® Google Scholar
26 Fuhrman M., A class of stochastic optimal control problems in Hilbert spaces: BSDEs and optimal control laws, state constraints, conditioned processes, Stochastic Processes and their Applications. (2003) 108, no. 2, 263–298, 2-s2.0-0142087747, https://doi.org/10.1016/j.spa.2003.09.002, ZBL1075.93044.
10.1016/j.spa.2003.09.002
Web of Science® Google Scholar
27 Masiero F., Stochastic optimal control problems and parabolic equations in banach spaces, SIAM Journal on Control and Optimization. (2008) 47, no. 1, 251–300, 2-s2.0-45749151863, https://doi.org/10.1137/050632725, ZBL1157.93043.
10.1137/050632725
Web of Science® Google Scholar
28 Da Prato G. and Zabczyk J., Ergodicity For Infinite-Dimensional Systems, 1996, Cambridge University Press.
10.1017/CBO9780511662829
Google Scholar
29 Zabczyk J., Parabolic equations on Hilbert spaces, StochaStic PDE′S and Kolmogorov Equations in Infinite Dimensions, 1999, 1715, Springer, Berlin, Germany, 117–213, Lecture Notes in Math.
10.1007/BFb0092419
Google Scholar
30 Da Prato G. and Zabczyk J., Stochstic Equations in Infinite Dimensions, 1992, Cambridge University Press.
10.1017/CBO9780511666223
Google Scholar
31 Fleming W. H. and Soner H. M., Controlled Markov Processes and Viscosity Solutions, 1993, 25, Springer, New York, NY, USA, Applications of Mathematics.
Google Scholar

All articles

Infinite Horizon Optimal Control of Stochastic Delay Evolution Equations in Hilbert Spaces

Abstract

1. Introduction

2. Preliminaries

3. The Backward-Forward System

4. The Fundamental Relation

5. Existence of Optimal Control

6. Applications

References

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley

Infinite Horizon Optimal Control of Stochastic Delay Evolution Equations in Hilbert Spaces

Abstract

1. Introduction

2. Preliminaries

3. The Backward-Forward System

4. The Fundamental Relation

5. Existence of Optimal Control

6. Applications

References

References

Related

Information