On one stochastic optimal control problem with variable delay
The purpose of this paper is to give necessary conditions for the optimality of nonlinear stochastic control systems with variable delay and with constraint on the right end of a trajectory. The necessary optimality conditions in the form of a stochastic analogy of the maximum principle are obtained...
Gespeichert in:
| Datum: | 2007 |
|---|---|
| Hauptverfasser: | , |
| Format: | Artikel |
| Sprache: | Englisch |
| Veröffentlicht: |
Інститут математики НАН України
2007
|
| Online Zugang: | https://nasplib.isofts.kiev.ua/handle/123456789/4501 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Назва журналу: | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| Zitieren: | On one stochastic optimal control problem with variable delay / Ch.A. Agayeva, J.J. Allahverdiyeva // Theory of Stochastic Processes. — 2007. — Т. 13 (29), № 3. — С. 3–11. — Бібліогр.: 6 назв.— англ. |
Institution
Digital Library of Periodicals of National Academy of Sciences of Ukraine| _version_ | 1860123775333302272 |
|---|---|
| author | Agayeva, Ch.A. Allahverdiyeva, J.J. |
| author_facet | Agayeva, Ch.A. Allahverdiyeva, J.J. |
| citation_txt | On one stochastic optimal control problem with variable delay / Ch.A. Agayeva, J.J. Allahverdiyeva // Theory of Stochastic Processes. — 2007. — Т. 13 (29), № 3. — С. 3–11. — Бібліогр.: 6 назв.— англ. |
| collection | DSpace DC |
| description | The purpose of this paper is to give necessary conditions for the optimality of nonlinear stochastic control systems with variable delay and with constraint on the right end of a trajectory. The necessary optimality conditions in the form of a stochastic analogy of the maximum principle are obtained. These conditions are contained in Theorems 1 and 2.
|
| first_indexed | 2025-12-07T17:40:47Z |
| format | Article |
| fulltext |
Theory of Stochastic Processes
Vol. 13 (29), no. 3, 2007, pp. 3–11
UDC 519.21
CH. A. AGAYEVA AND J. J. ALLAHVERDIYEVA
ON ONE STOCHASTIC OPTIMAL CONTROL
PROBLEM WITH VARIABLE DELAY
The purpose of this paper is to give necessary conditions for the optimality of non-
linear stochastic control systems with variable delay and with constraint on the right
end of a trajectory. The necessary optimality conditions in the form of a stochastic
analogy of the maximum principle are obtained. These conditions are contained in
Theorems 1 and 2.
Introduction
Stochastic differential equations with delay find many applications in authomatic con-
trol theory, in the theory of self-oscillating systems, etc., where real systems are subjected
to the influence of random disturbances which cannot be ignored [1,2]. Optimal control
problems for the systems described by means of such equations have been already in-
vestigated in [3]-[5]. This research is devoted to a problem of stochastic optimal control
with delay both on control and state, when the cost function contains a variable delay
as well.
Statement of the problem
Let (Ω, F, P ) be a complete probability space with the filtration {F t : t0 ≤ t ≤ t1}
generated by the Wiener process wt and F t = σ(ws; t0 ≤ s ≤ t). L2
F (t0, t1, Rn) – space
of predictable processes xt(ω) such that: E
∫ t1
t0
|xt|2dt < +∞. Consider the following
stochastic system with delay:
dxt = g(xt, xt−h(t), ut, ut−h1(t), t)dt + σ(xt, xt−h(t), t)dwt, t ∈ (t0, t1];(1)
xt = Φ(t), t ∈ [t0 − h(t0), t0);(2)
xt0 = x0;(3)
ut = Q(t), t ∈ [t0 − h1(t0), t0);(4)
ut ∈ U∂ ≡ {u(·, ·) ∈ L2
F (t0, t1; Rm)|u(t, ·) ∈ U ⊂ Rm, a.s.}(5)
where U – non-empty bounded set, Φ(t), Q(t) – piecewise continuous non-random func-
tions, h(t) ≥ 0 and h1(t) ≥ 0 – continuously differentiable non-random functions, and
dh(t)
dt < 1, dh1(t)
dt < 1.
It is required to minimize the following functional in a set of admissible controls:
(6) J(u) = E
{
p(xt1 ) +
∫ t1
t0
l(xt, xt−h(t), ut, ut−h1(t), t)dt
}
2000 AMS Mathematics Subject Classification. Primary 93E20.
Key words and phrases. Stochastic differential equations, variable delay, vtochastic optimal control
problem, necessary conditions of optimality, admissible controls.
3
4 CH. A. AGAYEVA AND J. J. ALLAHVERDIYEVA
under the condition
(7) Eq(xt1) ∈ G ⊂ Rk,
where G – closed convex set in Rk.
Let assume that the following requirements are satisfied:
I. Functions l, g, and σ are continuous in all arguments.
II. When (t, u) are fixed, then l, g, σ functions are continuously differentiable with
respect to (x, y) and satisfy the condition of linear growth:
(1 + |x| + |y|)−1(|g(x, y, u, v, t)| + |gx(x, y, u, v, t)|+
+|gy(x, y, u, v, t)| + |σ(x, y, t)| + |σx(x, y, t)| + |σy(x, y, t)|) ≤ N
(1 + |x|)−1(|l(x, y, u, v, t)| + |lx(x, y, u, v, t)| + |ly(x, y, u, v, t)|) ≤ N.
III. Function p(x) : Rn → R1 is continuously differentiable, and |p(x)| + |px(x)| ≤
N(1 + |x|).
IV. Function q(x) : Rm → Rk is continuously differentiable, and |q(x)| + |qx(x)| ≤
N(1 + |x|).
First, we consider the stochastic optimal control problem (1)-(6).
Problem without constraint
We obtained the following result that is a necessary condition of optimality for problem
(1)-(6):
Theorem 1. Let conditions I-III hold, and let (x0
t , u
0
t ) be a solution of problem (1)–(6).
Let there exist the random processes (ψt, βt) ∈ L2
F (t0, t1; Rn) × L2
F (t0, t1; Rn×n), which
are the solutions of the adjoint equation
(8)
⎧⎪⎪⎪⎨⎪⎪⎪⎩
dψt = −[Hx(ψt, x
0
t , y
0
t , u0
t , v
0
t , t) + Hy(ψz , x
0
z, y
0
z , u0
z, v
0
z , z)|z=s(t)s
′(t)]dt+
+βtdwt, t0 ≤ t < t1 − h(t1),
dψt = −Hx(ψt, x
0
t , y
0
t , u
0
t , v
0
t , t) + βtdwt, t1 − h(t1) ≤ t < t1,
ψt1 = −px(x0
t1 ).
Then, ∀ ũ ∈ U a.c., the following relations hold:
(9)
⎧⎪⎪⎪⎨⎪⎪⎪⎩
H(ψt, x
0
t , y
0
t , u, v0
t , t) − H(ψt, x
0
t , y
0
t , u0
t , v
0
t , t)+
+[H(ψz, x
0
z, y
0
z , u0
z, u, z)− H(ψz, x
0
z , y
0
z , u0
z, v
0
z , z)]|z=r(t)r
′(t) ≤ 0,
a.e. t ∈ [t0, t1 − h1(t1)),
H(ψt, x
0
t , y
0
t , u, v0
t , t) − H(ψt, x
0
t , y
0
t , u0
t , v
0
t , t) ≤ 0, a.e. t ∈ [t1 − h1(t1), t1].
Here, τ = s(τ) is a solution of the equation τ = t − h(t), τ = r(τ) is a solution of
equation τ = t − h1(t), yt = xt−h(t), vt = ut−h1(t), and
H(ψt, xt, yt, ut, vt, t) = ψ∗
t g(xt, yt, ut, vt, t) + β∗
t σ(xt, yt, t) − l(xt, yt, ut, vt, t).
Proof. Let u1 = u0
t + Δut be some admissible control, and let x1 = x0
t + Δxt be the
trajectory of system (1)-(5) corresponding to this control. We use the identities
dΔxt = [g(xt, yt, ut, vt, t) − g(x0
t , y
0
t , u0
t , v
0
t , t)]dt + [σ(xt, yt, t)−
− σ(x0
t , y
0, t)]dwt = {Δug(x0
t , y
0
t , u0
t , v
0
t , t) + Δvg(x0
t , y
0
t , u0
t , v
0
t , t)+
+ gx(x0
t , y
0
t , u0
t , v
0
t , t)Δxt + gy(x0
t , y
0
t , u0
t , v
0
t , t)Δyt}dt+(10)
+ {σx(x0
t , y
0
t , t)Δxt + σy(x0
t , y
0
t , t)Δyt}dwt + η1
t , t ∈ (t0, t1]
Δxt = 0, t ∈ [t0 − h(t0), t0],
ON ONE STOCHASTIC OPTIMAL CONTROL PROBLEM WITH VARIABLE DELAY 5
where
η1
t =
{∫ 1
0
[g∗x(x0
t + μΔxt, yt, ut, vt, t) − g∗x(x0
t , y
0
t , u0
t , v
0
t , t)]Δxtdμ+
+
∫ 1
0
[g∗y(x0
t , y
0
t + μΔyt, ut, vt, t) − g∗y(x0
t , y
0
t , ut, vt, t)]Δytdμ
}
dt+
+
{∫ 1
0
[σ∗
x(x0
t + μΔxt, yt, t) − σ∗
x(x0
t , yt, t)]Δxtdμ+
+
∫ 1
0
[σ∗
y(x0
t , y
0
t + μΔyt, t) − σ∗
y(x0
t , y
0
t , t)]Δytdμ
}
dwt
and
d(ψ∗
t · Δxt) = dψ∗
t · Δxt + ψ∗
t · dΔxt + {β∗
t σx(x0
t , y
0
t , t) · Δxt + β∗
t σy(x0
t , y
0
t , t) · Δyt+
+ β∗
t
∫ 1
0
[σx(x0
t + μΔxt, y, t) − σx(x0
t , y, t)]Δxtdμ+
+ β∗
t
∫ 1
0
[σy(x0
t , y
0
t + μΔyt, t) − σy(x0
t , y
0
t , t)]Δytdμ}dt.(11)
The increment of functional (6) along the admissible control looks like
ΔuJ(u) = E
{
p
(
xt1 − p(x0
t1) +
∫ t1
t0
)
[l(xt, yt, ut, vt, t) − l(x0
t , y
0
t , u0
t , v
0
t , t)]dt
}
=
= Epx(x0
t1)Δxt1 + E
∫ t1
t0
[Δul(x0
t , y
0
t , u0
t , v
0
t , t) + Δvl(x0
t , y
0
t , u0
t , v
0
t , t)+
(12)
+ lx(x0
t , y
0
t , u0
t , v
0
t , t)Δxt + ly(x0
t , y
0
t , u0
t , v
0
t , t)Δyt]dt + η2,
where
η2 = E
∫ 1
0
[p∗x(x0
t1 + μΔxt1) − p∗x(x0
t1)]Δvxt1dμ
+ E
∫ t1
t0
{∫ 1
0
[l∗x(x0
t1 + μΔxt, yt, ut, vt, t) − l∗x(x0
t , yt, ut, vt, t)Δxtdμ
+
∫ 1
0
[l∗y(x0
t , y
0
t + μΔyt, ut, vt, t) − l∗y(x0
t , y
0
t , ut, vt, t)]Δytdμ
}
dt.
Taking (10) and (11) into consideration, expression (12) takes the form
ΔuJ(u0) = −E
∫ t1
t0
dψ∗
t Δxt − E
∫ t1
t0
ψ∗
t {[Δug(x0
t , y
0
t , u0
t , v
0
t , t)+
+ Δvg(x0
t , y
0
t , u0
t , v
0
t , t) + gx(x0
t , y
0
t , u0
t , v
0
t , t)Δxt+
+ gy(x0
t , y
0
t , u0
t , v
0
t , t)Δyt]dt + [σx(x0
t , y
0
t , t)Δxt + σy(x0
t , y
0
t , t)Δyt]}dwt−
(13)
− E
∫ t1
t0
β∗
t [σx(x0
t , y
0
t , t)Δxt + σy(x0
t , y
0
t , t)Δyt]dt + E
∫ t1
t0
[Δul(x0
t , y
0
t , u0
t , v
0
t , t)+
+ Δvl(x0
t , y
0
t , u0
t , v
0
t , t) + lx(x0
t , y
0
t , u0
t , v
0
t , t)Δxt + ly(x0
t , y
0
t , u0
t , v
0
t , t)Δytdt] + ηt0,t1 ,
6 CH. A. AGAYEVA AND J. J. ALLAHVERDIYEVA
where
ηt0,t1 = η2 + E
∫ t1
t0
{ ∫ 1
0
ψ∗
t (gx(x0
t + μΔxt, yt, u
0
t , v
0
t , t) − gx(x0
t , yt, u
0
t , v
0
t , t))Δxtdμ+
+
∫ 1
0
ψ∗
t (gy(x0
t , y
0
t + μΔyt, u
0
t , v
0
t , t) − gy(x0
t , y
0
t , u
0
t , v
0
t , t))Δytdμ
}
+
+ E
∫ t1
t0
{∫ 1
0
β∗
t (σx(x0
t + μΔxt, yt, t) − σx(x0
t , y
0
t , t))Δxtdμ+
+
∫ 1
0
β∗
t (σy(x0
t , y
0
t + μΔyt, t) − σy(x0
t , y
0
t , t))Δytdμ
}
dt.
Using simple transformations and taking (8) into consideration, expression (13) takes the
form
ΔJ(u0) = −E
∫ t1
t0
[ψ∗
t Δug(x0
t , y
0
t , u0
t , v
0
t , t) − Δul(x0
t , y
0
t , u0
t , v
0
t , t)]dt−
− E
∫ t1
t0
[ψ∗
t Δvg(x0
t , y
0
t , u0
t , v
0
t , t) − Δvl(x0
t , y
0
t , u0
t , v
0
t , t)]dt + ηt0,t1 .(14)
Let’s consider the following spike variation:
Δut = Δuθ
t,ε =
{
0, t∈[θ, θ + ε), ε > 0, θ ∈ [t0, t1)
ũ − u0
t , t ∈ [θ, θ + ε), ũ ∈ L2(Ω, F θ, P ; Rm).
Then (14) takes the form
ΔθJ(u0) = −E
∫ θ+ε
θ
[ψ∗
t Δug(x0
t , y
0
t , u
0
t , v
0
t , t) + ψ∗
t Δvg(x0
t , y
0
t , u0
t , v
0
t , t)−
− Δul(x0
t , y
0
t , u0
t , v
0
t , t) − Δvl(x0
t , y
0
t , u0
t , v
0
t , t)]dt + ηθ,θ+ε.
(15)
We will use the following lemma.
Lemma 1. Let conditions I-III be satisfied. Then
E|xθ
t,ε − x0
t |2 ≤ Nε2, if ε → 0,
where xθ
t,ε is the trajectory corresponding to the control uθ
t,ε = uθ
t + Δuθ
t,ε.
Proof. Let’s designate
x̃t,ε =
xθ
t,ε − x0
t
ε
, ỹt,ε = x̃t−h(t),ε =
xθ
t−h(t),ε − x0
t−h(t)
ε
.
It is clear that ∀ t ∈ [t0, θ) x̃t,ε = 0. Then, for ∀ t ∈ [θ, θ + ε),
dx̃t,ε =
1
ε
[g(x0
t + εx̃t,ε, y
0
t + εỹt,ε, ũ, v0
t , t) − g(x0
t , y
0
t , u0
t , v
0
t , t)]dt+
+
1
ε
[σ(x0
t + εx̃t,ε, y
0
t + εỹt,ε, t) − σ(x0
t , y
0
t , t)]dwt, t ∈ (θ, θ + ε)
x̃θ,ε = −(g(x0
θ, y
0
θ , ũ, v0
θ , θ) − g(x0
θ, y
0
θ , u0
θ, v
0
θ , θ)).
ON ONE STOCHASTIC OPTIMAL CONTROL PROBLEM WITH VARIABLE DELAY 7
Therefore, conditions I-II and the Gronwall inequality yield
E|x̃θ+ε,ε|2 ≤ N
[
E sup
θ≤t≤θ+ε
|xθ
t,ε − x0
t |2 + E sup
θ≤t≤θ+ε
|x0
t − x0
θ|2 + E sup
θ≤t≤θ+ε
|yθ
t,ε − y0
t |2+
+ E sup
θ≤t≤θ+ε
|y0
t − y0
θ |2 + sup
θ≤t≤θ+ε
E|g(x0
t , y
0
t , ũ, v0
t , t) − g(x0
θ, y
0
θ , ũ, v0
θ , θ)|2+
+
1
ε
ER
∫ θ+ε
θ
|g(x0
t , y
0
t , u0
t , v
0
t , t) − g(x0
θ, y
0
θ , u0
θ, v
0
θ , θ)|2dt
]
.
Hence: E|x̃t+ε,ε|2 ≤ N, ε → 0, ∀ t ∈ [θ, θ + ε). In the same way for ∀ t ∈ [θ + ε, t1], we
have
dx̃t,ε =
1
ε
[g(x0
t + εx̃t,ε, y
0
t + εỹt,ε, u
0
t , ũ, t) − g(x0
t , y
0
t , u0
t , v
0
t , t)]dt+
+
1
ε
[σ(x0
t + εx̃t,ε, y
0
t + εỹt,ε, t) − σ(x0
t , y
0
t , t)]dwt.
Whence we have E|x̃t,ε|2 ≤ N, for ∀ t ∈ [θ + ε, t1], if ε → 0. Thus, sup
t0≤t≤t1
E|x̃t,ε|2 ≤ N.
Lemma 1 is proved.
According to Lemma 1 and from expression for ηt0,t1 , we obtain ηθ,θ+ε = o(ε).
Then it follows from (15) that
ΔθJ(u0) = −E[ψ∗
θΔug(x0
θ, y
0
θ , x0
θ, u
0
θ, v
0
θ , θ) − Δul(x0
θ, y
0
θ , x0
θ, u
0
θ, v
0
θ , θ)+
+ [ψ∗
zΔvg(x0
z , y
0
z , x0
z, u
0
z, v
0
z , θ) − Δvl(x0
z, y
0
z , x0
z , u
0
z, v
0
z , z]|z=r(θ)r
′(θ)]ε + o(ε) ≥ 0.
Hence, due to the sufficient smallness of ε, relation (9) is fulfilled. Theorem 1 is proved.
Problem with constraint
Using the obtained result and the variation principle of Ekeland [6], we will prove the
following theorem for a stochastic optimal control problem with the endpoint constraint
(7).
Theorem 2. Let conditions I-IV hold, and let (x0
t , u
0
t ) be a solution of problem (1)–(7).
Let there exist the random processes (ψt, βt) ∈ L2
F (t0, t1; Rn) × L2
F (t0, t1; Rn×n) which
are solutions of the adjoint system
(16)
⎧⎪⎪⎪⎨⎪⎪⎪⎩
dψt = −[Hx(ψt, x
0
t , y
0
t , u
0
t , v
0
t , t) + Hy(ψz , x
0
z, y
0
z , u0
z, v
0
z , z)|z=s(t)s
′(t)]dt+
+βtdwt, t0 ≤ t < t1 − h(t1),
dψt = −Hx(ψt, x
0
t , y
0
t , u0
t , v
0
t , t)dt + βtdwt, t1 − h(t1) ≤ t < t1,
ψt1 = −λ0px(x0
t1) − λ1qx(x0
t1),
where (λ0, λ1) ∈ Rk+1, λ0 ≥ 0, λ1 is the normal to the set G at the point Eq(x0
t1 ), and λ2
0+
|λ1|2 = 1. Then, ∀ ũ ∈ U a.c., the following relations hold:
(17)
⎧⎪⎪⎪⎨⎪⎪⎪⎩
H(ψt, x
0
t , y
0
t , u, v0
t , t) − H(ψt, x
0
t , y
0
t , u0
t , v
0
t , t)+
+[H(ψz, x
0
z , y
0
z , u
0
z, u, z)− H(ψz , x
0
z, y
0
z , u0
z, v
0
z , z)]|z=r(t)r
′(t) ≤ 0,
a.e. t ∈ [t0, t1 − h1(t1)),
H(ψt, x
0
t , y
0
t , u, v0
t , t) − H(ψt, x
0
t , y
0
t , u0
t , v
0
t , t) ≤ 0, t ∈ [t1 − h1(t1), t1], a.e.
8 CH. A. AGAYEVA AND J. J. ALLAHVERDIYEVA
Proof. For any natural j, we introduce the approximating functional
Jj(u) = Sj(Ep(xt1) + E
∫ t1
t0
l(xt, yt, ut, vt, t)dt, Eq(xt1)) =
= min
(c,y)∈ε
√∣∣∣∣c − 1/j − Ep(xt1 ) − E
∫ t1
t0
l(xt, yt, ut, vt, t)dt
∣∣∣∣2 + ‖y − Eq(xt1 )‖2,
E = {(c, y) : c ≤ J0, y ∈ G}, where J0 is the minimal value of the functional in (1)-(7).
By V ≡ (U∂ , d), we denote the space of controls obtained by means of introducing the
metric
d(u, v) = (l ⊗ P ){(t, ω) ∈ [t0, t1] × Ω : vt �= ut},
so that V is a complete metric space. In what follows, we need the following lemma.
Lemma 2. We assume that conditions I-IV hold, un
t – a sequence of admissible controls
from V, xn
t – a sequence of the corresponding trajectories of system (1)-(3). If d(un
t , ut) →
0, n → ∞, then lim
n→∞
{
sup
t0≤t≤t1
E|xn
t − xt|2
}
= 0, where xt is a trajectory corresponding
to an admissible control ut.
Proof. Let un
t be a sequence of admissible controls from V, and let xn
t be a sequence of
the corresponding trajectories. Then, for any t ∈ (t0; t1], we have
|xn
t − xt| =
=
∣∣∣∣ ∫ t
t0
[g(xn
s , yn
s , un
s , vn
s , s) − g(xs, ys, us, vs, s)]ds +
∫ t
t0
[σ(xn
s , yn
s , s) − σ(xs, ys, s)]dws
∣∣∣∣.
Let’s square and take expectation of both sides of the last expression. Due to assumption
II, we have
E|xn
t − xt|2 ≤
≤ NE
∫ t
t0
|Δung(xs, ys, us, vs, s)|2ds + NE
∫ t
t0
|xn
t − xt|2dt + N
∫ t
t0
E|yn
t − yt|2dt.
Hence, condition I and the Gronwall inequality yield
E|xn
t − xt|2 ≤ C exp(C(t − t0)),
where C = NE
∫ t
t0
|Δung(xs, ys, us, vs, s)|2ds. Lemma 2 is proved.
Due to continuity of the functional Jj : V → Rn, according to the variation principle
of Ekeland, we have that there exists a control uj
t : d(uj
t , u
0
t ) ≤ √
εj and, ∀ u ∈ V , the
following inequality holds: Jj(uj) ≤ Jj(u) + √
εjd(uj , u), εj = 1
j .
This inequality means that (xj
t , u
j
t) is a solution of the following problem:
(18)
⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩
Ij(u) = Jj(u) + √
εjE
∫ t1
t0
δ(ut, u
j
t)dt → min
dxt = g(xt, yt, ut, vt, t)dt + σ(xt, yt, t)dwt, t ∈ (t0, t1]
xt = Φ(t), t ∈ [t0 − h(t0), t0]
ut = Q(t), t ∈ [t0 − h1(t0), t0]
ut ∈ U∂ .
The function δ(u, v) is determined in the following way: δ(u, v) =
{
0, u = v
1, u �= v.
ON ONE STOCHASTIC OPTIMAL CONTROL PROBLEM WITH VARIABLE DELAY 9
Let (xj
t , u
j
t ) be a solution of problem (18). If there exist the random processes ψj
t ∈
L2
F (0, t1; Rn), βj
t ∈ L2
F (t0, t1; Rn×n), which are solutions of the system
(19)
⎧⎪⎪⎪⎪⎨⎪⎪⎪⎪⎩
dψj
t = −[Hx(ψj
t , x
j
t , y
j
t , u
j
t , v
j
t , t) + Hy(ψj
z , x
j
z, y
j
z, u
j
z, v
j
z, z)|z=s(t)s
′(t)]dt+
+βj
t dwt, t0 ≤ t ≤ t1 − h(t1)
dψj
t = −Hx(ψj
t , x
j
t , y
j
t , u
j
t , v
j
t , t)dt + βj
t dwt, t1 − h(t1) ≤ t < t1
ψj
t1 = −λj
0px(xj
t1) − λj
1qx(xj
t1 ),
where the non-zero (λj
0, λ
j
1) ∈ Rk+1 meet the requirement
(20) (λj
0, λ
j
1) = (−cj + 1/j + Ep(xj
t1) + E
∫ t1
t0
l(xj
t , y
j
t , u
j
t , v
j
t , t)dt,−yj + Eq(xj
t1 ))/I0
j ,
then, according to Theorem 1,
(21)
⎧⎪⎨⎪⎩
H(ψj
t , x
j
t , y
j
t , u, vj
t , t) − H(ψj
t , x
j
t , y
j
t , u
j
t , v
j
t , t) + [H(ψj
z , x
j
z, y
j
z, u
j
z, u, z)−
−H(ψj
z, x
j
z , y
j
z, u
j
z, v
j
z , z)]|z=r(t)r
′(t) ≤ 0, a.c., a.e. t ∈ [t0, t1 − h1(t1)),
H(ψj
t , x
j
t , y
j
t , u, vj
t , t) − H(ψj
t , x
j
t , y
j
t , u
j
t , v
j
t , t) ≤ 0, a.c, a.e. t ∈ [t1 − h1(t1), t1].
Here,
I0
j =
√∣∣∣∣cj − 1/j − Ep(xj
t1 ) − E
∫ t1
t0
l(xj
t , y
j
t , u
j
t , v
j
t , t)dt
∣∣∣∣2 + |yj − Eq(xj
t1)|2.
Since ‖(λj
0, λ
j
1)‖ = 1, we can think that (λj
0, λ
j
1) → (λ0, λ1).
It is known that Sj is a convex function which is Gateaux-differentiable at a point:
(Ep(xj
t1) + E
∫ t1
t0
l(xj
t , y
j
t , u
j
t , v
j
t , t)dt, Eq(xj
t1 ). Then, for all (c, y) ∈ E ,(
λj
0, c −
1
j
− Ep(xj
t1) − E
∫ t1
t0
l(xj
t , y
j
t , u
j
t , v
j
t , t)dt
)
+ (λj
1, y − Eq(xj
t1 )) ≤
1
j
.
Proceeding to the limit in the last inequality, we get that λ0 ≥ 0 and λ1 is a normal
to the set G at Eq(x0
t1 ). Since
(22) ψj
t1 = −λj
0px(xj
t1) − λj
1qx(xj
t1 ), we have ψj
t1 → ψt1 in L2
F (t0, t1; Rn).
Lemma 3. Let ψj
t be a solution of system (19), and let ψt be a solution of system (16).
Then
E
∫ t1
t0
|ψj
t − ψt|2dt + E
∫ t1
t0
|βj
t − βt|2dt → 0, if d(uj
t , ut) → 0, j → ∞.
Proof. According to Ito formula ∀ s ∈ [t1 − h(t), t1],
E|ψj
t1 − ψt1 |2 − E|ψj
s − ψs|2 =
= 2E
∫ t1
s
[ψj
t − ψt][(g∗x(xj
t , y
j
t , u
j
t , v
j
t , t) − g∗x(x0
t , y
0
t , u0
t , v
0
t , t))ψj
t +
+ g∗x(x0
t , y
0
t , u0
t , v
0
t , t)(ψj
t − ψt) + (σ∗
x(xj
t , y
j
t , t) − σ∗
x(x0
t , y
0
t , t)×
× (βj
t − βt) − lx(xj
t , y
j
t , u
j
t , v
j
t , t) + lx(x0
t , y
0
t , u
0
t , v
0
t , t)]dt + E
∫ t1
s
|βj
t − βt|2dt.
Due to assumptions I-II and using simple transformations, we obtain
E
∫ t1
s
|βj
t − βt|2dt + E|ψj
t − ψt|2dt + ENε
∫ t1
s
|βj
t − βt|2dt + E|ψj
t1 − ψt1 |2.
10 CH. A. AGAYEVA AND J. J. ALLAHVERDIYEVA
Hence, according to the Gronwall inequality, we have
(23) E|ψj
s − ψs|2 ≤ DeN(t1−s) a.e. in [t1 − h(t), t1],
where the constant D is determined in the following way: D = E|ψj
t1 − ψt1 |2. According
to Ito formula ∀ s ∈ [t0, t1 − h(t1)),
E|ψj
t1−h(t1)
− ψt1−h(t1)|2 − E|ψj
s − ψs|2 = 2E
∫ t1−h(t1)
s
(ψj
t − ψt)[(g∗x(xj
t , y
j
t , u
j
t , v
j
t , t)−
− g∗x(x0
t , y
0
t , u0
t , v
0
t , t))ψj
t + g∗x(x0
t , y
0
t , u0
t , v
0
t , t)(ψj
t − ψt) + (σ∗
x(xj
t , y
j
t , t)−
− σ∗
x(x0
t , y
0
t , t))β
j
t + σ∗
x(x0
t , y
0
t , t)(βj
t − βt) + (g∗y(xj
z , y
j
z, u
j
z, v
j
z, z)−
− g∗y(x0
z , y
0
z , u0
z, v
0
z , z))ψj
zs
′(t) + g∗y(x0
z , y
0
z , u0
z, v
0
z , z)(ψj
z − ψz)s′(t)σ∗
y(xj
z , y
j
z, z)−
− σ∗
y(x0
z , y
0
z , z))βj
zs
′(t) + σ∗
y(x0
z , y
0
z , z)(βj
z − βz)s′(t) + lx(x0
t , y
0
t , u0
t , v
0
t , t)−
− lx(xj
t , y
j
t , u
j
t , v
j
t , t) + ly(x0
t , y
0
t , u0
t , v
0
t , t)−
− ly(xj
t , y
j
t , u
j
t , v
j
t , t)]dt + E
∫ t1−h(t1)
s
|βj
t − βt|2dt.
In view of assumptions I-II and expression (22), we obtain
E
∫ t1−h(t1)
s
|βj
t − βt|2dt + E|ψj
s − ψs|2 ≤
≤ EN
∫ t1−h(t1)
s
|ψj
t − ψt|2dt + ENε
∫ t1−h(t1)
s
|ψj
z − ψz|2dt+
+ ENε
∫ t1−h(t1)
s
|βj
t − βt|2dt + E|ψj
t1−h(t1)
− ψt1−h(t1)|2.
Hence, using simple transformations, we have
E(1 − 2Nε)
∫ t1−h(t1)
s
|βj
t − βt|2dt + E|ψj
s − ψs|2 ≤ E(N + Nε)
∫ t1−h(t1)
s
|ψj
t − ψt|2dt+
ENε
∫ t1
t1−h(t1)
|ψj
t − ψt|2dt + ENε
∫ t1
t1−h(t1)
|βj
t − βt|2dt + E|ψj
t1−h(t1)
− ψt1−h(t1)|2.
According to the Gronwall inequality,
E|ψj
s − ψs|2 ≤ D exp[−(N + Nε)(t1 − h(t1) − s)], a.e. in [t0, t1 − h(t1)),
where
D = E|ψj
t1−h(t1)
− ψt1−h(t1)|2 + ENε
∫ t1
t1−h(t1)
|ψj
t − ψt|2dt + ENε
∫ t1
t1−h(t1)
|βj
t − βt|2dt.
Due to sufficient smallness of ε and from inequality (23), we get D → 0. Thus, ψj
t − ψt
L2
F (t0, t1; Rn), βj
t → βt L2
F (t0, t1; Rn×n). Lemma 3 is proved.
It follows from Lemma 3 and assumptions I-III that we can proceed to the limit in
systems (19), (21) and get the fulfillment of (16) and (17). Theorem 2 is proved.
Corollary. In the case where g ≡ g(xt, yt, ut, t) and l ≡ l(xt, ut, t) we obtain the result
proved in [4].
ON ONE STOCHASTIC OPTIMAL CONTROL PROBLEM WITH VARIABLE DELAY 11
Bibliography
1. Kolmanovskii V.B., Myshkis A.D., Applied Theory of Functional Differential Equations, Klu-
wer, N.Y., 1992.
2. Tsarkov Ye.F., Random Perturbations of Differential-Functional Equations, Riga, 1989. (in
Russian)
3. Chernousko F.L., Kolmanovsky V.B., Optimal Control under Random Perturbations, Nauka,
Moscow, 1978. (in Russian)
4. Agayeva Ch.A., Allahverdiyeva J.J., Maximum principle for stochastic systems with variable
delay, Reports of NSA of Azerbaijan LIX (2003), no. 5-6, 61-65. (in Russian)
5. Agayeva Ch. A., A necessary condition for one stochastic optimal control problem with constant
delay on control and state, Transactions of NSA of Azerbaijan, Math. and Mech. Series, Baku
XXVI (2006), no. 1, 3-14.
6. Ekeland I., Nonconvex minimization problem, Bull. Amer. Math. Soc.,(NS) 1 (1979), 443-474.
E-mail : cher.agayeva@rambler.ru, agayeva.cherkez@yasar.edu.tr
|
| id | nasplib_isofts_kiev_ua-123456789-4501 |
| institution | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| issn | 0321-3900 |
| language | English |
| last_indexed | 2025-12-07T17:40:47Z |
| publishDate | 2007 |
| publisher | Інститут математики НАН України |
| record_format | dspace |
| spelling | Agayeva, Ch.A. Allahverdiyeva, J.J. 2009-11-19T13:56:32Z 2009-11-19T13:56:32Z 2007 On one stochastic optimal control problem with variable delay / Ch.A. Agayeva, J.J. Allahverdiyeva // Theory of Stochastic Processes. — 2007. — Т. 13 (29), № 3. — С. 3–11. — Бібліогр.: 6 назв.— англ. 0321-3900 https://nasplib.isofts.kiev.ua/handle/123456789/4501 519.21 The purpose of this paper is to give necessary conditions for the optimality of nonlinear stochastic control systems with variable delay and with constraint on the right end of a trajectory. The necessary optimality conditions in the form of a stochastic analogy of the maximum principle are obtained. These conditions are contained in Theorems 1 and 2. en Інститут математики НАН України On one stochastic optimal control problem with variable delay Article published earlier |
| spellingShingle | On one stochastic optimal control problem with variable delay Agayeva, Ch.A. Allahverdiyeva, J.J. |
| title | On one stochastic optimal control problem with variable delay |
| title_full | On one stochastic optimal control problem with variable delay |
| title_fullStr | On one stochastic optimal control problem with variable delay |
| title_full_unstemmed | On one stochastic optimal control problem with variable delay |
| title_short | On one stochastic optimal control problem with variable delay |
| title_sort | on one stochastic optimal control problem with variable delay |
| url | https://nasplib.isofts.kiev.ua/handle/123456789/4501 |
| work_keys_str_mv | AT agayevacha ononestochasticoptimalcontrolproblemwithvariabledelay AT allahverdiyevajj ononestochasticoptimalcontrolproblemwithvariabledelay |