Homogeneous Markov chains in compact spaces
For homogeneous Markov chains in a compact and locally compact spaces, the ergodic properties are investigated, using the notions of topological recurrence and connections.
Збережено в:
| Дата: | 2007 |
|---|---|
| Автор: | |
| Формат: | Стаття |
| Мова: | Англійська |
| Опубліковано: |
Інститут математики НАН України
2007
|
| Онлайн доступ: | https://nasplib.isofts.kiev.ua/handle/123456789/4509 |
| Теги: |
Додати тег
Немає тегів, Будьте першим, хто поставить тег для цього запису!
|
| Назва журналу: | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| Цитувати: | Homogeneous Markov chains in compact spaces / A.V. Skorokhod // Theory of Stochastic Processes. — 2007. — Т. 13 (29), № 3. — С. 80–95. — Бібліогр.: 5 назв.— англ. |
Репозитарії
Digital Library of Periodicals of National Academy of Sciences of Ukraine| _version_ | 1859633502919589888 |
|---|---|
| author | Skorokhod, A.V. |
| author_facet | Skorokhod, A.V. |
| citation_txt | Homogeneous Markov chains in compact spaces / A.V. Skorokhod // Theory of Stochastic Processes. — 2007. — Т. 13 (29), № 3. — С. 80–95. — Бібліогр.: 5 назв.— англ. |
| collection | DSpace DC |
| description | For homogeneous Markov chains in a compact and locally compact spaces, the ergodic properties are investigated, using the notions of topological recurrence and connections.
|
| first_indexed | 2025-12-07T13:13:18Z |
| format | Article |
| fulltext |
Theory of Stochastic Processes
Vol. 13 (29), no. 3, 2007, pp. 80–95
UDC 519.21
ANATOLY V. SKOROKHOD
HOMOGENEOUS MARKOV CHAINS IN COMPACT SPACES
For homogeneous Markov chains in a compact and locally compact spaces, the er-
godic properties are investigated, using the notions of topological recurrence and
connections
1. Introduction
Let X be a metric compact space with a distance d(x, x′), x, x′ ∈ X . Denote, by
C(X), the set of continuous functions f : X → R. We use the same notation to the
Banach space C(X) with the norm
||f || = sup
x∈X
|f(x)|.
We investigate ergodic properties of a discrete time homogeneous Markov process {ξn, n ≥
0} in X with transition probability for one step P (x, B), x ∈ X, B ∈ B(X), where B(X)
is the Borelian σ -algebra in X . We assume that the transition probability satisfies the
Feller condition, this means that the function
Tf(x) =
∫
f(z)P (x, dz)
is a continuous linear operator C(X) → C(X).
The main result which will be used in our investigation of ergodic properties of the
Markov chains is the weak compactness of the set of all probability measures on B(X),
and the main tool is the topological recurrence and connections.
The application of the weak compactness to the investigation of the ergodicity of
dynamical systems in compact phase spaces was proposed by N.M. Krylov and N.N.
Bogolubov [1], who developed a method of construction of invariant measures which is
used here. In the work of the same authors [2], a variant of the ergodic theorem for a
Markov chain (which is treated as a random dynamic system) in the compact space is
proved.
The investigation of the ergodicity of Markov chains founded by Krylov and Bogolubov
was extended by M.V. Bebutov [3], who formulated and proved the main theorem on
the ergodicity of a Markov chain in the compact space, in which all ergodic invariant
measures were described. But the proof of the result is not so rigorous as one needs in
the modern considerations. In this article, we try to correct the proof using the notions
of topological recurrence and connections which were introduced by A.N. Kolmogorov in
[4] for countable Markov chains.
In this article, we will prove that any topologically recurrent state is represented as
a union of topologically connected subsets and obtain the representation of the ergodic
measure for each subset of this kind.
2000 AMS Mathematics Subject Classification. Primary 60J10.
Key words and phrases. Markov chain, topologically recurrent state, topologically connected states,
invariant measure.
80
HOMOGENEOUS MARKOV CHAINS IN COMPACT SPACES 81
We consider Markov chains in a locally compact space too. In this case, the set of
topologically recurrent states may be empty, so the Markov chain is transient. If the
set of topologically recurrent states is not empty, then it is represented as a union of
topologically connected subsets, and there exists an invariant measure on any topologi-
cally connected subset which is unique to a multiplicative constant (if the space is not
a compact, this measure may be infinite). In any way, the Birkhoff’s ratio theorem is
valid.
For a transient Markov chain {ξn, n ≥ 1}, we prove the existence of a finite limit
limEx
∑
k≤n
1{ξk∈B} = Q(x, B)
for any bounded B ∈ B.
2. Topological recurrence and connections
Definition. A point x ∈ X is said to be topologically recurrent to a Markov chain
{ξn, n ≥ 0} if, for any a > 0, the relation
Px(
∑
n
1{ξn∈Ba(x)} = ∞) = 1
is fulfilled. Here, Px is the conditional distribution of the discrete-time stochastic process
{ξn, n ≥ 0} under the condition ξ0 = x, Ba(x) is the ball in X of radius a with the
center at a point x.
Definition. Let x be a topologically recurrent state. It is topologically connected to a
state y ∈ X if the relation ∑
n
Px(ξn ∈ Ba(y)) > 0
is fulfilled for any a > 0.
Lemma 1. The set of topologically recurrent points is not empty.
Proof. If this set is empty, then, for any x ∈ X , there exists a(x) > 0 for which the
following relation is fulfilled:
Px(
∑
n
1{ξn∈Bo
a(x)(x)} < ∞/{ξn, n ≥ 0} = 1.
Here, Bo
a(x) is the open ball of radius a with the center at a point x. This implies the
relation
X ⊂
⋃
x∈X
Bo
a(x)(x).
It follows from the compactness of the set X that there exists a finite sequence {xk, k ≤ l}
for which
X ⊂
⋃
k≤l
Bo
a(xk)(xk),
so ∑
n
1{ξn∈X} < ∞.
This is impossible. Lemma 1 is proved.
Introduce an auxiliary homogeneous Markov chain in the space X depending on a
parameter 0 < λ < 1
{xλ
k , k ≥ 0}
82 ANATOLY V. SKOROKHOD
with transition probability
Pλ(x, B) = (1 − λ)
∑
k>0
λk−1Px(ξk ∈ B).
It satisfies the Feller condition. The Markov chain {ξλ
k , k ≥ 0} may be represented by the
Markov chain {ξk, k ≥ 0}. For this, introduce the sequence of independent identically
distributed integer-valued random variables {θk, k ≥ 1} with the distribution
P (θk = m) = (1 − λ)λm−1, m ≥ 1,
and set
ζ0 = 0, ζn =
∑
k≤n
θk.
Then the stochastic processes {ξλ
n, n ≥ 0} and
{ξζn , n ≥ 0}
have the same distribution.
Remark 1. The relation
Px(lim
n
∑
k≤n f(ξλ
k )∑
k≤n f(ξk)
= 1 − λ) = 1
is fulfilled for all x ∈ X and f ∈ C(X). The proof follows from the equality
∑
k≤n
f(ξλ
k ) =
∑
k≤n
∑
j
1{ζk=j}f(ξj),
and the Blackwell’s renewal theorem which implies the relation
lim
k→∞
∑
n
P (ζn = k) =
1
Eθ1
.
Definition. A state x ∈ X is topologically regular if the following conditions are fulfilled:
1) The measure Pλ(a, ·) is not pure atomic, i.e. the relation
inf{Pλ(x, X \ Λ) : Λ ∈ (CS)} > 0
is valid, where (CS) is the set of countable subsets of X,
2) Denote, by S∗
x, the support of the measure Pλ(x, ·), then
Ba(x) ⊂ S∗
x
for some a > 0.
Lemma 2. Let x be a topologically recurrent state, and let a topologically regular state
y satisfy the relation
Pλ
x (
∑
k
1{ξλ
k
∈Ba(y)} = ∞) = 1
for all a > 0.
Then the state y is topologically connected to the state x, and the state y is topologically
recurrent.
Proof. Denote
Cx = {z ∈ X : Pλ(z, Ba(x)) > 0, a > 0}.
Let a > 0 satisfy the relation
Ba(y) ⊂ S∗
y .
HOMOGENEOUS MARKOV CHAINS IN COMPACT SPACES 83
The relation
Pλ(x, Ba(y)) > 0
implies the formula
Cx ∩ Ba = ∅
because, if it is wrong, the Markov process starting at the state x will not return in any
neighborhood of x after it visited the ball Ba(y). So
Pλ(y, Ba(x)) > 0
for all a > 0. Lemma 2 is proved.
Invariant sets
Definition. A set S ∈ B(X) is called an invariant set for the Markov chain {ξn, n ≥ 0}
if the relation
P (ξ1 ∈ S/ξ0 = x) = 1
holds for all x ∈ S.
Remark 2. Let S be an invariant set for the Markov chain {ξn, n ≥ 0}, and let Ŝ be a
closure of the set S.
Then the set Ŝ is an invariant to the same Markov chain.
The proof of this statement is based on the Feller property of the transition probability.
Theorem 1. Let {ξn, n ≥ 0} have the distribution Px where x is a topologically recurrent
and topologically regular state. Introduce the set
Sx =
⋂
n
Closure{ξk, k ≥ n}.
Then
1) Sx is a closed invariant non-random set.
2) Any y ∈ Sx is a topologically recurrent state. If y is a topologically regular state,
then x and y are topologically connected.
3) Sx is the minimal closed invariant set containing the state x.
Proof.
1) It is easy to see that the function
F (z, {ξn, n ≥ 0}) = 1{z∈Sx}
is an invariant function for the Markov chain {ξn, n ≥ 0} because of the relation
F (z, ξn, n ≥ 0}) = F (z, {ξn, n ≥ 1}).
It is known that any invariant function is a function of ξ0, so Sx is a non-random closed
set depending on x only. The last equation implies that the set Sx is an invariant set.
2) Note that the relation y ∈ Sx implies the relation
Px(
∑
k
1{ξk∈Ba(y)} = ∞) = 1
for all a > 0. This implies the topological recurrence of the state y and its topological
connection to the state x because of Lemma 2 and the relation of equality Sx = Sy which
follows from the relations
Sy ⊂ Sx, Sx ⊂ Sy.
3) Assume J ⊂ Sx, Sx \ J = ∅ is an invariant closed set. Lemma 1 implies that there
exists a recurrent state z ∈ J. Then
Sz = Sx, J ⊂ Sx \ Sz = ∅.
84 ANATOLY V. SKOROKHOD
The theorem is proved.
3. Invariant measures
Let m be a probability measure on B(X). It is an invariant measure for the transition
probability P (x, B) if the relation
(1)
∫
P (x, B)m(dx) = m(B)
is fulfilled for any B ∈ B(X).
Remark 3. A measure m is invariant iff the relation
(2)
∫
Tg(x)m(dx) =
∫
g(x)m(dx)
is fulfilled for all g ∈ C(X). This follows from the observation that formula (1) can be
rewritten as formula (2) with g = 1B.
Remark 4. Let m be an invariant probability measure. Denote, by Pm, the distribution
of the Markov process {ξk, k ≥ 0} if the distribution of ξ0 is the measure m. Denote
Sm = Closure{ξk, k ≥ 1}.
Then Sm is the closed invariant set which is the support of the measure m.
For any x ∈ X and n ∈ N+, where N+ is the set of all integer numbers, introduce
probability measures mn(x, dz) by the relations∫
f(z)mn(x, dz) =
1
n
∑
k<n
T kf(x), f ∈ C(X), T 0f(x) = f(x).
Theorem 2. Assume that all states of Sx are topologically regular.
Then, for any z ∈ Sx, there exists an invariant measure m(z, dy) satisfying the relation
(3)
∫
f(y)m(z, dy) = lim
n
∫
f(y)mn(z, dy), f ∈ C(X).
These measures have properties
a) for any invariant measure ρ on the set Sx, the relation∫
f(z)ρ(dz) =
∫
(
∫
f(y)m(z, dy))ρ(dz), f ∈ C(X)
is fulfilled,
b) for any z ∈ Sx, the measure m(z, ·) is ergodic.
Proof. Since the sequence of measures {mn(x, dz), n ∈ N+} is compact, there exists a
subsequence nl, nl → ∞ as l → ∞ and a probability measure m∗(z, dy) for which the
relation
lim
l→∞
∫
f(y)mnl
(z, dy) =
∫
f(y)m∗(z, dy)
holds for all f ∈ C(X). This implies the relation∫
Tf(y)m∗(z, dy) = lim
l→∞
∫
Tf(y)mnl
(z, dy).
The relation ∫
Tf(y)mn(z, dy) =
1
n
∑
1≤k≤n+1
T kf(z)
HOMOGENEOUS MARKOV CHAINS IN COMPACT SPACES 85
implies the inequality
sup
z∈X
|
∫
Tf(y)mn(z, dy) −
∫
f(y)mn(z, dy)| ≤ 2||f ||
n
,
from which we obtain the equality
∫
Tf(y)m∗(z, dy) =
∫
f(y)m∗(z, dy).
So m∗(z, dy) is an invariant measure for the transition probability P (z, B).
Consider the Markov chain {ξk, k ≥ 0} with ξ0 having the distribution m∗ = m∗(z, ·).
It is a stationary process with the invariant measure m∗. The Birkhoff ergodic theorem
implies that, for all f ∈ C, the relation
(4) Py(lim
1
n
∑
k≤n
f(ξk) = f(ξ0)) = 1
is fulfilled for almost all y with respect to the measure m∗ which is the distribution of
ξ0. In particular, we have the relation
lim
n→∞Em∗
1
n
∑
k≤n
f(ξk) =
∫
f(y)m∗(dy)
which implies formula (3).
Statement a) follows from the formula
∫
f(z)ρ(dz) =
∫
Tf(z)ρ(dz) =
∫ ∫
f(y)mn(z, dy)ρ(dz)
and formula (3).
To prove statement b), consider the set IM(Sx) of all probability invariant measures
on the set Sx. It is a weakly compact convex set in the space M(X) of all finite measures
on B(X). Denote, by EIM(Sx), the set of all extreme invariant measures, they are
ergodic. Any measure m ∈ IM(Sx) is a mixture of ergodic measures,
m =
∫
EIM(Sx)
ναm(dν),
where αm is a probability measure on the Borelian σ-algebra of the set EIM(Sx). This
representation is unique. So formula (3) implies the relation
EIM(Sx) = {m(z, ·), z ∈ Sx}.
4. Ergodicity
The Markov chain {ξn, n ≥ 0} is ergodic if the set IM(X) of all probability invariant
measures for the transition probability P (x, B) contains only one element.
Theorem 3. The Markov chain {ξn, n ≥ 0} is ergodic iff the Markov chain {ξλ
n , n ≥ 0}
with the transition probability Pλ(x, B) is.
Proof. It follows from the relation (IM)(X) = (IM)λ(X), the last being the set of
invariant probability measures for the transition probability Pλ. This follows from the
following statement.
86 ANATOLY V. SKOROKHOD
Lemma 3. If a probability measure m is the invariant measure for the transition prob-
ability Pλ for some λ0 ∈ (0, 1), then it is an invariant measure for the transition proba-
bility P .
Proof. Introduce operators in the space MB of bounded measurable functions f :→ R
with the norm
||f || = sup
x∈X
|f(x)|,
T f(x) =
∫
f(y)P (x, dy), Tλf(x) =
∫
f(y)Pλ(x, dy).
Then
Tλ = (1 − λ)Rλ = (1 − λ)(I − λT )−1.
A measure m is invariant for the transition probability Pλ if the relation∫
Tf(x)m(dx) =
∫
f(x)m(dx)
holds. The function Rλ is an analytic function of λ, |λ| < 1. The derivatives of this
function are represented by the formula
(5) Dn
λ =
dn
dλn
Rλ = (−1)nn!(Rλ)n.
This formula implies that the measure m is invariant for the operator Dλ0
n , i.e.∫
Dλ0
n f(x)m(dx) =
∫
f(x)mdx), f ∈ BM.
The Taylor’s formula implies the relation
(6) Rλ = Rλ0 +
∑
n>0
(n!)−1Dn
λ0
(λ − λ0)n.
So the measure m is invariant for all
Pλ, |λ| < 1.
It follows from the formula∫
f(x)m(dx) =
∫
Pλf(x)m(dx) =
∑
k>0
∫
T kf(x)m(dx)
that
∫
T kf(x)m(dx) =
∫
f(x)m(dx) for all k > 0, f ∈ MB. Lemma 3 is proved.
Assume that any state x ∈ X is topologically recurrent and topologically regular, and
any state x is topologically connected to all states y ∈ X.
Lemma 4. Let a function f ∈ C(X) satisfy the condition
sup{f(x) − f(y) : x ∈ X, y ∈ X} > 0.
Then the inequality
inf{Tf(x) : x ∈ X} > inf{f(x) : x ∈ X}
is valid.
Proof. It suffices to consider the case
f ≥ 0, inf{f(x) : x ∈ X} = 0.
The open set
{x : f(x) > 0}
HOMOGENEOUS MARKOV CHAINS IN COMPACT SPACES 87
is not the empty set so Tf(x) > 0 because all states are topologically connected. Com-
pactness of the set X implies the relation
inf{Tf(x) : x ∈ X} > 0.
Corollary 1. If a function f satisfies the condition of Lemma 3, then the relations
inf
x∈X
T n+1f(x) > inf
x∈X
T nf(x), n ∈ N+,
inf
x∈X
T n+1
λ f(x) > inf
x∈X
T n
λ f(x), n ∈ N+
are fulfilled for any λ ∈ (0, 1).
Denote
Mλ(f) = sup
n
inf
x∈X
T n
λ f(x).
Then Mλ(f) > 0 for all f ∈ C∗+(X) where C∗
+ = {f ∈ C+ :: sup{f(x) − f(x′) > 0}.
Introduce the condition
SUC
inf{M−(f) : f ∈ C∗
+(X), sup{
∫
f(x)m(dx) : m ∈ IM(X)} = 1} = δ > 0.
A Markov chain satisfying this condition is called strong uniform connected.
Theorem 4. The Markov chain {ξn, n ≥ 0} is ergodic iff condition SUC is fulfilled.
Proof. Assume that the Markov chain {ξn, n ≥ 0} is ergodic with ergodic probability
measure m. Then the relation
lim
n
Snf(x) =
∫
f(z)m(dz), f ∈ C(X), Snf(x) =
1
n
∑
1≤k≤n
T kf(x).
is fulfilled for almost all x ∈ X with respect to the measure m.
Prove the formula
(7) lim
n
sup
x∈X
|Snf(x) −
∫
f(z)m(dz)| = 0, f ∈ C(X), Snf(x) =
∑
1≤k≤n
T kf(x).
If formula (7) is not true for some function f , then there exist sequences
{zk ∈ X, k ≥ 1}, {nk ∈ N+, nk ↑ +∞}
satisfying the inequality
|Snk
f(zk) −
∫
f(z)m(dz)| > ρ > 0.
We can assume that, for any g ∈ C(X), there exists
lim
k
Snk
g(zk) =
∫
g(z)m∗(dz)
where m∗ is a probability measure. It is easy to check that m∗ is an invariant measure.
In addition,
|
∫
f(z)m∗(dz) −
∫
f(z)m(dz)| ≥ ρ,
so m∗ = m. This contradicts the ergodicity of the Markov chain.
Formula (7) implies condition SUC because of Remark 1.
Let condition SUC be fulfilled, and let mk, k = 1, 2 be two different probability ergodic
distributions for the Markov chain.
88 ANATOLY V. SKOROKHOD
Then m1 ⊥ m2 and, for any ρ > 0, there exist the closed sets Fk, k = 1, 2, satisfying
the conditions
F1 ∩ F2 = ∅, mk(Fk) > 1 − ρ, k = 1, 2.
Let a function
fρ : X → R+
satisfy the conditions
fρ ∈ C+(X), fρ(x) ≤ 1, fρ(x) = 0, x ∈ F2, eρ(x) = 1, x ∈ F1.
Then the inequalities∫
fρ(x)m1(dx) > 1 − ρ,
∫
fρ(x)m2(dx) < ρ
are valid. If
ρ =
δ
1 + δ
where δ is from condition SUC, then∫
(1 − ρ)−1fρ(x)m1(dx) > 1
so M−(fρ) > δ and ∫
Snfρ(x)m2(dx) > δ(1 − ρ) = ρ
for all sufficiently large n. On the other hand,∫
Snfρm2(dx) =
∫
fρ(x)m2(dx) < ρ.
We obtain a contradiction. Theorem 4 is proved.
5. Irregular Markov chains
A Markov chain is irregular if the transition probability of the chain does not satisfy
the condition of topological regularity. We consider some examples of this kind proposed
by Professor A.M. Kulik (Kyiv Institute of Mathematics).
Example 1. Let X = IN∪{∞} with d(x, y) = | 1x − 1
y | ( 1
∞ ≡ 0). Define the transition
probability P of the Markov chain in X by
P (x, A) = Q(x, A ∩ IN)1Ix∈IN + δ∞1Ix=∞,
where Q is a transition probability of some ergodic Markov chain in IN such that the
ergodic measure for this chain is supported by the whole IN. Then the following features
hold.
a) All states from the set X \ {∞} are topologically recurrent, topologically regular,
and topologically connected to one another,
b) The state ∞ is absorbing and is not topologically connected to any x ∈ X \ {∞},
while any x ∈ X \ {∞} is topologically connected to ∞,
c) There exist two ergodic distributions: one on the set X \ {∞} and another on the
singlet {∞}.
Example 2 Let X = {0, 1}∞, i.e. points of the set are represented in the form
x = {[x]k, k ≥ 1}, [x]k ∈ {0, 1},
and the distance is determined by the formula
d(x, y) =
∑
k≥1
2−k|[x]k − [y]k|.
HOMOGENEOUS MARKOV CHAINS IN COMPACT SPACES 89
Let the transition probability from a state x to a state y be given by the formula
P (x, y) =
⎧⎪⎨
⎪⎩
2
3 , [y]k = [x]k+1, k ≥ 1
1
6 , [y]k = [x]k−1, k ≥ 2, [y]1 = 0
1
6 , [y]k = [x]k−1, k ≥ 2, [y]1 = 1
.
For this Markov chain, all states are topologically recurrent and pairwise topologically
connected. For any x ∈ X , the set
Ix =
⋃
m∈IN
{y ∈ X :
∑
k
1I[x]k =[y]m+k
< +∞}
is a countable invariant set. If x is periodic (i.e., ∃m : [x]k = [x]k+m, k ∈ IN), then there
exists a unique invariant measure ρx on the set Ix. The peculiarity of this statement is
the fact: the closure of the set Ix (i.e., the set Sx) coincides with X for any x, while
there exists a wide variety of (mutually singular) ergodic measures on X.
Consider a general result related to irregular Markov chains.
Theorem 5. Consider a Markov chain in X with transition probability P (x, A), x ∈
X, A ∈ B(X) satisfying the conditions
1) the Feller condition,
2) for any x ∈ X, the support of the measure Pλ(x, ·) is the set Λx ∈ (CCS) where
(CCS) is the set of all compact countable subsets of X,
3) any x ∈ X is topologically recurrent.
Then, for any x ∈ X, the set Λx is a closed invariant set, and there exists a unique
ergodic measure ρx on the set Λx.
The proof follows from Theorem 2.
6. Markov chains in locally compact spaces
In this section, we denote, by X, a locally compact metric space with a distance
d(x, x′). Consider a Markov chain {ξn, n ≥ 0} in X. Assume that the transition proba-
bility P (x, B), x ∈ X, B ∈ B(X) of the chain satisfies the C0 Feller condition, i.e.
Tf(x) =
∫
f(z)P (x, dz) ∈ C0(X), f ∈ C0(X),
where
C0(X) = {f ∈ C(X) : lim
x→∞ f(x) = 0}
and C(X) is the Banach space of all continuous bounded fumctions f : X → R.
Introduce the C0-weak convergence of measures on a σ-algebra B(X), for which the
sequence of measures {mn, n ≥ 1} is convergent to a measure m if the relation
lim
n
∫
f(x)mn(dx) =
∫
f(x)m(dx)
is fulfilled for any f ∈ C0(X).
It is known that the set M(X) of all finite measures on B(X) is a locally compact set
with respect to the C0-weak convergence. That is, for any bounded sequence of measures
{mn, n ≥ 1}, there exists a C0-weakly convergent subsequence {mnk
.nk ∈ N , nk → ∞}.
This implies the existence of invariant measures for the Markov chain {ξn, n ≥ 1}.
Compactly imbedded Markov chains
Denote by Φ the set of the functions φ ∈ C0(X) such that the set Fφ = Closure{x ∈
X : φ(x) > 0} is compact.
90 ANATOLY V. SKOROKHOD
Lemma 5. Let φ ∈ Φ, 0 ≤ φ ≤ 1. Suppose that every x ∈ Fφ is topologically recurrent
and, for any y ∈ X, there exists x ∈ {φ > 0} that is topologically connected to y. Introduce
a function
Pφ(x, A) = Ex
∑
n≥1
∏
0≤k<n
(1 − φ(ξk))φ(ξn)1{ξn∈A}, x ∈ X, A ∈ B(X).
Then Pφ(x, A) is the transition probability of a homogeneous Markov process in X that
satisfies the Feller property on the compact set Fφ.
Proof. Note that
Pφ(x, X) = Ex
∑
n≥1
φ(ξn)
∏
k<n
(1 − φ(ξk)) = 1 + Ex
∏
k≥1
(1 − φ(ξk))
because of the formula
1 −
∏
k≥1
(1 − ak) =
∑
n≥1
an
∏
k<n
(1 − ak), 0 < ak < 1, k ≥ 1.
In addition, ∏
k
(1 − φ(ξk)) ≤ exp {−
∑
k
φ(ξk)} = 0,
because the recurrence condition imposed on the Markov chain {ξkk ≥ 0} implies the
relation
∑
k φ(ξk) = +∞ Py-a.s. for every y ∈ X.
So Pφ(x, X) = 1 for all x ∈ X , and Pφ is a transition probability.
To prove the Feller property of the transition probability Pφ use the formula
∫
f(y)Pφ(x, dy) = Ex
∑
n≥1
(
∏
k<n
(1 − φ(ξk)))φ(ξn)f(ξn).
Denote
Inf(x) = Ex(
∏
k<n
(1 − φ(ξk)))φ(ξn)f(ξn).
This function is continuous in x because of the Feller property of the transition probability
P (x, A). If f > 0, then Inf(x) > 0. The series
∫
f(y)Pφ(x, dy) =
∑
n≥1
Inf(x)
of non-negative functions on the compact set Fφ converges uniformly to a continuous
function.
Extended filtrations
Let {θk, k ≥ 0} be a sequence of independent random variables which uniformly dis-
tributed on the interval (0, 1) and are independent on the sequence {ξk, k ≥ 0}.
Denote, by F̂ , the σ-algebra generated by the random variables
{ξk, k ≤ n} ∪ {θk, k ≤ n}.
The filtration {F̂n, n ≥ 0} is an extended filtration for the Markov chain {ξk, k ≥ 0}.
HOMOGENEOUS MARKOV CHAINS IN COMPACT SPACES 91
Lemma 6. Consider a stopping time with respect to the extended filtration:
νφ = inf{k ∈ N : θk < φ(ξk)}.
Then the relation
P (ξ(νφ) ∈ B/ξ0 = x) = Pφ(x, B)
is valid.
The proof follows from the relations
P (ξ(νφ ∈ B/ξ0)
=
∑
n≥1
= Ex1{θ0≥φ(ξ0),···θn−1≥φ(ξn−1),θn<φ(ξn)}1{ξn∈B}
=
∑
n≥1
(
∏
k<n
(1 − φ(ξk))φ(ξn)1{ξn∈B}.
Theorem 6. Introduce a sequence of stopping times with respect to the filtration
{F̂n, n ≥ 0} :
νφ
0 = νφ, νφ
l = inf{k > νφ
l−1 : θk < φ(ξk)}, l > 0.
Then the sequence {ξφ = ξ(νφ
l ), l ≥ 0} is a homogeneous Markov chain with values in
the compact set Fφ and the transition probability function Pφ(x, A)
The proof follows from the strong Markov property of the Markov chain {ξk, k ≥ 0}
and Lemma 6.
Remark 5. Assume that, for every φ ∈ Φ, the Markov chain {ξφ
k , k ≥ 1} is ergodic with
ergodic measure mφ on the σ -algebra B(Fφ). Then the Markov chain {ξn, n ≥ 0} is
ergodic with ergodic measure m on the σ-algebra B(X) for which
∫
g(x)m(dx) =
∫
Sφ(g, x)mφ(dx)∫
Sφ(1, x)mφ(dx)
, φ ∈ Φ,
where
Sφ(g, x) = Ex
∑
k
g(ξk)
∏
j<k
(1 − φ(ξj))φ(ξk).
The proof follows from the relations∑
k<νφ
g(ξk)
νφ
=
∑
l<n Ul(g)∑
l<n Ul(1)
, Ul(g) =
∑
g(ξj)1{νφ
l−1<j≤νφ
l }
and the ratio ergodic theorem.
7. Transient Markov chains in a locally compact space
Definition. The Markov chain {ξn, n ≥ 0} in the locally compact space X is called
transient iff the condition
Px(
∑
k
1{ξk∈Ba(y)} < ∞) = 1
is fulfilled for all a > 0, x ∈ X, y ∈ X.
It is easy to check that, for the transient Markov chain {ξk, k ≥ 0}, the relation
Px(
∑
k
1{ξk∈C} < ∞) = 1
is fulfilled for any x ∈ X and a compact set C ⊂ X.
92 ANATOLY V. SKOROKHOD
Lemma 7. The relation
P (lim
n
d(x̄, ξn) = ∞) = 1
is fulfilled for any x̄ ∈ X.
Proof. Set
νr = Card{ξk : d(x̄, ξk)}, r = 1, 2, ·.
Then νr are F∞ measurable random variables for which the relations
P (ν1 ≤ ν2 ≤ · ≤ νm ≤ · < ∞) = 1,
d(x̄, ξk) > r, k > νr
are fulfilled. Lemma 7 is proved.
Theorem 7. Let C ⊂ X be a compact set. Then the relation
Ex
∑
k
1{ξk∈C} < ∞
is fulfilled for any x ∈ X.
Proof. Introduce the random variables
ρ(C) =
∑
k
1{ξk∈C}, ρn(C) =
∑
k≥n
1{ξk∈C},
and the stopping times
θ1 = min{k : ξk ∈ C}, θ∗1 = min{k > θ1 : ξk ∈ C},
θn = min{k > θ∗n−1 : xik ∈ C}, θ∗n = min{k > θn : ξk ∈ C}, n > 1.
Note that, for x ∈ C, the relations Px(θ1 = 0) = 1 and
Px(θ∗1 ≥ l) ≥ Exg(C, ξl)
are fulfilled, where l > 0, l ∈ N+ and
g(C, z) = d(z, C) ∧ 1.
Since
Exg(C, ξl), x ∈ C, l > 0, l ∈ N+
is a continuous function, so
inf
x∈C
Exg(C, ξl)) > 0
for some l > 0, l ∈ N+. The inequality
Px(θ∗1 ≥ l) ≥ α, x ∈ C
is fulfilled for some l ∈ N+ and α > 0. This implies the formula
(8) sup
x∈C
Exθ∗1 ≤ l
1 − α
.
It is easy to see that the conditional distribution of the random variable θ∗n − θn with
respect to the σ-algebra Fxθn
coincides with the conditional distribution of the random
variable θ1 with respect to the random variable ξ0 if ξ0 = ξθn . This follows from the
strong Markov property of the Markov chain {ξk, k ≥ 0}. So the inequality
(9) Ex(θ∗n − θn/Fθn) ≤ l
1 − α
is fulfilled for all n for which
Px(θn < ∞) > 0
HOMOGENEOUS MARKOV CHAINS IN COMPACT SPACES 93
and, in this case,
P (θ∗n < ∞/Fθn) = 1{θn<∞}.
Consider a sequence
ηn = E(φ(ρn+1(C))/Fn, n > 0,
where φ(t) = 1 − exp {−t}. The inequalities
ρn(C) ≥ ρn+1(C), φ(ρn(C)) ≥ φ(ρn+1(C))
imply the relation
E(ηn+1/Fn) ≤ ηn.
So the sequence {ηn, n ≥ 0} is a non-negative supermartingale, and the relation
(10) Px( sup
n≤N
ηn > a) ≤ ExηN
a
holds. Note that the sequence Exηn is decreasing to zero because of the relation
Px(ρn(C) → 0) = 1.
Let
a =
e − 1
e
,
and
N0 = min{k : Exηk ≤ a
2
}.
Then, for any l ∈ N+ for which θ∗l ≥ N0 and
Px(θ∗l < ∞) > 0,
the relation
P (θl+1 < ∞/Fθ∗
l
) ≤ 1
2
.
is fulfilled. Note that, for this l and k ∈ N+, k > 0, the relation
P (θl+k < ∞/Fθ∗
l
) ≤ 1
2
E(1{θl+k−1<∞}/Fθ∗
l
) ≤ 1
2k
.
is fulfilled too. Using the relation
∑
n
1{ξn∈C} =
∑
i≥0
1{θi<∞}
∑
k
1{θi≤k<θ∗
i }
we obtain the inequality
Exρ(C) ≤ 1
1 − α
Ex
∑
i≥0
Ex1{θi<∞} ≤ 1
1 − α
(N0 +
∑
k≥0
1
2k
) < ∞.
Theorem 7 is proved.
Corollary 2. A function
(11) Q(x, E) = Ex
∑
k
1{ξk∈E}
is determined for all x ∈ X and
E ∈
⋃
n∈N+
B(Bn),
94 ANATOLY V. SKOROKHOD
where
Bn = Bn(x̄), x̄
is a point in X.
Remark 6. Assume that the relation
(12)
∑
k
Px(ξk ∈ Ba(y)) > 0
is fulfilled for some y ∈ X and all a > 0. Then the formula
Q(y, E) = Q(x, E) − Px(ξ0 ∈ E) − Px(ξ1 ∈ E) + Py(ξ0 ∈ E) + Py(ξ1 ∈ E)
is valid. The proof follows from the proof of Lemma 2.
Subharmonic functions in Rd
Assume X = Rd. For a transient Markov chain {ξk, k ≥ 0}, consider the sequence of
random variables
ηn = min{|ξk| : k ≥ n}, n ≥ 0.
Theorem 8. Let formula (10) be fulfilled for all x ∈ Rd and y ∈ Rd. Then the sequence
{ηn, n ≥ 0} satisfies the conditions
(i) ηn+1 ≥ ηn for all n ≥ 0;
(ii) Px(supn ηn = ∞) = 1 for all x ∈ Rd;
(iii) there exists a non-negative concave function g : R+ → R+ for which Exg(ηn) <
∞;
(iv) set
h(x) = Exg(η0),
then {h(ξk), k ≥ 0} is a submartingale, so h(x) is a continuous submartingale function.
Proof. Statement (i) follows from the definition of ηn, statement (ii) follows from Theo-
rem 7.
To prove statement (iii), it suffices to prove that, for a subsequence {nk, k ≥ 1} with
nk ↑ ∞, there exists a function of the kind mentioned in (iii) for which Exg(ηnk
) < ∞
for all k ≥ 1.
The relation
lim
a
Px(|ηn| > a) = 0
for all n ∈ N+ implies the existence ofsequences {nk, k ≥ 1} and
{pk, k ≥ 1}, pk ≥ 0,
∑
k
pk = 1,
for which
Px(
∑
k
pkηnk
< ∞) = 1.
Denote
ζ =
∑
k
pkηk.
Then Px(0 ≤ ζ < ∞) = 1 and there exists a function g satisfying the conditions of
statement (iii) for which Exg(ζ) < ∞, so
∞ > Exg(ζ) ≥
∑
k
pkEgnk
.
To prove statement (iv), consider a sequence
ζn = E(g(ηn/Fn), n ≥ 0.
HOMOGENEOUS MARKOV CHAINS IN COMPACT SPACES 95
The inequality ηn+1 ≥ ηn implies the relation
E(ζn+1/Fn = E(g(ηn+1)/Fn) ≥ E(g(ηn)/Fn) = ζn
so the sequence {ζn, n ≥ 0} is a submartingale and
ζn = h(ξn)
because of the strong Markov property of the Markov chain {ξn, n ≥ 0}. Theorem 8 is
proved.
Bibliography
1. N.M. Krylov and N.N. Bogolubov, The general measure theory and its application to dynamical
systems of non-linear mechanics, Zapysky Kafedry Mat. Fiz. AN SSSR 3 (1937), 55-112.
2. N.M. Krylov and N.N. Bogolubov, On some problems of ergodic theory of stochastic systems,
Zapysky Kafedry Mat. Fiz. AN SSSR 4 (1939), 243-287.
3. M.V. Bebutov, Markov chains with a compact state space, Matem.Sb. 10(52) (1942), 213-238.
4. A.N. Kolmogorov, Denumerable Markov chains, Bull. MGU 1 (1937), no. 3, 1-16.
5. S. Orey, Recurrent Markov chains, Pacific J. Math. 9 (1959), 805-827.
|
| id | nasplib_isofts_kiev_ua-123456789-4509 |
| institution | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| issn | 0321-3900 |
| language | English |
| last_indexed | 2025-12-07T13:13:18Z |
| publishDate | 2007 |
| publisher | Інститут математики НАН України |
| record_format | dspace |
| spelling | Skorokhod, A.V. 2009-11-19T14:00:43Z 2009-11-19T14:00:43Z 2007 Homogeneous Markov chains in compact spaces / A.V. Skorokhod // Theory of Stochastic Processes. — 2007. — Т. 13 (29), № 3. — С. 80–95. — Бібліогр.: 5 назв.— англ. 0321-3900 https://nasplib.isofts.kiev.ua/handle/123456789/4509 519.21 For homogeneous Markov chains in a compact and locally compact spaces, the ergodic properties are investigated, using the notions of topological recurrence and connections. en Інститут математики НАН України Homogeneous Markov chains in compact spaces Article published earlier |
| spellingShingle | Homogeneous Markov chains in compact spaces Skorokhod, A.V. |
| title | Homogeneous Markov chains in compact spaces |
| title_full | Homogeneous Markov chains in compact spaces |
| title_fullStr | Homogeneous Markov chains in compact spaces |
| title_full_unstemmed | Homogeneous Markov chains in compact spaces |
| title_short | Homogeneous Markov chains in compact spaces |
| title_sort | homogeneous markov chains in compact spaces |
| url | https://nasplib.isofts.kiev.ua/handle/123456789/4509 |
| work_keys_str_mv | AT skorokhodav homogeneousmarkovchainsincompactspaces |