Matrix parameter estimation in an autoregression model
The vector difference equation ξk = Af(ξk−1)+εk, where (εk) is a square integrable difference martingale, is considered. A family of estimators ˇAn depending, besides the sample size n, on a bounded Lipschitz function is constructed. Convergence in distribution of √n (ˇAn − A) as n→∞is proved wit...
Gespeichert in:
| Datum: | 2006 |
|---|---|
| Hauptverfasser: | , |
| Format: | Artikel |
| Sprache: | English |
| Veröffentlicht: |
Інститут математики НАН України
2006
|
| Online Zugang: | https://nasplib.isofts.kiev.ua/handle/123456789/4450 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Назва журналу: | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| Zitieren: | Matrix parameter estimation in an autoregression model / A.P. Yurachkivsky, D.O. Ivanenko // Theory of Stochastic Processes. — 2006. — Т. 12 (28), № 1-2. — С. 154–161. — Бібліогр.: 4 назв.— англ. |
Institution
Digital Library of Periodicals of National Academy of Sciences of Ukraine| id |
nasplib_isofts_kiev_ua-123456789-4450 |
|---|---|
| record_format |
dspace |
| spelling |
nasplib_isofts_kiev_ua-123456789-44502025-02-09T23:11:11Z Matrix parameter estimation in an autoregression model Yurachkivsky, A.P. Ivanenko, D.O. The vector difference equation ξk = Af(ξk−1)+εk, where (εk) is a square integrable difference martingale, is considered. A family of estimators ˇAn depending, besides the sample size n, on a bounded Lipschitz function is constructed. Convergence in distribution of √n (ˇAn − A) as n→∞is proved with the use of stochastic calculus. Ergodicity and even stationarity of (εk) is not assumed, so the limiting distribution may be, as the example shows, other than normal. 2006 Article Matrix parameter estimation in an autoregression model / A.P. Yurachkivsky, D.O. Ivanenko // Theory of Stochastic Processes. — 2006. — Т. 12 (28), № 1-2. — С. 154–161. — Бібліогр.: 4 назв.— англ. 0321-3900 https://nasplib.isofts.kiev.ua/handle/123456789/4450 519.21 en application/pdf Інститут математики НАН України |
| institution |
Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| collection |
DSpace DC |
| language |
English |
| description |
The vector difference equation ξk = Af(ξk−1)+εk, where (εk) is a square integrable
difference martingale, is considered. A family of estimators ˇAn depending, besides
the sample size n, on a bounded Lipschitz function is constructed. Convergence in
distribution of √n (ˇAn − A) as n→∞is proved with the use of stochastic calculus.
Ergodicity and even stationarity of (εk) is not assumed, so the limiting distribution
may be, as the example shows, other than normal. |
| format |
Article |
| author |
Yurachkivsky, A.P. Ivanenko, D.O. |
| spellingShingle |
Yurachkivsky, A.P. Ivanenko, D.O. Matrix parameter estimation in an autoregression model |
| author_facet |
Yurachkivsky, A.P. Ivanenko, D.O. |
| author_sort |
Yurachkivsky, A.P. |
| title |
Matrix parameter estimation in an autoregression model |
| title_short |
Matrix parameter estimation in an autoregression model |
| title_full |
Matrix parameter estimation in an autoregression model |
| title_fullStr |
Matrix parameter estimation in an autoregression model |
| title_full_unstemmed |
Matrix parameter estimation in an autoregression model |
| title_sort |
matrix parameter estimation in an autoregression model |
| publisher |
Інститут математики НАН України |
| publishDate |
2006 |
| url |
https://nasplib.isofts.kiev.ua/handle/123456789/4450 |
| citation_txt |
Matrix parameter estimation in an autoregression model / A.P. Yurachkivsky, D.O. Ivanenko // Theory of Stochastic Processes. — 2006. — Т. 12 (28), № 1-2. — С. 154–161. — Бібліогр.: 4 назв.— англ. |
| work_keys_str_mv |
AT yurachkivskyap matrixparameterestimationinanautoregressionmodel AT ivanenkodo matrixparameterestimationinanautoregressionmodel |
| first_indexed |
2025-12-01T15:40:45Z |
| last_indexed |
2025-12-01T15:40:45Z |
| _version_ |
1850321033530179584 |
| fulltext |
Theory of Stochastic Processes
Vol. 12 (28), no. 1–2, 2006, pp. 154–161
UDC 519.21
A. P. YURACHKIVSKY AND D. O. IVANENKO
MATRIX PARAMETER ESTIMATION
IN AN AUTOREGRESSION MODEL
The vector difference equation ξk = Af(ξk−1)+ εk, where (εk) is a square integrable
difference martingale, is considered. A family of estimators Ǎn depending, besides
the sample size n, on a bounded Lipschitz function is constructed. Convergence in
distribution of
√
n Ǎn − A as n → ∞ is proved with the use of stochastic calculus.
Ergodicity and even stationarity of (εk) is not assumed, so the limiting distribution
may be, as the example shows, other than normal.
Introducton
We consider the vector autoregression process
(1) ξk = Af(ξk−1) + εk, k ∈ N.
Here, A is an unknown square matrix, f is a prescribed function, and (εk) is a square
integrable difference martingale with respect to some flow (Fk, k ∈ Z+) of σ-algebras
such that the random variable ξ0 is F0-measurable. In the detailed form, the assumption
about (εk) means that for any k εk is Fk-measurable,
(2) E|εk|2 < ∞
and
(3) E(εk|Fk−1) = 0.
All vectors are regarded, unless otherwise stated, as columns. Then a�b and ab�
signify scalar and tensor product respectively. The latter is otherwise denoted a ⊗ b
(this is a (0, 2)-tensor), in particular a⊗2 = aa�. We use the Euclidean norm of vectors,
denoting it |·|, and the operator norm of matrices. Other notation: B† – the pseudoinverse
to B; O – the null matrix; l.i.p. – limit in probability; d→ – the weak convergence of the
finite-dimensional distributions of random functions, in particular the convergence in
distribution of random vectors.
Let h be a vector function such that for some n
E (|ξn| + |Af(ξn)|) |h(ξn−1)| + E|h(ξn−1)| < ∞.
Then from (1) – (3) we have E (ξn − Af(ξn−1)) ⊗ h(ξn−1) = O , whence
A = (Eξn ⊗ h(ξn−1)) (Ef(ξn−1) ⊗ h(ξn−1))
−1
2000 AMS Mathematics Subject Classification. Primary 62F12; Secondary 60F05.
Key words and phrases. Autoregression, martingale, estimator, tensor, convergence.
154
MATRIX PARAMETER ESTIMATION 155
provided the inverse exists. This prompts the estimator
(4) Ǎn =
(
n∑
k=1
ξk ⊗ h(ξk−1)
) (
n∑
k=1
f(ξk−1) ⊗ h(ξk−1)
)†
coinciding in the case f(x) = x with the LSE.
The goal of the article is to study the asymptotic behaviour of the normalized deviation√
n
(
Ǎn − A
)
as n → ∞. The use of stochastic calculus underlying our approach allows
us to dispense with the assumptions of ergodicity and even asymptotic stationarity of
the sequence (εk), thereat the limiting distribution of the studied statistic may be other
than normal. This is the main distinction of our results from A.Ya. Dorogovtsev’s ones
[1] essentially based on the ergodicity assumption.
Preliminaries
Let E0 denote E(· · · |F0).
Lemma 1. Let conditions (2) and (3) be fulfilled and there exist a number q such that
for all x
(5) |Af(x)| ≤ q|x|.
Then, for any k,
E0|ξk|2 ≤ q2k|ξ0|2 +
k−1∑
i=0
qiE0|εk−i|2.
Proof. Writing, on the basis of (1),
(6) |ξk|2 = |Af(ξk−1)|2 + 2Af(ξk−1)�εk + |εk|2
we deduce our assertion from (2), (3) and (5) by induction.
Denote further σ2
k = E
(
ε⊗2
k |Fk−1
)
, χN
k = I{|ξk| > N}, IN
k = I{|εk| > (1 − q)N},
bN
k = E0|ξk|2χN
k . Obviously,
(7) E
(|εk|2|Fk−1
)
= trσ2
k.
Lemma 2. Let conditions (2), (3) and (5) be fulfilled and
(8) q < 1.
Then for any k
bN
k ≤ q2bN
k−1 + E0|εk|2χN
k−1 + 2(q/(1 − q))2N−2E0|ξk−1|2trσ2
k + 2E0|εk|2IN
k .
Proof. Due to (1) and (5),
χN
k ≤ χN
k−1 + IN
k ,
which together with (6), (5) and the obvious inequality |a�b| ≤ |a|2 + |b|2 yields
|ξk|2χN
k ≤ q2|ξk−1|2χN
k−1 + 2Af(ξk−1)�εkχN
k−1 + |εk|2χN
k−1 + 2(q2|ξk−1|2 + |εk|2)IN
k .
By Lemma 1 and condition (5), E0|Af(ξk−1)|2 < ∞. Hence, because of (2) and (3),
E0
(
Af(ξk−1)�εk|Fk−1
)
= 0. The equality
E0|ξk−1|2IN
k = E0
(|ξk−1|2P{|εk| > (1 − q)N |Fk−1}
)
,
together with condition (8), Chebyshev’s inequality, and equality (7) completes the proof.
In what follows, C is a generic constant.
Obviously, E0χN
i ≤ N−2bN
i . Hence and from the previous lemmas we deduce (the
details can be found in the proof of Theorem 2 [2])
156 A. P. YURACHKIVSKY AND D. O. IVANENKO
Corollary 1. Let conditions (2), (3), (5), and (8) be fulfilled,
(9) lim
N→∞
lim
n→∞
1
n
n∑
k=1
E|εk|2I{|εk| > N} = 0,
and let there exist an F0-measurable random variable υ such that for all k
(10) E
(|εk|2|Fk−1
) ≤ υ.
Then with probability 1
lim
N→∞
lim
n→∞
1
n
n∑
k=1
bN
k = 0.
The main results
Let h be a Borel function such that
(11) |h(x)| ≤ C|x|.
Denote ηk = h(ξk), Kn = 1√
n
∑n
k=1 εk ⊗ ηk−1, Qn = 1
n
∑n−1
k=0 f(ξk) ⊗ ηk,
Tn =
√
n(AQnQ†
n − A), Gn = 1
n
∑n
k=1 σ2
k ⊗ η⊗2
k−1,
(12) Yn(t) =
1√
n
[nt]∑
k=1
εk ⊗ ηk−1.
Then, because of (4),
(13)
√
n(Ǎn − A) = KnQ†
n + Tn.
By construction and conditions (2), (3) and (5), Yn is a locally square integrable
martingale with quadratic characteristic
(14) 〈Yn〉(t) = n−1[nt]G∗
[nt],
where ∗ is a linear operation in the space of 4-valent tensors such that
(a ⊗ b ⊗ c ⊗ d)∗ = a ⊗ c ⊗ b ⊗ d.
Theorem 1. Let conditions (2), (3), (5) and (8) – (11) be fulfilled, and let there exist
a random (0, 4)-tensor G such that
(15) Gn
d→G.
Then Yn
d→Y , where Y is a continuous local martingale with quadratic characteristic
〈Y 〉(t) = G∗t.
Proof. According to Corollary in [3] and in view of (14) and (15), it suffices to show that
for any t
(16) E sup
s≤t
‖Yn(s) − Yn(s−)‖2 → 0.
The argument in [3] doess not change if the expectation is taken conditioned on F0, so
in (16) E and → may be substituted by E0 and P→, respectively. This weakened version
of (16) is equivalent, because of (12), to the following relation:
n−1E0 max
k≤nt
ρk
P→0,
MATRIX PARAMETER ESTIMATION 157
where ρk = |εk|2|ηk−1|2. Since for any δ > 0
max
k
ρk ≤ δn +
∑
k
ρkI{ρk > δn},
it remains to prove that the random variables
√
ρk/n satisfy the Lindeberg condition:
for any δ > 0
(17)
1
n
∑
k≤nt
E0ρkI{ρk > δn} P→0.
Writing on the basis of (11)
ρkI{ρk > δn} (I{|ξk−1| ≤ N} + I{|ξk−1| > N})
≤ C2
(
N2|εk|2I{|εk|2 > (CN)−2δn} + |εk|2|ξk−1|2χN
k−1
)
,
we deduce (17) from both the conditions and the conclusion of Corollary 1.
Applying Theorem 1 to the compound processes (Yn, Qn) where the second component
does not depend on t, we obtain
Corollary 2. Let conditions (2), (3), (5), and (8) – (11) be fulfilled, and let there exist
given on a common probability space random (0, 4)-tensor G and (0, 2)-tensor Q such
that
(18) (Gn, Qn) d→(G, Q).
Then (Yn, Qn) d→(Y, Q), where Y is a continuous local martingale w. r. t. some flow
(F(t), t ∈ R+) such that 〈Y 〉(t) = G∗t and the tensor-valued r. v. Q is F(0)-measurable.
Theorem 2. Let the conditions of Corollary 2 be fulfilled and detQ �= 0 a. s. Then
(19)
√
n(Ǎn − A) d→Y (1)Q−1.
Proof. By Corollary 2,
(Yn(1), Qn) d→(Y (1), Q).
But Yn(1) = Kn, which together with the nondegeneracy of Q implies that
KnQ†
n
d→KQ−1. Now, to obtain the assertion of the theorem from (13), it remains to
note that
P{Tn �= O} ≤ P{detQn �= O} → 0.
Simpler versions of condition (18)
Denote f0(x) = x and, for r ≥ 1,
(20) fr(x0, . . . , xr) = Af(fr−1(x0, . . . , xr−1)) + xr.
Then
(21) ξk = fr(ξk−r , εk−r+1, . . . , εk), r < k,
and
(22) |fr(x0, . . . , xr)| ≤
r∑
i=0
qi|xr−i|.
Below Xr stands for (x1, . . . , xr), and d is the dimensionality of each xj .
158 A. P. YURACHKIVSKY AND D. O. IVANENKO
Lemma 3. Let for all x, y
(23) |Af(x) − Af(y)| ≤ q|x − y|.
Then for all x, y, r, Xr
|fr(x, Xr) − fr(y, Xr)| ≤ qr|x − y|.
Proof. Due to (20) and (23),
|fr(x, Xr) − fr(y, Xr)| ≤ q|fr−1(x, Xr−1) − fr−1(y, Xr−1)|,
so it remains to apply the induction.
Corollary 3. Under the conditions of Lemma 3, for any N
lim
r→∞ sup
|x|≤N,Xr∈Rrd
|fr(x, Xr) − fr(0, Xr)| = 0.
Corollary 4. Let conditions (5), (8), (11) and (23) be fulfilled and for any x, y
(24) |h(x) − h(y)| ≤ C|x − y|.
Then for any N > 0
(25) lim
r→∞ sup
|x|≤N,Xr∈Rrd
‖f(fr(x, Xr)) ⊗ h(fr(x, Xr)) − f(fr(0, Xr))⊗ h(fr(0, Xr))‖ = 0,
(26) lim
r→∞ sup
|x|≤N,Xr∈Rrd
‖h(fr(x, Xr))⊗2 − h(fr(0, Xr))⊗2‖ = 0.
Denote further ξr
k = fr(0, εk−r+1, . . . , εk), ηr
k = h(ξr
k), Qr
n = 1
n
∑n−1
k=r f(ξr
k) ⊗ ηr
k,
Gr
n = 1
n
∑n
k=r σ2
k ⊗ (ηr
k−1)
⊗2. We endow the space of (0, 4)-tensors with such a norm
that for any (0, 2)-tensors A1 and A2, ‖A1 ⊗ A2‖ = ‖A1‖‖A2‖.
Lemma 4. Let conditions (2), (3), (5), (8) – (11), (23) and (24) be fulfilled and
(27) |f(x)| ≤ C|x|.
Then almost surely
(28) lim
r→∞ lim
n→∞E0‖Qn − Qr
n‖ = 0,
(29) lim
r→∞ lim
n→∞E0‖Gn − Gr
n‖ = 0.
Proof. By Corollary 4 for any N > 0,
(30) lim
r→∞ lim
n→∞
1
n
n−1∑
k=r
E‖f(ξk) ⊗ ηk − f(ξr
k) ⊗ ηr
k‖I{|ξk| ≤ N} = 0.
Due to (11) and (27),
E0‖f(ξk) ⊗ ηk‖χN
k ≤ C2bN
k ,
so, by Corollary 1,
(31) lim
N→∞
lim
n→∞
1
n
n−1∑
k=0
E0‖f(ξk) ⊗ ηk‖χN
k = 0.
Further, for k ≥ r
E0‖f(ξr
k) ⊗ ηr
k‖ = E0|f(fr(0, εk−r+1, . . . , εk))||h(fr(0, εk−r+1, . . . , εk))|,
MATRIX PARAMETER ESTIMATION 159
whence, in view of (22), (27), and (11),
(32) E‖f(ξr
k) ⊗ ηr
k‖χN
k ≤ C2E
(
r−1∑
i=0
qi|εk−i|
)2
χN
k .
Writing the Cauchy–Buniakowsky inequality
(
r−1∑
i=0
qi|εk−i|
)2
≤
r−1∑
j=0
qj
r−1∑
i=0
qi|εk−i|2,
we get for an arbitrary L > 0
E
(
r−1∑
i=0
qi|εk−i|
)2
χN
k
≤ (1 − q)−1
(
E
r−1∑
i=0
qi|εk−i|2I{|εk−i| > L} + L2P{|ξk| > N}
r−1∑
i=0
qi
)
.
(33)
Lemma 1 together with (8) and (10) implies that
(34) lim
N→∞
lim
n→∞
1
n
n∑
k=0
P{|ξk| > N} = 0.
Obviously, for arbitrary nonnegative numbers u0, . . . , ur−1, v1, . . . , vn−1,
n−1∑
k=r
r−1∑
i=0
uivk−i ≤
r−1∑
i=0
ui
n−1∑
j=1
vj ,
so conditions (8) and (9) imply that
lim
L→∞
sup
r
lim
n→∞
1
n
n−1∑
k=r
E
r−1∑
i=0
qi|εk−i|2I{|εk−i| > L} = 0,
whence, in view of (32) – (34),
lim
N→∞
sup
r
lim
n→∞
1
n
n−1∑
k=r
E‖f(ξr
k) ⊗ ηr
k‖χN
k = 0.
Combining this with (30) and (31), we arrive at (28).
The proof of (29) is similar.
Corollary 5. Let the conditions of Lemma 4 be fulfilled and for any r ∈ N there exist a
pair (Qr, Gr) of tensors such that
(Qr
n, Gr
n) d→(Qr, Gr) as n → ∞.
Then the sequence ((Qr, Gr), r ∈ N) converges in distribution to some limit (Q, G) and
relation (18) holds.
160 A. P. YURACHKIVSKY AND D. O. IVANENKO
Lemma 5. Let the sequence (εk) satisfy conditions (9) and (10) and for any uniformly
bounded sequence (αk) of Borel functions on R
rd
(36)
1
n
n−1∑
k=r
(
αk(εk−r+1, . . . , εk) − E0αk(εk−r+1, . . . , εk)
) P→O.
Then this relation holds for any sequence (αk) of Borel functions such that
(37) |αk(x1, . . . , xr)| ≤ C
(
r∑
i=1
|xi|2 + 1
)
.
Proof. Denote ζk = αk(εk−r+1, . . . , εk). Then for any N > 0
1
n
n−1∑
k=r
(
ζkI{|εk−r+1| ≤ N, . . . , |εk| ≤ N} − E0ζkI{|εk−r+1| ≤ N, . . . , |εk| ≤ N}) P→0,
so it suffices to prove that, for j = 0, . . . , r − 1,
(38) lim
N→∞
lim
n→∞
1
n
n∑
k=1
E0|ζk|I{|εk−j| > N} = 0 a.s.
By assumption,
(39) E0|ζk|I{|εk−j | > N} ≤ C
(
P{|εk−j| > N |F0} +
k−1∑
i=0
E0|εk−i|2I{|εk−j | > N}
)
.
Due to (10), P{|εi| > N |F0} ≤ υ2N−2 and E0|εk−i|2I{|εk−j | > N} ≤ υ4N−2 as i �= j,
which together with (39) and (9) implies (38).
Remark. Obviously, if relation (36) holds for any sequence of R-valued functions (uni-
formly bounded or satisfying (37)), then for any m ∈ N it is valid for any sequence of
R
m-valued functions with the same property.
The proof of the following statement is similar.
Lemma 6. Let the sequence (εk) satisfy conditions (9) and (10) and for any uniformly
bounded sequence (αk) of R-valued Borel functions on R
rd
(40)
1
n
n−1∑
k=r
(
σ2
k ⊗ αk(εk−r+1, . . . , εk) − E0(σ2
k ⊗ αk(εk−r+1, . . . , εk))
) P→O
(here ⊗ signifies the multiplication of a tensor by a real number). Then this relation
holds for any sequence (αk) of tensor-valued functions satisfying (37) (with ‖ · ‖ instead
of | · | on the left-hand side).
Corollary 6. Let the conditions of Lemmas 4 and 5 be fulfilled and for any uniformly
bounded sequence (αk) of R-valued Borel functions on R
rd the sequence(
1
n
n−1∑
k=r
E0αk(εk−r+1, . . . , εk), n = r, r + 1, . . .
)
converge in probability. Then the sequence (Qr
n, n = r, r + 1 . . . ) converges in probabil-
ity.
MATRIX PARAMETER ESTIMATION 161
Corollary 7. Let the conditions of Lemmas 4 and 6 be fulfilled and for any uniformly
bounded sequence of R-valued functions the sequence(
1
n
n−1∑
k=r
E0σ2
kαk(εk−r+1, . . . , εk), n = r, r + 1, . . .
)
converge in probability. Then the sequence (Gr
n, n = r, r + 1 . . . ) converges in probabil-
ity.
An example
Suppose that conditions (5), (8), (11), (23) and (24) are fulfilled. Let also εk = γkχk,
where (γk) and (χk) are independent sequences of random variables and i.i.d. random
vectors, respectively, |γk| ≤ C, and let for any r ∈ N and bounded Borel function g the
sequence (
1
n
n−1∑
k=r
g(γk−r+1, . . . , γk), n = r, r + 1, . . .
)
converge in probability; Eχ1 = 0, Eχ⊗2
1 = I.
For Fk, we take the σ-algebra generated by ξ0; χ1, . . . , χk; γ1, γ2, . . . (so that the whole
sequence (γk) is F0-measurable). Then σ2
k = γ2
kI,
(41) Gr
n = I ⊗ 1
n
n∑
k=r
(γkηr
k)⊗2
and conditions (2), (3), and (10) are fulfilled. So is (9), because the γk’s are uniformly
bounded and χk’s are identically distributed.
To deduce (19) from Theorem 2 and Corollary 5 it suffices to verify the conditions of
Corollaries 6 and 7. In view of (41) and the expressions for Qr
n and ηr
k, we may confine
ourselves with the case αk = α.
By the Stone – Weierstrass theorem, α can be approximated uniformly on compacta
with finite linear combinations of functions of the kind h(y)h1(x1) . . . hr(xr) (y ∈ R
r, xj ∈
R
d). By the choice of Fk and the assumptions on (γk) and (χk),
E0h(γk)h1(χk−r+1) . . . hr(χk) = h(γk)
r∏
i=1
Ehi(χ1),
where γk = (γk−r+1, . . . , γk).
Hence and from the Chebyshev’s inequality, (36) emerges. The last condition of Corol-
lary 6 follows from (41) and the above assumption on (γk).
If detQ �= 0, then Theorem 2 asserts (19). If herein l.i.p.n→∞
1
n
∑n
k=r g(γk) is random,
then the limiting distribution will not be Gaussian.
Bibliography
1. A.Ya. Dorogovtsev, Estimation theory for parameters of random processes, Kyiv University
Press, Kyiv, 1982. (Russian)
2. A.P. Yurachkivsky, D.O. Ivanenko, Matrix parameter estimation in an autoregression model
with non-stationary noise, Th. Prob. Math. Stat. 72 (2005), 158–172. (Ukrainian)
3. A.P. Yurachkivsky, Conditions for convergence of a sequence of martingales in terms of their
quadratic characteristics, Reports of the Nat. Ac. Sc. Ukr. 1 (2003), 33–36.
4. R.Sh. Liptser, A.N. Shiryaev, Theory of Martingales, Nauka, Moscow, 1986. (Russian)
E-mail : yap@univ.kiev.ua, ida@univ.kiev.ua
|