Modified orthogonal regression estimator in the quadratic errors-in-variables model
The quadratic functional measurement error model with equal error variances is
 considered. The asymptotic bias of an orthogonal regression estimator is derived. A
 modified estimator which has smaller asymptotic bias for small measurement errors
 is presented.
Saved in:
| Date: | 2005 |
|---|---|
| Main Author: | |
| Format: | Article |
| Language: | English |
| Published: |
Інститут математики НАН України
2005
|
| Online Access: | https://nasplib.isofts.kiev.ua/handle/123456789/4432 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Journal Title: | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| Cite this: | Modified orthogonal regression estimator in the quadratic errors-in-variables model / G. Repetatska // Theory of Stochastic Processes. — 2005. — Т. 11 (27), № 3-4. — С. 110–120. — Бібліогр.: 5 назв.— англ. |
Institution
Digital Library of Periodicals of National Academy of Sciences of Ukraine| _version_ | 1860255121587306496 |
|---|---|
| author | Repetatska, G. |
| author_facet | Repetatska, G. |
| citation_txt | Modified orthogonal regression estimator in the quadratic errors-in-variables model / G. Repetatska // Theory of Stochastic Processes. — 2005. — Т. 11 (27), № 3-4. — С. 110–120. — Бібліогр.: 5 назв.— англ. |
| collection | DSpace DC |
| description | The quadratic functional measurement error model with equal error variances is
considered. The asymptotic bias of an orthogonal regression estimator is derived. A
modified estimator which has smaller asymptotic bias for small measurement errors
is presented.
|
| first_indexed | 2025-12-07T18:47:57Z |
| format | Article |
| fulltext |
Theory of Stochastic Processes
Vol. 11 (27), no. 3–4, 2005, pp. 110–120
UDC 519.21
GALYNA REPETATSKA
MODIFIED ORTHOGONAL REGRESSION ESTIMATOR
IN THE QUADRATIC ERRORS-IN-VARIABLES MODEL
The quadratic functional measurement error model with equal error variances is
considered. The asymptotic bias of an orthogonal regression estimator is derived. A
modified estimator which has smaller asymptotic bias for small measurement errors
is presented.
Introduction
We consider a quadratic functional errors-in-variables model
(1)
yi = a0ξ
2
i + b0ξi + c0 + δi,
xi = ξi + εi, 1 � i � n,
where (xi, yi), 1 � i � n, are observed, ξi are unknown nonrandom parameters, εi, δi are
i.i.d. normal error terms, and the vector β0 = (a0, b0, c0)
T consists of the parameters
to be estimated. Noise variances are unknown.
The general discussion of the linear error-in-variables model is given in [4]. Concern-
ing the orthogonal regression estimator, it is proved in [1] that, for nonlinear errors-in-
variables models including (1), this estimator is inconsistent. In [2], a new corrected
estimator is presented which has smaller asymptotic bias. In [5], this estimator with
some changes was extended for a model, where all variables are vectors.
In this paper, the next term of the asymptotic bias is derived, and a new estimator is
proposed. A similar estimator can be used for other nonlinear regression models, but we
consider, for simplicity, only the quadratic regression function. In Section 1, the model
assumptions and an orthogonal regression estimator are presented. In Section 2, two
leading terms of the asymptotic bias of the estimator are derived. In Section 3, two
corrected estimators are proposed. The first estimator has been proposed in [2], another
one is original. It has less asymptotic deviation than the first one.
Some calculations were performed with the Mathematica 3.0 program. The proofs of
Theorems 1 and 2 are put in Appendix.
1. Model assumptions and orthogonal regression estimator
Let g(ξ, β) = aξ2 + bξ + c be a regression function, where β = (a; b; c)T ∈ Θ
is the vector of the unknown parameters. In the paper, all the vector values are column
vectors. The derivatives are denoted by superscripts, and the vector derivatives are row
vectors. For example, gξ(ξ, β) = 2aξ + b, and gβ =
(
ξ2; ξ; 1
)
is the derivative with
respect to the vector variable β. The expectation of a random variable ζ is denoted by
2000 AMS Mathematics Subject Classification. Primary 62J02; Secondary 62F12, 62H12.
Key words and phrases. Asymptotic bias, bias correction, orthogonal regression estimator, quadratic
errors-in-variables regression model, functional model.
110
MODIFIED ORTHOGONAL REGRESSION ESTIMATOR 111
E ζ, and its variance is denoted by D ζ. A sequence {χn(θ), n � 1} of random functions
is denoted by OP (1) if it is uniformly stochastically bounded.
Let G(x, y, β, u)=(y − g(u, β))2 + (x − u)2. Then q(x, y, β) := minu∈R G(x, y, u, β) is
the squared distance between a point (x, y) and a parabola y = g(u, β), u ∈ R.
Introduce the objective function Q(β) = 1
n
∑n
i=1 q(xi, yi, β). Then the orthogonal
regression estimator β̂ is defined as a measurable solution to the optimization problem:
Q(β)— min, β ∈ Θ,
where Θ is a parameter set.
Assume that the following conditions hold:
(i) β0 ∈ intΘ, Θ is a compact set in R
3.
(ii) |ξi| � A, i � 1, where A is unknown.
(iii) εi, δi ∼ N(0, σ2) i.i.d., i � 1, where σ > 0 is the unknown parameter.
(iv) a0 �= 0, i.e., the true regression function is nonlinear.
Consider the problem of existence and uniqueness of a minimum point of the function
G(x, y, β, u), u ∈ R.
1. Existence. The function G(x, y, β, u) is continuous and tends to +∞ as u →
∞. Therefore, there exists at least one minimum point. Denote one of such points by
h(x, y, β). Note that, for any minimum point h,
q(x, y, β) = G(x, y, β, h(x, y, β)).
2. Uniqueness. For a minimum point, Gu(x, y, β, u)|u=h(x,y,β) = 0 holds. Hence,
h(x, y, β) is implicitly defined by the normal equation
(2) F (x, y, β, h) := −1
2
Gu|u=h = (y − g(h, β)) gξ(h, β) + x − h = 0.
Hence, h is a solution to the cubic equation
(
y − ah2 − bh − c
)
(2ah + b)+x−h = 0. The
equation can have from one to three solutions, some of them can not be a global minimum
point. Note that F (ξ, g(ξ, β), β, ξ) ≡ 0 and Fu (ξ, g(ξ, β), β, ξ) = −1 − [
gξ(u, β)
]2 �= 0.
Then the Implicit Functions Theorem implies the following: there exists a neighbourhood
of a point (ξ, g(ξ, β), β), Uν(ξ, β) := Bν(ξ) × Bν (g(ξ, β)) × Bν(β), ν = ν(ξ, β), such
that h : Uν(ξ, β) → R is a uniquely defined infinitely differentiable function. Since ξ
and β belong to compact sets, it is possible to find a common value ν0 > 0 for all
β ∈ Θ, ξ ∈ [−A − 1, A + 1].
If the absolute value of both error terms εi, δi is less than ν0, then there exists only
one perpendicular from (xi, yi) to any of the curves y = g(ξ, β), ξ ∈ R; β ∈ Uν0(β0). Let
ν be a fixed positive constant, ν ∈ (0, ν0], such that Uν(β0) ⊂ Θ. We define the index set
Bn(ν) =
{
i = 1, n : |εi| < ν, |δi| < ν
}
and divide the objective function into two parts:
Q(β) = Q1(β) + Q2(β) :=
1
n
∑
i∈Bn(ν)
q(xi, yi, β) +
1
n
∑
i/∈Bn(ν)
q(xi, yi, β).
Here, Q1(β) is the leading term and Q2(β) is the remainder one. Now we find an
asymptotic expansion of Q1(β0) and its derivatives in σ2.
We will widely use the following statement.
Lemma 1. Let {ζi : i � 1} be an i.i.d. sequence with D ζ1 = 1, and let {ai : i � 1} be a
bounded sequence of real numbers. Then
1
n
n∑
i=1
aiζi =
E ζ1
n
n∑
i=1
ai +
1√
n
OP (1).
The derivatives gβ , gβξ, gβξξ are row vectors. For a couple of row vectors �a,�b, we
define a symmetric matrix �a ∗ �b = 1
2 (�aT�b + �bT�a). For a triple �a,�b,�c, let �a ∗ �b ∗ �c be a
112 GALYNA REPETATSKA
cubic matrix corresponding to a symmetric trilinear form which acts on a vector �x as
(�a, �x) · (�b, �x) · (�c, �x).
Define the following functions:
k(ξ, β) =
gξξ
(1 + (gξ)2)2
gβ, V (ξ, β) =
1
1 + (gξ)2
gβTgβ,
p(ξ, β) =
9a3
0(g
ξ)2
(1 + (gξ)2)5
gβ +
3a2
0g
ξ
(1 + (gξ)2)4
gβξ − a0
2 (1 + (gξ)2)3
gβξξ,
W (ξ, β) =
(gξξ)2
(
7(gξ)2 − 2
)
(1 + (gξ)2)4
gβTgβ − 8gξξgξ
(1 + (gξ)2)3
gβ ∗ gβξ +
1
(1 + (gξ)2)
gβ ∗ gβξξ,
T(ξ, β) =
gξξ(gξ)2
(1 + (gξ)2)3
gβ ∗ gβ ∗ gβ − 2gξ
(1 + (gξ)2)2
gβ ∗ gβ ∗ gβξ.
For an arbitrary function F (ξ, β), let Fn = 1
n
∑n
i=1 F (ξi, β0). In this way, we can
define the quantities kn, Vn, pn, Wn, Tn.
Definition 1. A sequence of random vectors ηn(β, σ) = oσP (1), if, for each c > 0,
lim
σ→0+
sup
n�1
P
(
sup
β∈Θ
‖ηn(β, ν, σ)‖ > c
)
= 0.
The following theorem gives the asymptotic expansions of the function Q1(β) and its
derivatives.
Theorem 1. Suppose that, for model (1), assumptions (i)–(iii) are satisfied. Then
Q(β) = Q1(β) + σ8oσP (1),(3)
Q(β0) = σ2 − σ4
4n
n∑
i=1
(gξξ)2
(1 + (gξ)2)3
|(ξi,β0) +
σ2
√
n
OP (1) + σ4oσP (1),
Qβ
1 (β0) = σ2kn + σ4pn + σ6R1 + σ6oσP (1) +
σ√
n
OP (1),(4)
Qββ
1 (β0) = 2Vn + 2σ2Wn + σ4R2 + σ4oσP (1) +
σ√
n
OP (1),
Qβββ
1 (β0) = 6Tn + σ2R3 + σ2oσP (1) +
σ√
n
OP (1),
Qββββ
1 (β0) = R4 + σ2oσP (1) +
σ√
n
OP (1),
where R1, R2, R3, R4 are bounded nonrandom terms.
The inconsistency of an orthogonal regression estimator was proved in [1] in the case
where kn is separated from zero. Theorem 1 helps us to find two leading terms of the
asymptotic expansion of β̂ − β0 in powers of σ2.
2. Asymptotic deviation
Definition 2. A sequence of random vectors ηn(σ) = ÕσP (1), if
∀ ε > 0 ∃C > 0 : lim
σ→0+
lim sup
n→∞
P (‖ηn(σ)‖ > C) < ε.
Definition 3. A sequence of random vectors ηn = õσP (1), if
lim
σ→0+
lim sup
n→∞
P (‖ηn(σ)‖ > C) → 0, C → ∞.
MODIFIED ORTHOGONAL REGRESSION ESTIMATOR 113
Let Mi(xi, yi), M0
i (ξi, g(ξi, β0)) be the points on the plane, Γβ := {(ξ, g(ξ, β)) : ξ ∈ R}
be a plot of the regression function with a parameter β, ρ be the Euclidean metrics, and
ρ(M, Γβ) be the distance between a point M and the plot Γβ .
We need the following contrast condition:
(con) ∀ δ > 0 : lim infn→∞ inf‖β−β0‖>δ
1
n
∑n
i=1 ρ2(M0
i , Γβ) > 0.
This condition makes it possible to estimate consistently the parameter β0 by β̂, as
n → ∞ and σ → 0.
Lemma 2 [2]. Suppose that, for model (1), the contrast condition (con) is satisfied.
Then a.s. ∀ γ > 0 ∃σγ > 0 ∃nγ = nγ(ω) ∀n � nγ ∀σ � σγ : ‖β̂n − β0‖ < γ.
This implies that β̂n − β0 = õσP (1).
Denote the minimal eigenvalue of a matrix A by λmin(A). To find the asymptotic
deviation of the estimate, we need the following assumption:
(v) lim infn→∞ λmin(Vn) > 0.
Theorem 2. Suppose that, for model (1), conditions (i)–(v) and (con) are satisfied.
Then
β̂n = β0 + σ2zn + σ4ÕσP (1),
β̂n = β0 + σ2zn + σ4an + σ4õσP (1), (5)
where zn := − 1
2V −1
n kT
n , an := − 1
2V −1
n
(
pn + 2zT
n Wn + 3Tn(zn)2
)
.
3. Modified estimators
We will construct consequently two estimators which have smaller asymptotic bias
than β̂. We have to estimate the terms zn and an of the asymptotic expansion in (3).
Let F (ξ, β), ξ ∈ R, β ∈ Θ be an arbitrary twice differentiable function.
1) For Fn = 1
n
∑n
i=1 F (ξi, β0), we introduce the following estimator:
F̂n =
1
n
n∑
i=1
F (xi, β̂).
Thus, we have the estimators of the terms kn, Vn, pn, Wn, Tn in the form k̂n, V̂n, etc.
For σ2, we have the estimator σ̂2 := Q(β̂). Next, we have a new estimator of the
parameter β0:
β̃n = β̂n +
σ̂2
2
V̂ −1
n k̂n.
2) Define the more precise estimators of Fn and σ2,
F̃n =
1
n
n∑
i=1
F (xi, β̃) − σ̂2
2n
n∑
i=1
F ξξ(xi, β̂),
σ̃2 = Q(β̂) +
Q2(β̂)
4
⎛
⎝k̂nV̂ −1
n k̂T
n +
1
n
n∑
i=1
(gξξ)2
(1 + (gξ)2)3
∣∣∣∣∣
(xi,β̂)
⎞
⎠ .
Let ẑn = − 1
2 V̂ −1
n k̂T
n , z̃n = − 1
2 Ṽ −1
n k̃T
n , ân = − 1
2 V̂ −1
n
(
p̂n + 2Ŵnẑn + 3T̂nẑ2
n
)
.
A more precise estimator of β0 is
˜̃
βn = β̂ − σ̃2z̃n − σ̂4ân.
Theorem 3. Suppose that conditions (i)–(v) and (con) hold for model (1). Then
1) β̃n − β0 = σ2õσP (1),
2) ˜̃βn − β0 = σ4õσP (1).
114 GALYNA REPETATSKA
Proof. We start with some auxiliary statements. In these statements, we suppose that
the conditions of Theorem 3 hold, the function F is three times differentiable, and, for
some positive C, k, the inequality
‖F β(ξ, β)‖ + ‖F ξξξ(ξ, β)‖ � C(1 + |ξ|k), ξ ∈ R, β ∈ Θ, (6)
holds. We normalize the error terms to obtain standard normal variables:
ε̃i = εi/σ, δ̃i = δi/σ. (7)
Proposition 1. σ̂2 − σ2 = σ4ÕσP (1), σ̂4 − σ4 = σ6ÕσP (1).
Proof. Theorem 1 states that Q(β0) = σ2 + σ4ÕσP (1), and formula (11) from Appendix
implies that Q(β̂) − Q(β0) = σ4ÕσP (1). Hence, we obtain
σ2 − σ̂2 = Q(β0) + σ4ÕσP (1) − Q(β̂) = σ4ÕσP (1), and
σ̂4 − σ4 = (σ̂2 − σ2)
(
σ̂2 + σ2
)
= σ4ÕσP (1)
(
2σ2 + (σ̂2 − σ2)
)
= σ6ÕσP (1). �
Proposition 2. F̂n = Fn + õσP (1).
Proof. F̂n − Fn = 1
n
∑n
i=1
(
F (xi, β̂n) − F (xi, β0)
)
+ 1
n
∑n
i=1 (F (xi, β0) − F (ξi, β0)) =:
r1 + r2.
1) r1 = 1
n
∑n
i=1 F β(xi, β̄i)(β̂n − β0) = σ2ÕσP (1) 1
n
∑n
i=1 F β(xi, β̄i), β̄i ∈ [β0, β̂n], and∥∥∥∥∥ 1
n
n∑
i=1
F β(xi, β̄i)
∥∥∥∥∥ � C
n
n∑
i=1
(|ξi + εi|k + 1
)
� C +
C · 2k−1
n
n∑
i=1
(|ξi|k + σk|ε̃i|k
)
�
� C
(
1 + 2k−1ak + 2k−1σkOP (1)
)
= OP (1).
2) r2 = 1
n
∑n
i=1 F ξ(ξ̄i, β0) · εi = σ · 1
n
∑n
i=1 F ξ(ξ̄i, β0)ε̃i = σOP (1), similarly to r1. �
Proof of Statement 1) of Theorem 3. Theorem 2 states that
β̃ − β0 =
σ̂2
2
V̂ −1
n k̂T
n − σ2
2
V −1
n kT
n + σ2õσP (1).
The functions k(ξ, β) and V (ξ, β) satisfy inequality (4). Hence,
σ̂2
2
V̂ −1
n k̂T
n − σ2
2
V −1
n kT
n = σ2õσP (1). �
Proposition 3. F̃n − Fn = σ2õσP (1).
Proof. By the Taylor expansion,
1
n
n∑
i=1
F (xi, β̃) − 1
n
n∑
i=1
F (ξi, β0) =
1
n
n∑
i=1
(
F (xi, β̃) − F (xi, β0)
)
+
+
1
n
n∑
i=1
(F (xi, β0) − F (ξi, β0)) =: A1 + A2.
We have A1 = 1
n
∑n
i=1 F β(xi, β̄i)(β̃n − β0) = σ4ÕσP (1) · 1
n
∑n
i=1 F β(xi, β̄i), where
1
n
∑n
i=1 F β(xi, β̄i) = OP (1) similarly to r1 from Proposition 2. Hence, A1 = σ2õσP (1).
Next,
A2 =
1
n
n∑
i=1
F ξ(ξi, β0) · εi +
1
2n
n∑
i=1
F ξξ(ξi, β0) · ε2
i +
+
1
6n
n∑
i=1
F ξξξ(ξ̄i, β0) · ε3
i =: R1 + R2 + R3,
MODIFIED ORTHOGONAL REGRESSION ESTIMATOR 115
where R1 = 1
n
∑n
i=1 F ξ(ξi, β0) · εi = 0 + σ√
n
OP (1) = σ2õσP (1);
R2 = 1
2n
∑n
i=1 F ξξ(ξi, β0) · ε2
i = σ2
2 F ξξ
n + σ2√
n
OP (1) = σ2
2 F̂ ξξ
n + σ2õσP (1);
‖R3‖ � C·σ3
6n
∑n
i=1
(
1 + |ξ̄i|k
)
ε3
i � C·σ3
6n
∑n
i=1
(
1 + 2k−1(|ξi|k + σk|ε̃i|k
) |ε̃i|3 = σ3OP (1).
Summarizing we have
1
n
∑n
i=1 F (xi, β̃) = Fn + σ2
2 F̂ ξξ
n + σ2õσP (1) = Fn + σ̂2
2 F̂ ξξ
n + σ2õσP (1). �
Proposition 4. σ̃2 − σ2 = σ4õσP (1).
Proof. Formulae (11) and Δϕ̂ − zn = õσP (1) from the proof of Theorem 2 (see below)
imply that Q(β̂) − Q(β0) = σ4
(
knzn + Vn(zn)2
)
+ σ4õσP (1), whence
Q(β0) = Q(β̂) − σ4
(− 1
2knV −1
n kT
n + 1
4knV −1
n kT
n
)
+ σ4õσP (1) =
= σ̂2 + σ4
4 knV −1
n kT
n + σ4õσP (1).
We replace σ2, kn and Vn by their estimators. Then, by Propositions 1 and 2,
Q(β0) = σ̂2 + 1
4 σ̂4 · k̂nV̂ −1
n k̂T
n + σ4õσP , (1)
and the second formula from the condition of Theorem 1 takes the form
Q(β0) = σ2 − σ̂4
4n
n∑
i=1
(gξξ)2
(1 + (gξ)2)3
|(xi,β̂) + σ4õσP (1).
From the last two expansions, we obtain
σ2 = σ̂2+
σ̂4
4
(
k̂nV̂ −1
n kT
n +
1
n
n∑
i=1
(gξξ)2
(1 + (gξ)2)3
|(xi,β̂)
)
+σ4õσP (1), where σ̂2 = Q(β̂). �
Proof of Statement 2) of Theorem 3.
Theorem 2 states that β0 = β̂ − σ2zn − σ4an + σ4õσP (1). The functions k, V, p, W, and
T satisfy inequality (6). Therefore, in view of Propositions 2 and 4, ân − an = õσP (1),
z̃n − zn = σ2õσP (1). Hence, σ̂4ân − σ4an = σ4õσP (1), σ̃2z̃n − σ2zn = σ4õσP (1). Then
we obtain
˜̃
βn − β0 = σ̃2z̃n − σ2zn + σ̂4ân − σ4an = σ4õσP (1).
Theorem 3 is proved. �
APPENDIX
Proof of Theorem 1.
1) Proof of (3). We show that Q2(β) = σ8oσP (1). Consider a component of this sum:
q(x, y, β) = (y − g (h(x, y, β), β))2 + (x − h(x, y, β))2 � (y − g(ξ, β))2 + (x − ξ)2 �
� 2 (y − g(ξ, β0))
2 + 2 (g(ξ, β0) − g(ξ, β))2 + (x − ξ)2 � 2δ2 + ε2 + const.
Remember that ε̃i and δ̃i were defined in (7). We have
Q2(β) =
1
n
∑
i/∈Bn(ν0)
q(xi, yi, β) � 1
n
∑
i/∈Bn(ν0)
(
2δ2 + ε2 + const
)
�
� 1
n
n∑
i=1
(
2δ2 + ε2 + const
) · [I (|εi| � ν) + I (|δi| � ν)] =
=
σ2
n
n∑
i=1
(
2δ̃2
i + ε̃2
i + const
) [
I (|ε̃i| � ν/σ) + I
(
|δ̃i| � ν/σ
)]
.
Let us consider the expectations of the terms in the former expression by using the
following inequality: 1−FN (x) � 1
xfN (x), x > 0, where fN and FN are, respectively, the
116 GALYNA REPETATSKA
standard normal density and the normal distribution function. Hence, P (|ε̃i| � ν/σ) =
2 (1 − FN (ν/σ)) � 2σ
ν · 1√
2π
e−
ν2
2σ2 and then
σ2E I (|ε̃i| � ν/σ) · ε̃2
i � σ2 ·
√
6σ
ν
· (2π)−1/4e−
ν2
4σ2 .
Similar inequalities can be obtained for other terms, and we have finally
EQ2(β) � Cσe−
ν2
4σ2 = C1σ
8o(1), as σ → 0+. Hence, by the Chebyshev inequality
P
(
Q2(β)
σ8
> C
)
� EQ2(β)
σ8C
=
o(1)
C
→ 0 as σ → 0+,
and Q2(β) = σ8oσP (1), where σ8 can be replaced by any positive degree of σ.
2) Now consider the case i ∈ Bn(ν). We denote hi = h(xi, yi, β0). We omit the index
i for the terms xi, yi, εi, δi, hi. All these terms belong to a compact set for all i ∈ Bn(ν).
Introduce Δ = h − ξ. Note that Δ = O (|ε| + |δ|) . Indeed,
Δ2 = (ξ − h)2 � 2
[
(ξ − x)2 + (x − h)2
]
� 2ε2 + 2
[
(y − g(h, β0))
2 + (x − h)2
]
�
� 2ε2 + 2
[
(y − g(ξ, β0))
2 + (x − ξ)2
]
= 4ε2 + 2δ2.
We write down the Taylor expansion for the regression function g. When some function
is taken at the point (ξ, beta0), we write it without the argument. Then
g(h, β0) = g + gξΔ + 1
2gξξΔ2 = g + gξΔ + a0Δ2,
gξ(h, β0) = gξ + gξξΔ = gξ + 2a0Δ, (8)
gβ(h, β0) = gβ + gβξΔ + 1
2gβξξΔ2, gβξ(h, β0) = gβξ + gβξξΔ.
We substitute it into (8) and obtain the equation for Δ:(
δ − gξΔ − a0Δ2
) (
gξ + 2a0Δ
)
+ ε − Δ = 0 (9)
with the unknown parameters ξ and β0 = (a0, b0, c0)
T. The equation has a unique solu-
tion Δ = Δ(ε, δ), for any i ∈ Bn(ν). The function Δ = Δ(ε, δ) is infinitely differentiable
for |ε| < ν, |δ| < ν, and we can find its Taylor expansion. For an arbitrary function
s(ε, δ), we denote the k-th term of the expansion by sk. Then
Δ = Δ1 + . . . + Δ6 + O
(|ε|7 + |δ|7) , Δk =
∑
i+j=k
c
(k)
ij εiδj .
Here, Δk is a polynomial of ε and δ with the coefficients depending of gξ and a0.
Substituting (8) in (9), we find Δk as
(δgξ + ε) + 2a0Δδ − 3a0g
ξΔ2 − 2a2
0Δ
3 =
(
(gξ)2 + 1
)
Δ ⇔
Δ =
(δgξ + ε) + 2a0Δδ − 3a0g
ξΔ2 − 2a2
0Δ
3
(gξ)2 + 1
.
Hence,
Δ1 =
δgξ + ε
(gξ)2 + 1
, Δ2 =
2a0Δ1δ − 3a0g
ξΔ2
1
(gξ)2 + 1
,
Δ3 =
2a0Δ2δ − 6a0g
ξΔ1Δ2 − 2a2
0Δ3
1
(gξ)2 + 1
,
Δ4 =
2a0Δ3δ − 3a0g
ξ(Δ2
2 + 2Δ1Δ3) − 6a2
0Δ2
1Δ2
(gξ)2 + 1
.
Similarly, one can find Δ5 and Δ6.
MODIFIED ORTHOGONAL REGRESSION ESTIMATOR 117
3) We now find the Taylor expansions of q, qβ , qββ and qβββ at the point (x, y, β0) as
functions of ε and δ and their expectations. The expectations of odd terms are zeros,
and those of even terms are certain functions of σ, ξ, β0.
a) Consider q(x, y, β0):
q(x, y, β0) = (y − g(h, β0)) + (x − h)2 =
(
δ − gξΔ − 1
2gξξΔ2
)
+ (ε − Δ)2 =
= q2(ε, δ) + q3(ε, δ) + q4(ε, δ) + O(|ε|5 + |δ|5), where
q2(ε, δ) = (δ − gξΔ1)2 + (ε − Δ1)2,
q4(ε, δ) =
(
gξΔ2 + 1
2gξξΔ2
1
)2 − 2(δ − gξΔ1)(gξΔ3 + gξξΔ1Δ2).
The expectations of these terms are E q2(ε, δ) = σ2, E q4(ε, δ) = −σ2
4 (gξξ)2
(
1 + (gξ)2
)−3.
b) Consider qβ(x, y, β0).
qβ(x, y, β) = Gβ(x, y, β, u)|u=h + Gu(x, y, β, u)|u=h · hβ(x, y, β) =
= −2 (y − g(h, β)) gβ(h, β), because Gu|u=h = 0.
From (8), we obtain
qβ(x, y, β0) =
(
δ − gξΔ − 1
2gξξΔ2
) (
gβ + gβξΔ + 1
2gβξξΔ2
)
, whence
qβ
2 =
(
2gξΔ2 + 1
2gξξΔ2
1
)
gβ − 2
(
δ − gξΔ1
)
Δ1g
βξ;
qβ
4 =
(
2gξΔ4 + gξξ(Δ2
2 + 2Δ1Δ3)
)
gβ +
(
3gξξΔ2
1Δ2 + 2gξΔ2
2 − 2(δ − 2gξΔ1)Δ3
)
gβξ+
+
(
1
2gξξΔ4
1 − Δ1Δ2(2δ − 3gξΔ1)
)
gβξξ.
The expectations of the expansion terms are E qβ
2 (ε, δ) = σ2k(ξ, β0), E qβ
4 (ε, δ) =
σ4p(ξ, β0).
c) Consider qββ(x, y, β0).
qββ(x, y, β) = Gββ +
(
Gβu
)T
hβ = Gββ − (
Gβu
)T
Gβu · (Guu)−1, hβ = −(Guu)−1Gβu.
We write down the expansion terms of Gββ, Gβu, and Guu at the point (x, y, β0, h):
Gββ(x, y, β0, h) = 2gβ T(h, β0)gβ(h, β0),
Gββ
0 = 2gβ Tgβ, Gββ
2 = 2Δ2
1g
βξ Tgβξ + 4Δ2
(
gβ ∗ gβξ
)
+ 2Δ2
1
(
gβ ∗ gβξξ
)
,
Gβu(x, y, β0, h) = 2gξ(h, β0)gβ(h, β0) − 2 (δ − g(h, β0)) gβξ(h, β0),
Gβu
0 = 2gξgβ , Gβu
1 = 2gξξΔ1g
β − (δ − 2gξΔ1)gβξ,
Gβu
2 = 2gξξΔ2g
β + 2
(
2gξΔ2 + 3
2gξξΔ2
1
)
gβξ − (2δ − 3gξΔ1)Δ1g
βξξ,
Guu(x, y, β0, h) = 2
(
1 + (gξ(h, β0))2 − (y − g(h, β0))gξξ(h, β0)
)
,
Guu
0 = 2
(
1 + (gξ)2
)
, Guu
1 = 2gξξ
(
3gξΔ1 − δ
)
, Guu
2 = 6gξξ
(
gξΔ2 + 1
2gξξΔ2
1
)
.
Then the Taylor expansion of (Guu)−1 is
(Guu)−1 = 1
Guu
0
(
1 −
(
Guu
1
Guu
0
+ Guu
2
Guu
0
)
+
(
Guu
1
Guu
0
+ Guu
2
Guu
0
)2
+ O
(|ε|3 + |δ|3)) ,
whence three first terms of the expansion are
(Guu)−1
0 := (Guu
0 )−1, (Guu)−1
1 := − Guu
1
(Guu
0 )2 , (Guu)−1
2 := 1
Guu
0
((
Guu
1
Guu
0
)2
− Guu
2
Guu
0
)
.
The terms of degrees 0 and 2 for qββ(x, y, β0) are as follows:
qββ
0 = Gββ
0 − 1
Guu
0
(Gβu
0 )TGβu
0 = 2
(
1 − (gξ)2
1+(gξ)2
)
gβ Tgβ = 2
1+(gξ)2
gβ Tgβ = 2V (ξ, β0);
qββ
2 = Gββ
2 −
(
Gβu T
0 Gβu
2 + Gβu T
1 Gβu
1 + Gβu T
2 Gβu
0
)
(Guu)−1
0 −
−
(
Gβu T
0 Gβu
1 + Gβu T
1 Gβu
0
)
(Guu)−1
1 −
(
Gβu T
0 Gβu
0
)
(Guu)−1
2 .
The expectation of qββ
2 is E qββ
2 (ε, δ) = 2σ2W (ξ, β0).
118 GALYNA REPETATSKA
d) Consider qβββ(x, y, β0). The third order derivatives of G at the point (x, y, β, u)
are as follows:
Gβββ = 0, Gββu = 2
(
gβ Tgβξ + gβξ Tgβ
) |(u,β), Guuu =
(
6gξgξξ
) |(u,β),
Gβuu =
(
2gξgβξ + 2gξξgβ + 2gξgβξ − (y − g) gβξξ
) |(u,β), and
hββ = −(Guu)−1
(
Gββu + 2Gβuuhβ + Guuu · (hβ Thβ)
)
, as a derivative of the implicit
function. Differentiating the function qββ(x, y, β) = Gββ + Gβu · hβ, we have
qβββ = Gβββ + Gββu ∗ hβ + hβ ∗ Gβuu ∗ hβ + Gβu ∗ hββ = −3(Guu)−1
(
Gββu ∗ Gβu
)
+
+ 3(Guu)−2
(
Gβuu ∗ Gβu ∗ Gβu
) − (Guu)−3Guuu
(
Gβu ∗ Gβu ∗ Gβu
)
.
It can be easily found from the above-written that
qβββ
0 =
6gξξ(gξ)2
(1 + (gξ)2)3
gβ ∗ gβ ∗ gβ − 12gξ
(1 + (gξ)2)2
gβ ∗ gβ ∗ gβξ = 6T(ξ, β0)
(Remember that the notation �a ∗�b ∗ �c was given just after Lemma 1).
4) Proof of the statements of Theorem 1. The first of them has been already proved,
and the rest ones are easily inferred from the formulas stated above. We derive expansion
(4) for Qβ
1 (β0):
Qβ
1 (β0)=
1
n
∑
i∈Bn(ν)
qβ(xi, yi, β0) =
6∑
j=1
Ak +
1
n
∑
i∈Bn(ν)
O
(|ε|7 + |δ|7), Ak =
∑
i∈Bn(ν)
qβ
k (εi, δi).
We use Lemma 1 several times. Start with A1. The term qβ
1 (ε, δ) is a linear form of
ε̃ and δ̃ with bounded coefficients. It follows from (ii) that, for arbitrary i, 1 � i � n,
qβ
1 (εi, δi) = −2
(
δi − gξΔ1(εi, δi)
)
gβ = σ
2(gξε̃i − δ̃i)
(gξ)2 + 1
gβ .
To apply Lemma 1, we divide A1 into two sums:
A1 =
1
n
n∑
i=1
qβ
1 (xi, yi, β0) − 1
n
∑
i/∈Bn(ν)
qβ
1 (xi, yi, β0) =: S1 − S2,
where S1 = σ√
n
OP (1), and S2 = σ8oσP (1) like a sum in 1). Other Ak can be expanded
in a similar way. Consider A6 separately. Introduce
R1 = σ−6 1
n
n∑
i=1
E qβ
6 (εi, δi) =
1
n
n∑
i=1
6∑
l=0
�cl(ξi, β0) · E ε̃l
iδ̃
6−l
i .
It is a bounded nonrandom vector depending only on (ξi, β0). Dividing A6 into two sums
similarly to A1, we obtain
A6 =
1
n
n∑
i=1
qβ
6 (εi, δi) + σ6oσP (1) = σ6
(
R1 +
OP (1)√
n
)
+ σ6oσP (1).
Now consider the last term. We get∥∥∥∥ 1
n
∑
i∈Bn(ν)
O
(|εi|7 + |δi|7
)∥∥∥∥ � const
σ7
n
n∑
i=1
(
|ε̃i|7 + |δ̃i|7
)
= σ7OP (1).
The expansion of Qβ
1 (β0) follows from the preceding formulas. In a similar way, we
can obtain the expansions for Q(β0), Qββ
1 , Qβββ
1 , and Qββββ. The last two derivatives
are considered as the matrices corresponding to tri- and four-linear forms. �
Proof of Theorem 2.
1) We find an expansion of Q(β). Consider the case where β̂n ∈ Uν(β0). It occurs for
some σ � σε, n � nε with probability at least 1− ε, where ε can be an arbitrary positive
quantity.
MODIFIED ORTHOGONAL REGRESSION ESTIMATOR 119
2) Write the Taylor expansion of Q1(β) in Δβ = β − β0:
Q1(β) = Q1(β0) + Qβ
1 (β0)Δβ +
1
2!
Qββ
1 (β0) (Δβ)2 +
1
3!
Qβββ
1 (β0) (Δβ)3 +
+
1
4!
∂4Q1(β0)
∂β4
(Δβ)4 +
1
5!
∂5Q1(β̄)
∂β5
(Δβ)5, β̄ ∈ [β0, β].
The derivative ∂5Q1(β̄)
∂β5 is bounded because all the partial derivatives of Q1(β) are
bounded. Take the expansions from Theorem 1 and denote Δϕ = σ2Δβ. We obtain
Q(β) − Q(β0) = Q1(β) − Q1(β0) + σ8oσP (1) = σ4
(
knΔϕ + Vn(Δϕ)2
)
+
+ σ6
(
pnΔϕ + Vn(Δϕ)2 + Tn(Δϕ)3
)
+ σ8R(ϕ) + rest(Δϕ),
(10)
where R(ϕ) := R1Δϕ + R2(Δϕ)2 + R3(Δϕ)3 + R4(Δϕ)4,
rest = σ8
((
1 + ‖Δϕ‖4
)
õσP (1) + ‖Δϕ‖4‖Δβ‖O(1)
)
. Let Δϕ̂ = σ2(β̂ − β0) = σ2Δβ̂.
Since Δβ̂ = õσP (1), relation (10) yields that
Q(β̂) − Q(β0)
σ4
= knΔϕ̂ + Vn(Δϕ̂)2 + õσP (1)
(
1 + ‖Δϕ̂‖2
)
� 0. (11)
Let c = lim infn→∞ λmin(Vn) > 0; c > 0, as follows from (v). Then, for n � n0, VnΔϕ̂2 �
c
2 ·‖Δϕ̂‖2 and ∀ ε > 0 ∃σε > 0 ∀σ ∈ (0, σε] ∃nε,σ ∀n � nε,σ : P (|õσP (1)| < c/4) > 1−ε.
Then (10) implies that, for σ ∈ (0, σε], n � max{n0, nε,σ},
c
4‖Δϕ̂‖2 + knΔϕ̂ + õσP (1) � 0
with probability at least 1 − ε. This implies Δϕ̂ = ÕσP (1).
3) Write expansions (10) for Δϕ̂ and Δϕ = zn and subtract them. We recall that
Δϕ̂ = ÕσP (1) and zn are some nonrandom bounded vectors. We obtain
Q(β̂) − Q
(
β0 + σ2zn
)
σ4
= knΔϕ̂ + Vn (Δϕ̂)2 − knzn − Vn (zn)2 + σ2ÕσP (1) =
= Vn (Δϕ̂ − zn)2 + (2Vnzn + kn) (Δϕ̂ − zn) + σ2ÕσP (1) � 0.
(12)
Let zn = − 1
2V −1
n kT
n . Then (12) changes into Vn (Δϕ̂ − zn)2 = σ2ÕσP (1), and condi-
tion (v) implies that Δϕ̂ − zn = σÕσP (1) = õσP (1).
4) Let Δϕ = zn + t, Δϕ̂ = zn + t̂, where t = σõσP (1). Subtract expansions (10) for
zn and Δϕ = zn + t.
L(t) : = σ−4
[
Q(β0 + σ2(zn + t)) − Q(β0 + σ2zn)
]
=
knt + Vn
(
(zn + t)2 − z2
n
)
+ σ2
[
pnt + Wn
(
(zn + t)2 − z2
n
)
+ Tn
(
(zn + t)3 − z3
n
)]
+
σ4[ R(zn + t) − R(zn)] + σ−4[rest(zn + t) − rest(zn)],
where jn := pn + 2Wnzn + 3Tnz2
n. Since zn + t = ÕσP (1), we have
rest(zn + t) − rest(z) = σ8õσP (1). R(zn + t) − R(zn) = õσP (1) by the definition of R.
Then L(t) has an expansion
L(t) = Vnt2 + σ2jnt + σ2
(
Wnt2 + 3Tnznt2 + Tnt3
)
+ σ4õσP (1). (13)
Prove that Δϕ̃ = ÕσP (1), i.e., t̂ = σ2ŝ, where ŝ = ÕσP (1). We have
Q(β̂)−Q(β0+σ2zn) = L(t̂) = σ4[Vnŝ2+jnŝ+
(
Wnŝ t̂ + 3Tnznŝ t̂ + Tnŝ2 t̂
)
+ õσP (1)] � 0.
Hence,
Vnŝ2 � −jnŝ + õσP (1)
(
1 + ‖ŝ‖2
)
,
and we have from (v) that ŝ = ÕσP (1).
5) The first leading term of the asymptotic deviation is σ2zn. Show that the second
leading term is σ4an. Note that an is a bounded nonrandom vector. Then (13) implies
L(σ2an) = σ4
(
Vna2
n + jnan + õσP (1)
)
.
120 GALYNA REPETATSKA
From (11) and ŝ = ÕσP (1), we obtain
L(σ2ŝ) = σ4
(
Vnŝ2 + jnŝ + õσP (1)
)
.
Subtracting these equalities, we have
L(σ2ŝ) − L(σ2an)
σ4
= Vn
(
ŝ2 − â2
n
)
+ jn(ŝ − an) + õσP (1) =
= Vn(ŝ − an)2 + (2Vnan + jn) (ŝ − an) + õσP (1) � 0.
Remember that an = − 1
2V −1
n jn, then we have the inequality
Vn (ŝ − an)2 � õσP (1),
whence ŝ = an + õσP (1). Theorem 2 is proved. �
Conclusion
We have found the second term of the asymptotic bias of the orthogonal regression
estimator. It is possible to find the subsequent terms in a similar way and then, with
sufficiently precise estimates, to construct more accurate estimators. The corrected es-
timators can be found for any nonlinear regression function in the way like that used in
the proof of Theorem 1.
The condition of normality of the error terms εi, δi is important for calculations. The
results can be extended to the non-normal case where the error terms have a symmetric
distribution with finite fourth-order moments. The deviation of the proposed estimators
is less than the deviation of β̂ for sufficiently small but fixed σ and n → ∞.
We intend to test the quality of the proposed estimators by simulations and to consider
an implicit regression model. In such models, there are no dependent and independent
variables, and xi and yi appear in a symmetric way, see [3].
Bibliography
1. I. Fazekas, A.G. Kukush, S. Zwanzig,, On inconsistency of the least squares estimator in non-
linear functional relations, Preprint, Department of Statistics and Demography, Odense Uni-
versity, Denmark, 1998.
2. I. Fazekas, A. Kukush, S. Zwanzig, Bias correction of nonlinear orthogonal regression, Ukr.
Math. Journ. 56 (2004), no. 8, 1101–1118.
3. A.G. Kukush, S. Zwanzig, About the adaptive minimum contrast estimator in a model with
non-linear functional relations, Ukr. Math. Journ. 53 (2001), no. 9, 1445–1452.
4. W.A. Fuller, Measurement errors models, Wiley, New York, 1987.
5. G.S. Repetatska, On inconsistency of orthogonal regression estimator in a nonlinear vector
error-in-variables model, Th. Prob. and Math. Stat. (to appear).
E-mail : galchonok@univ.kiev.ua
|
| id | nasplib_isofts_kiev_ua-123456789-4432 |
| institution | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| issn | 0321-3900 |
| language | English |
| last_indexed | 2025-12-07T18:47:57Z |
| publishDate | 2005 |
| publisher | Інститут математики НАН України |
| record_format | dspace |
| spelling | Repetatska, G. 2009-11-09T15:36:56Z 2009-11-09T15:36:56Z 2005 Modified orthogonal regression estimator in the quadratic errors-in-variables model / G. Repetatska // Theory of Stochastic Processes. — 2005. — Т. 11 (27), № 3-4. — С. 110–120. — Бібліогр.: 5 назв.— англ. 0321-3900 https://nasplib.isofts.kiev.ua/handle/123456789/4432 519.21 The quadratic functional measurement error model with equal error variances is
 considered. The asymptotic bias of an orthogonal regression estimator is derived. A
 modified estimator which has smaller asymptotic bias for small measurement errors
 is presented. en Інститут математики НАН України Modified orthogonal regression estimator in the quadratic errors-in-variables model Article published earlier |
| spellingShingle | Modified orthogonal regression estimator in the quadratic errors-in-variables model Repetatska, G. |
| title | Modified orthogonal regression estimator in the quadratic errors-in-variables model |
| title_full | Modified orthogonal regression estimator in the quadratic errors-in-variables model |
| title_fullStr | Modified orthogonal regression estimator in the quadratic errors-in-variables model |
| title_full_unstemmed | Modified orthogonal regression estimator in the quadratic errors-in-variables model |
| title_short | Modified orthogonal regression estimator in the quadratic errors-in-variables model |
| title_sort | modified orthogonal regression estimator in the quadratic errors-in-variables model |
| url | https://nasplib.isofts.kiev.ua/handle/123456789/4432 |
| work_keys_str_mv | AT repetatskag modifiedorthogonalregressionestimatorinthequadraticerrorsinvariablesmodel |