The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors
The nonlinear structural errors-in-variables model is investigated. We consider a Simex estimator with polynomial extrapolation function. The expansion of a Simex estimator is based on the asymptotic expansion of a naive estimator for small measurement errors. It is shown that the Simex estimator ha...
Saved in:
| Date: | 2008 |
|---|---|
| Main Authors: | , |
| Format: | Article |
| Language: | English |
| Published: |
Інститут математики НАН України
2008
|
| Online Access: | https://nasplib.isofts.kiev.ua/handle/123456789/4534 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Journal Title: | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| Cite this: | The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors / O. Gontar, H. Kuchenhoff // Theory of Stochastic Processes. — 2008. — Т. 14 (30), № 1. — С. 39–48. — Бібліогр.: 6 назв.— англ. |
Institution
Digital Library of Periodicals of National Academy of Sciences of Ukraine| _version_ | 1859585429112619008 |
|---|---|
| author | Gontar, O. Kuchenhoff, H. |
| author_facet | Gontar, O. Kuchenhoff, H. |
| citation_txt | The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors / O. Gontar, H. Kuchenhoff // Theory of Stochastic Processes. — 2008. — Т. 14 (30), № 1. — С. 39–48. — Бібліогр.: 6 назв.— англ. |
| collection | DSpace DC |
| description | The nonlinear structural errors-in-variables model is investigated. We consider a Simex estimator with polynomial extrapolation function. The expansion of a Simex estimator is based on the asymptotic expansion of a naive estimator for small measurement errors. It is shown that the Simex estimator has an asymptotic deviation from a true value of the unknown parameter which is negligible compared with a measurement error variance, while the deviation of the naive estimator is proportional to the measurement error variance.
|
| first_indexed | 2025-11-27T09:58:09Z |
| format | Article |
| fulltext |
Theory of Stochastic Processes
Vol. 14 (30), no. 1, 2008, pp. 39–48
UDC 519.21
OLENA GONTAR AND HELMUT KÜCHENHOFF
THE EXPANSION OF A SIMEX ESTIMATOR
IN THE NONLINEAR ERRORS-IN-VARIABLES
MODEL WITH SMALL MEASUREMENT ERRORS
The nonlinear structural errors-in-variables model is investigated. We consider a
Simex estimator with polynomial extrapolation function. The expansion of a Simex
estimator is based on the asymptotic expansion of a naive estimator for small mea-
surement errors. It is shown that the Simex estimator has an asymptotic deviation
from a true value of the unknown parameter which is negligible compared with a mea-
surement error variance, while the deviation of the naive estimator is proportional
to the measurement error variance.
Introduction
The regression problems where predictors are measured with additive errors are con-
sidered. We denote the response variable by y and the d-dimensional predictor by ξ
which cannot be observed. Instead, we observe x = ξ + σδγ, σδ > 0, where γ is a stan-
dard normal vector in Rd independent of ξ. The term σδγ is the measurement error. We
will estimate the unknown parameter vector β0 related to the distribution of (y, ξ). We
consider the structural case, so that {yi, xi, ξi}, i = 1, n, are independent and identi-
cally distributed. If one could observe xii, then we suppose that one could consistently
estimate β0 by solving the estimating equation
1
n
n∑
i=1
ψ(yi, ξi, β) = 0. This estimator
is usually called a naive estimator, when it is used in spite of measurement errors in
regressors, i.e. the naive estimator is a solution to
(1)
1
n
n∑
i=1
ψ(yi, xi, β) = 0.
Denote the naive estimator by β̂naive(σ2
δ ). Naive estimators are used to obtain a Simex
estimator. Simex is the simulation-based method of estimating and reducing the bias
due to a measurement error. The technique was proposed by Cook and Stefanski (1994).
The Simex procedure contains two main steps.
Simulation. Let Λ = {0, λ1, . . . , λM}, λk > 0, k = 1,M. For all i = 1, n, b = 1, B,
where B is the number of additional samples, the independent standard normal variables
ε∗i,b are generated. For each λ ∈ Λ, an additional measurement error is added to regres-
sors: xi,b(λ) = xi +
√
λσδε
∗
i,b, σδ > 0. Using xi,b(λ) instead of xi and taking the average
over b, a set of averaged naive estimators β̂naive((1 + λk)σ2
δ ), k = 1,M is constructed.
2000 AMS Mathematics Subject Classification. Primary 62J02, 62F10, 62F12.
Key words and phrases. Simex estimator, errors-in-variables models.
We would like to thank Prof. A. Kukush for the helpful and fruitful discussions of the Simex expansion
and the proofs of theorems, as well as for his valuable comments.
39
40 OLENA GONTAR AND HELMUT KÜCHENHOFF
Extrapolation. Let a parametric model which describes the dependence of the naive
estimator on the extra measurement error variance be G(λ,Γ), and let Γ̂ be an estimator
of Γ. The Simex estimator is defined as β̂Simex = G(−1, Γ̂).
It was shown that the Simex estimator is consistent if the model G(λ,Γ) corresponds
to the true model asymptotically [1]. Typically one uses the model G(λ,Γ) which is
different from the true model, but the Simex estimator shows better numerical results
for small and medium samples as compared with the consistent estimator Corrected
Score. It has been proposed to use the quadratic functions GQ(λ,Γ) = γ1 + γ2λ+ γ3λ
2
for extrapolation. Carroll et al. (1996) proved asymptotical the normality of Simex
estimators using the assumption that the exact dependence of a naive estimator from
the additional measurement error variance is known:
Eβ̂naive((1 + λ)σ2
δ ) = G(λ,Γ)
for a certain parameter value Γ. We consider the Simex estimator with polynomial ex-
trapolation function of any fixed degree. Using the asymptotic expansion of the naive
estimator, we will show that the Simex estimator has an asymptotic deviation from the
true value of the unknown parameter which is negligible as compared with a measure-
ment error variance, while the deviation of the naive estimator is proportional to the
measurement error variance.
We denote the Euclidean norm of a vector ξ by ‖ξ‖, the neighborhood of x of radius
r by B(x, r), the closure of B(x, r) by B(x, r), the identity matrix of order n by by In,
and a transposed vector x by xt.
The paper is organized as follows. We start with assumptions about the model and
the expansion of a naive estimator. Then the main result of the paper - the expansion of
the Simex estimator - is proved and applied to the exponential family and mean-variance
models, and we conclude by a discussion.
Assumptions
We assume that the regressors ξi ∈ R
d are independent identically distributed random
vectors and ∀λ ∈ R, Eeλ‖ξ‖ <∞. Suppose that the regressors are measured with error,
and xi = ξi + σδγi, σδ > 0, rather than ξi, are observed, where γi � N(0, Id), and
σ2
δ is known. The predictors yi are scalar variables. The variables ξi, γi are mutually
independent. We assume that β0 ∈ intK is a true value of the parameter β, where K is a
convex compact set in Rp. The function ψ : R×Rd×K → Rp is a vector function which
is smooth enough. The naive estimator β̂naive is defined as a solution of the estimating
equation
(2) Sn(β) :=
1
n
n∑
i=1
ψ(yi, xi, β) = 0.
We assume that ‖ψ(y, ξ, β)‖ ≤ k1e
k2‖ξ‖ and the derivative ‖ψβ(y, ξ, β)‖ ≤ k3e
k4‖ξ‖,
where ki ∈ R, i = 1, 4 are constants. The next convergence takes place:
(3) P{Sn(β)→ Eψ(y, x, β), uniformly in β ∈ K, as n→∞} = 1.
An analogous convergence was demonstrated by Schneeweiss and Kukush (2006) in the
proof of Theorem 4.1.
Denote F (β, σδ) = Eψ(y, x, β), β ∈ K. We obtain
(4) F (β, σδ) = Eψ(y, ξ + σδγ, β), β ∈ K.
Here, we allow σδ to be negative or equal to 0 since the distributions of σδγ and −σδγ
are identical.
THE EXPANSION OF A SIMEX ESTIMATOR 41
Expansion of a naive estimator
Hereafter, we use the assumptions of the general model stated above. The next theo-
rem makes it possible to expand a naive estimator for small measurement errors.
Theorem 1. Assume that the following conditions hold:
1 . The function ψ(y, ξ, β) ∈ C1(R× Rd × U → Rp), U ⊃ K, where U is open.
2 . For the function F (β, 0) = Eψ(y, ξ, β), there exists the unique solution β0 of the
equation F (β, 0) = 0 on the convex compact set K.
3 . The matrix V = Eψβ(y, ξ, β0) is nonsingular.
Then there exists σ > 0 such that, for all σδ ∈ B(0, σ), the equation F (β, σδ) = 0
has the unique solution βnaive(σδ) in K. Moreover, the function βnaive(σδ) is an even
function of σδ ∈ B(0, σ).
Proof. 1. The first and third conditions of the theorem allow us to use the implicit
function theorem (see Appendix A). According to this theorem, there exist δ1 > 0 and
ρ > 0 such that the equation F (β, σδ) = 0 has the unique solution
βnaive(σδ):B(0, δ1)→ B(β0, ρ).
The neighborhood B(β0, ρ) ⊂ K. The implicit function theorem states that βnaive(σδ) ∈
C1(B(0, δ1) → K). Note that the function F (β, σδ) is an even function of the second
variable. Indeed,
F (β,−σδ) = Eψ(y, ξ − σδγ, β) = Eψ(y, ξ + σδγ, β) = F (β, σδ).
Consider the equation F (β,−σδ) = 0 which is equivalent to F (β, σδ) = 0. The solution
to the equation F (β, σδ) = 0 is unique for σδ ∈ B(0, δ1) on K, so this implies that
βnaive(σδ) = βnaive(−σδ), and the function βnaive(σδ) is an even function for σδ ∈
B(0, δ1).
2. Consider the set K1 := K \ B(β0, ρ). The set K1 is a compact set as well. We
will prove that the function βnaive is the unique solution over the whole compact set
K. Consider the function F (β, 0). According to the second condition, it has the unique
solution β0 on the compact set K. Due to the continuity of F (β, 0), this means that
there exists a constant c > 0 such that ‖F (β, 0)‖ > c, for all β ∈ K1.
3. Admit that the function F (β, σδ) is continuous on the compact set K1 × B(0, δ1).
Then the function F (β, σδ) ⇒ F (β, 0) uniformly in β over K1 as σδ → 0. This means
that ∀ε > 0 ∃δ2 > 0 such that ∀σδ ∈ B(0, δ2) and ∀β ∈ K1, and the following inequality
holds: ‖F (β, σδ)− F (β, 0)‖ ≤ ε. This implies that
| ‖F (β, σδ)‖ − ‖F (β, 0)‖ |≤ ‖F (β, σδ)− F (β, 0)‖ ≤ ε.
We state that, for all β ∈ K1 and for all σδ ∈ B(0, δ2), ‖F (β, σδ)‖ ≥ ‖F (β, 0)‖−ε ≥ c−ε
holds. As ε can be chosen arbitrary, we set ε =
c
2
. Then ‖F (β, σδ)‖ ≥ c
2
> 0, σδ ∈
B(0, δ2).
4. We now set δ = min(δ1, δ2). Then, for all σδ ∈ B(0, δ), there exists the unique
solution βnaive(σδ) to the equation F (β, σδ) = 0 on the compact set K and ‖βnaive(σδ)−
β0‖ ≤ ρ. Theorem 1 is proved.
Theorem 2. Assume that conditions 2 and 3 of Theorem 1 hold and, for fixed l ≥ 1,
the function ψ(y, ξ, β) satisfies the following conditions:
1 . The score function ψ(y, ξ, β) ∈ C2l+2(R × Rd × U → Rp) with respect to ξ and β,
and U ⊃ K, where U is open.
2 . For any partial derivative Dqψ(y, ξ, β) of order q ≤ 2l+2 with respect to components
of ξ and components of β, ‖Dqψ(y, ξ, β)‖ ≤ c1ec2‖ξ‖, where c1, c2 are constants.
42 OLENA GONTAR AND HELMUT KÜCHENHOFF
Then there exists σ > 0 such that, for all σδ ∈ B(0, σ),
(5) β̂naive(σδ) = βnaive(σδ) + o(1) a.s., as n→∞,
where
(6) βnaive(σδ) = β0 +
l∑
j=1
β
(2j)
naive(0)
(2j)!
σ2j
δ +O(σ2l+2
δ ), as σδ → 0.
Remark 1. Below, we will write relations like (5) and (6) as
(7) β̂naive(σδ) = β0 +
l∑
j=1
β
(2j)
naive(0)
(2j)!
σ2j
δ +O(σ2l+2
δ )σδ→0 + o(1)n→∞.
Remark 2. Expansion (7) resembles the expansion of an orthogonal regression estimator
for the functional model from Fazekas et al. (2002). But the expansion (7) is much
simpler, since the naive estimator converges a.s., as n→∞, see (5), while the orthogonal
regression estimator for the functional model need not converge.
Proof. First, we note that Theorem 1 holds. The idea is to apply a Taylor expan-
sion to the function βnaive(σδ). From the first condition of Theorem 2, it follows that
F (β, σδ) ∈ C2l+2(Rd×U → R
p), and this implies that βnaive(σδ) ∈ C2l+2(R→ R
p). The
second condition of Theorem 2 and the condition Eeλ‖ξ‖ <∞ yield the boundedness of
β
(2l+2)
naive (σδ) for all σδ ∈ B(0, δ), where δ is defined from Theorem 1. Now we can use the
Taylor expansion for βnaive(σδ). As βnaive(σδ) is an even function, the Taylor expansion
of this function will have only summands of even powers. Thus,
βnaive(σδ) = β0 +
l∑
j=1
β
(2j)
naive(0)
(2j)!
σ2j
δ +O(σ2l+2
δ ), as σδ → 0.
Finally, it follows from (3) and Lemma 1 from Appendix A that β̂naive(σδ) = βnaive(σδ)+
o(1) a.s., as n→∞, and this proves the theorem. Theorem 2 is proved.
Remark 3. If l = 2, then βnaive(σδ) = β0 − V −1Kσ2
δ + O(σ4
δ ), as σδ → 0, where V is
defined in Theorem 1 and K = Eψξ(y, ξ, β0).
Simex with polynomial extrapolant function
We introduce the polynomial extrapolant function for a Simex estimator. We show
that the Simex with polynomial extrapolant has an asymptotic deviation from the true
value, which is negligible as compared with a measurement error variance.
Supplementary sample generation. Let Λ = {0, λ1, . . . , λM}, λk > 0, k = 1,M.
Let B be a large fixed natural number. For all i = 1, n and for all b = 1, B, standard
normal variables ε∗i,b � N(0, Id) are generated. For each λ ∈ Λ, an additional variance is
added to regressors xi,b(λ) = xi +
√
λσδε
∗
i,b. Here, σδ is a true standard deviation of the
measurement error. Now xi,b(λ) are used as new regressors.
Estimation. For each λ ∈ Λ averaged over b, naive estimators β̂naive(λ) are calcu-
lated.
Parametric model for naive estimators. Let the j-th coordinate of β̂naive(λ)
depends on λ by a polynomial law gj(λ) := γj0 +γj1λ+ · · ·+γjmλ
m, j = 1, p, and m ≥ 1
is fixed. Denote the extrapolant function G(λ,Γ) := (g1(λ), . . . , gp(λ))t. We estimate
the unknown parameter Γ by the method of least squares:
Γ̂ = arg min
Γ∈R(m+1)×p
1
M + 1
M∑
k=0
‖β̂naive(λk)−G(λk,Γ)‖2.
THE EXPANSION OF A SIMEX ESTIMATOR 43
Extrapolation. A Simex estimator is defined as β̂Simex = G(−1, Γ̂).
Theorem 3. Let Theorem 2 hold and l ≤ m ≤ M . Then the following expansion of
Simex is true:
β̂Simex = βSimex(σδ) + o(1) a.s., as n→∞,
where βSimex(σδ) = β0 +O(σ2l+2
δ ) as σδ → 0.
Proof. 1. It is not difficult to find the explicit form of Γ̂. We have G(λ,Γ) = s(λ)Γ,
where the p× (m+ 1)p matrix s(λ) is equal to
s(λ) =
⎛⎜⎜⎝
1 λ . . . λm 0 0 . . . 0 0 0 . . . 0
0 0 . . . 0 1 λ . . . λm 0 0 . . . 0
...
. . .
...
0 0 . . . 0 0 0 . . . 0 1 λ . . . λm
⎞⎟⎟⎠ ,
and Γ = (γ10, γ11, . . . , γ1m, γ20, γ21, . . . , γ2m, . . . , γp0, γp1, . . . , γpm)t. The least squares es-
timator of Γ equals
Γ̂ =
(
1
M + 1
M∑
k=0
st(λk)s(λk)
)−1
1
M + 1
M∑
k=0
st(λk)β̂naive(λk).
2. Note that the matrix inverse to the (m+1)p×(m+1)pmatrix
1
M + 1
M∑
k=0
st(λk)s(λk)
exists for all m ≤ M . To prove this, consider a discrete random variable ζ which takes
the values {0, λ1, . . . , λM} with equal probabilities 1/(M + 1). For the vector ψ =
(1, ζ, . . . , ζm)t, let us consider the matrix Eψψt. The matrix
1
M + 1
M∑
k=0
st(λk)s(λk)
is block-diagonal with (m + 1) × (m + 1) block Eψψt on diagonal p times. So the
existence of the matrix inverse to Eψψt is sufficient for the existence of the matrix
inverse to
1
M + 1
M∑
k=0
st(λk)s(λk). The Gram matrix Eψψt is nonsingular. Indeed, let
us suppose that there exists a �= 0, a ∈ Rm+1, such that atEψψta = E(atψ)2 = 0.
This means that atψ = amζ
m + am−1ζ
m−1 + · · · + a0 = 0 a.s. Therefore, for each
k = 0, 1, . . . ,M, amλ
m
k + am−1λ
m−1
k + · · · + a0 = 0. But this is impossible since, by
the conditions of Theorem 3, M + 1 > m. This contradiction proves that Eψψt is
nonsingular.
3. Consider the (m+ 1)p× p matrix Jr, r = 1,m+ 1 : the elements
(Jr)(j−1)(m+1)p+r,j = 1, j = 1, . . . , p,
and other elements are zero. The transpose matrix equals
J t
r :=
⎛⎜⎜⎝
0 1 . . . 0 0 0 . . . 0 0 0 . . . 0
0 0 . . . 0 0 1 . . . 0 0 0 . . . 0
...
. . .
...
0 0 . . . 0 0 0 . . . 0 0 1 . . . 0
⎞⎟⎟⎠ .
Denote s(−1) = s(λ)|λ=−1. The following equalities hold:
(8)
1
M + 1
M∑
k=0
st(λk)s(λk)Jr+1 =
1
M + 1
M∑
k=0
st(λk)λr
k, s(−1)Jr+1 = (−1)rIp, r = 0,m.
44 OLENA GONTAR AND HELMUT KÜCHENHOFF
4. Consider β̂naive((1+λ)σ2
δ ) =
l∑
j=0
β
(2j)
naive(0)
(2j)!
σ2j
δ
j∑
r=0
Cr
j λ
r +O(σ2j+2
δ )σ2
δ→0+o(1)n→∞.
Remember that l ≤ m, and, therefore, we can use (8) for r ≤ l.
We have
β̂Simex = s(−1)Γ̂ = s(−1)
(
1
M + 1
M∑
k=1
st(λk)s(λk)
)−1
1
M + 1
M∑
k=0
st(λk)β̂naive(λk)
= s(−1)
(
1
M + 1
M∑
k=0
st(λk)s(λk)
)−1
1
M + 1
M∑
k=0
st(λk)
l∑
j=0
β
(2j)
naive(0)
(2j)!
σ2j
δ
j∑
r=0
Cr
j λ
r
k
+O(σ2l+2
δ )σ2
δ→0 + o(1)n→∞
=
l∑
j=0
β
(2j)
naive(0)
(2j)!
σ2j
δ
j∑
r=0
Cr
j s(−1)
(
1
M + 1
M∑
k=0
st(λk)s(λk)
)−1
1
M + 1
M∑
k=0
st(λk)λr
k
+O(σ2l+2
δ )σ2
δ→0 + o(1)n→∞
=
l∑
j=0
β
(2j)
naive(0)
(2j)!
σ2j
δ
j∑
r=0
Cr
j (−1)r +O(σ2l+2
δ )σ2
δ→0 + o(1)n→∞
=
l∑
j=0
β
(2j)
naive(0)
(2j)!
σ2j
δ (1 − 1)j +O(σ2l+2
δ )σ2
δ→0 + o(1)n→∞
= β0 +O(σ2l+2
δ )σ2
δ→0 + o(1)n→∞,
and this proves the theorem.
Exponential Family
The regression model is described by a conditional distribution of y given ξ and given
an unknown parameter vector θ. We assume this distribution to be represented by a
probability density function
(9) f(y|ξ, β, ϕ) = exp
(
yη − c(η)
ϕ
+ a(y, ϕ)
)
with η = η(ξ, β),
where β is a regression parameter vector, and ϕ is a scalar dispersion parameter such that
θ = (βt, ϕ)t, and a, c, and η are known functions. The function c(·) is smooth enough,
and c′′(·) > 0. We assume that β0 ∈ intK is a true value of the parameter β, where K
is a convex compact set in R
p, and ϕ0 ∈ [a1, b1], a1 > 0, b1 < ∞, ϕ0 is a true value of
the parameter ϕ. If the variable ξ would be observable, one could estimate β and ϕ by
maximum likelihood. Consider the corresponding likelihood-score function for β in two
cases:
1. The dispersion parameter ϕ is known:
(10) ψ1(y, ξ, β) = (y − c′(η))ηβ .
2. The dispersion parameter ϕ is unknown:
(11) ψ2(y, ξ, β, ϕ) =
(
(y − c′(η))ηβ
(y − c′(η))2 − c′′(η)ϕ
)
.
The score function in both cases is unbiased. This implies, under natural regularity
conditions, that the estimators of the unknown parameters, obtained as solutions to the
THE EXPANSION OF A SIMEX ESTIMATOR 45
equation
1
n
n∑
i=1
ψ1(yi, ξi, β) = 0 or
1
n
n∑
i=1
ψ2(yi, ξi, β, ϕ) = 0, respectively, are consistent.
But we observe not ξ, but x that differs from the latent variable ξ by a measurement error
σδγ which is independent of ξ and y. We assume that γ � N(0, Id), and σ2
δ is known.
The naive estimator is obtained as a solution to the equation
1
n
n∑
i=1
ψ1(yi, xi, β) = 0 in
the case of the known dispersion parameter and
1
n
n∑
i=1
ψ2(yi, xi, β, ϕ) = 0 in the case of
the unknown dispersion parameter. The next corollary is true in both cases.
Corollary 1. Consider model (9). The naive estimator is calculated with the help of
(10) if the dispersion parameter ϕ is known and with the help of (11) if the dispersion
parameter is unknown. Let the following conditions hold for fixed l ≥ 1:
1. The function c(η) ∈ C2l+3(R) and, for all q ≤ 2l+3, the derivative |c(q)(η)| ≤ c1ec2‖ξ‖
for some constants c1, c2.
2. The function η(ξ, β) ∈ C2l+3(R) with respect to β, and, for all q ≤ 2l+ 3, any partial
derivative of order q with respect to components of β |η(q)(ξ, β)| ≤ c1e
c2‖ξ‖ for some
constants c1, c2.
3. The identifiability condition for the error-free model: the equation E(c′(η0)−c′(η))ηβ =
0, β ∈ K, where η0 = η(ξ, β0), has the unique solution β = β0.
4. The matrix Eηβ
0 (ηβ
0 )t is nonsingular, and, for each ξ, β and for some constants a1 and
a2, the inequality c′′(η) ≥ a1e
−a2‖ξ‖ holds.
Then, for l ≤ m ≤M , the expansion of Simex with polynomial extrapolant function is
true:
β̂Simex = βSimex(σδ) + o(1) a.s., as n→∞,
where βSimex(σδ) = β0 +O(σ2l+2
δ ), as σδ → 0. If ϕ is estimated, then
ϕ̂Simex = ϕSimex(σδ) + o(1) a.s., as n→∞,
where ϕSimex(σδ) = ϕ0 +O(σ2l+2
δ ), as σδ → 0.
Proof. In the case of the known dispersion parameter, the corollary follows directly from
Theorem 3. So we consider only the case of the unknown dispersion parameter. Introduce
a new parameter θ = (β, ϕ)t ∈ K × [a1, b1], and its true value θ0 = (β0, ϕ0)t. The naive
estimator for this parameter is obtained via the estimating function:
ψ2(y, x, θ) =
(
(y − c′(η))ηβ
(y − c′(η))2 − c′′(η)ϕ
)
.
The first two conditions of the corollary correspond to the conditions of Theorem 2. We
need to check the second and third conditions of Theorem 1. The second condition of
Theorem 1 states that the equation Eψ2(y, ξ, θ) = 0 has the unique solution θ = θ0 on
K × [a1, b1]. The equation
Eψ2(y, ξ, θ) =
(
E(c′(η0)− c′(η))ηβ
E(c′′(η0)ϕ0 − c′′(η)ϕ) + E(c′(η0)− c′(η))2
)
= 0
has the unique solution θ0 = (β0, ϕ0)t, if the third condition of Corollary 1 holds. The
last condition to be checked is the third condition of Theorem 1. It requires Eψθ
2(y, ξ, θ0)
to be nonsingular. Consider
Eψθ
2(y, ξ, θ0) = −
(
Ec′′(η0)η
β
0 (ηβ
0 )t 0
Ec′′′(η0)η
β
0ϕ0 Ec′′(η0)
)
.
46 OLENA GONTAR AND HELMUT KÜCHENHOFF
It is nonsingular, if the fourth condition of Corollary 1 holds. Then the next expansion
of the Simex estimator of θ is true: θ̂Simex = θSimex + o(1) a.s., as n → ∞, where
θSimex = θ0 +O(σ2l+2
δ ), as σδ → 0. And this is the statement of Corollary 1.
Application for the Gaussian model. Consider a nonlinear errors-in-variables model:{
yi = g(ξi, β0) + εi;
xi = ξi + δi;
i = 1, n.
The regressors ξi ∈ Rd are independent identically distributed random vectors, and
∀λ ∈ R, Eeλ‖ξ‖ <∞. The errors in regressor δi ∈ Rd are normal identically distributed
random vectors with zero expectation and the variance of σ2
δId. The errors in predictors
εi are normal independent identically distributed with zero expectation. The random
variables ξi, δi, εi are mutually independent.
Here, η(ξ, β) = g(ξ, β), c(η) = 1
2η
2, ϕ = σ2
ε . The corollary holds if the function g(ξ, β)
satisfies conditions 1 to 4 from Corollary 1, where g(ξ, β) is used instead of η(ξ, β).
Application for the Loglinear Poisson model. For the variable ξ, define λ = exp(ξtβ).
Then the loglinear Poisson model is defined as y ∼ P0(λ), where P0(λ) stands for the
Poisson distribution with intensity λ. Here, η = logλ, c(η) = eη, and ϕ = 1. The first
two conditions of Corollary 1 hold for any l ≥ 1. The third condition is equivalent to the
existence of a unique solution to the equation Eξξt(β0 − β) = 0, and this holds, while
Eξξt is nonsingular as a covariance matrix. The fourth condition requires Eexp(ξtβ0)ξξt
to be nonsingular, which is also true. Thus, Corollary 1 holds.
Corollary 1 holds also for the Gamma and Logit models. All the considered models
are with errors in the variables (concerning these models, see [2]).
Mean-Variance model
Suppose that a relation between the response variable y and the regressor ξ is given
by the conditional mean and the conditional variance:
(12) E(y|ξ) = m(ξ, β), var(y|ξ) = v(ξ, β, ϕ),
where ϕ is the scalar dispersion parameter. It is supposed that v(ξ, β, ϕ) > 0 for all
ξ, β, ϕ. We assume that the true value of the parameter β0 ∈ intK, where K is a
convex compact set in Rp and ϕ0 ∈ [a1, b1], a1 > 0, b1 < ∞, ϕ0 is the true value of
the parameter ϕ. We assume that the regressors ξi ∈ Rd are independent identically
distributed random vectors and ∀λ ∈ R, Eeλ‖ξ‖ < ∞. The specification of only the
mean and the variance in model (12) allows one to construct the consistent estimator of
β and ϕ. The conditionally unbiased estimating function is
(13) ψ(y, ξ, β, ϕ) =
(
(y −m(ξ, β))(v(ξ, β, ϕ))−1mβ(ξ, β)
(y −m(ξ, β))2 − v(ξ, β, ϕ).
)
If the dispersion parameter is known, we omit the second line in (13). In the case of
measurement errors, when we do not observe the latent ξ, but observe x, which equals
x = ξ + σδγ, the naive estimator is obtained from (1). We assume that γ � N(0, Id),
and σ2
δ is known.
Let us introduce a new parameter θ = (β, ϕ)t ∈ K × [a1, b1], and let its true value
θ0 = (β0, ϕ0)t.
Corollary 2. Assume that, for model (12), the next conditions hold for fixed l ≥ 1:
1. The function m(ξ, β) ∈ C2l+3(Rd ×R
p), and, for all q ≤ 2l+ 3, any partial derivative
of order q satisfies ‖m(q)(ξ, β)‖ ≤ c1ec2‖ξ‖ for some constants c1, c2.
THE EXPANSION OF A SIMEX ESTIMATOR 47
2. The function v(ξ, θ) ∈ C2l+3(Rd × Rp+1) with respect to components of ξ and θ, and,
for all q ≤ 2l + 3, any partial derivative of order q satisfies ‖v(q)(ξ, θ)‖ ≤ c1ec2‖ξ‖ for
some constants c1, c2, and v(ξ, θ) ≥ c3ec4‖ξ‖ for some constants c3, c4.
3. The identifiability condition for the error-free model is as follows: the system of equa-
tions {
E(m(ξ, β0)−m(ξ, β))(v(ξ, θ))−1mβ(ξ, β) = 0
E(v(ξ, θ0)− v(ξ, θ)) + E(m(ξ, β0)−m(ξ, β))2 = 0
, θ ∈ K × [a1, b1]
has the unique solution θ = θ0.
4. The matrices Emβ(ξ, β0)(mβ(ξ, β0))t and Evϕ(ξ, θ0) are nonsingular.
Then, for l ≤ m ≤ M , the expansion of the Simex estimator with polynomial extrap-
olant function is true:
θ̂Simex = θSimex(σδ) + o(1), a.s., as n→∞,
where θSimex(σδ) = θ0 +O(σ2l+2
δ ), as σδ → 0.
Proof. We need only to check the third condition of Theorem 1. Consider
Eψθ(y, ξ, θ0) = −
(
Emβ(ξ, β0)(mβ(ξ, β0))t(v(ξ, θ0))−1 0
vβ(ξ, θ0) vϕ(ξ, θ0)
)
.
This matrix is nonsingular due to the fourth condition of Corollary 2.
Discussion
We have given some theoretical reasons for good performance of the Simex estimator
for the polynomial extrapolant function. Our key idea was to use the Taylor expansion
of the estimator in σδ = 0. Therefore, one limitation of our result is that it can be
applied only to the case of small measurement errors. To illustrate this point, let us
consider the simplest linear model y = β1 + β2ξ with β1 = 0 and β2 = 1. We assume the
variance of ξ is known and equal to 1. In this case, the function βnaive(σδ) is obtained
explicitly: βnaive(σδ) =
1
1 + σ2
δ
. We take the Taylor expansion of the function βnaive(σδ)
as an extrapolation model for Simex, and the value of the y-intercept gives the Simex
estimator. In Fig. 1, the Taylor expansion of βnaive(σδ) up to the forth and sixth
powers of σδ was used. It can be seen that it works well for small values of σδ and does
not if measurement errors are not small. But the function βnaive(σδ) still can be well
approximated by a polynomial. Figure 2 shows such an approximation of βnaive(σδ) by
the polynomial 1.20769− 0.923077σδ + 0.215385σ2
δ, and Simex still works.
So the above theory is good for small measurement errors, and what is the behavior
of the Simex estimator in the case of large measurement errors should be investigated.
Appendix A
The Implicit Function Theorem. Let x ∈ Rn, y ∈ Rm, A be an open set in Rm+n,
and (x0, y0) ∈ A. Consider the function F : A→ Rn with the properties:
1. F (x0, y0) = 0.
2. F ∈ C1(A→ Rn).
3. F y(x0, y0) is non-singular.
Then ∃σ > 0, ∃ρ > 0, and there exists the unique function f : B(x0, σ) → B(y0, ρ)
with the following properties:
1.B(x0, σ)×B(y0, ρ) ⊂ A. 2.f(x0) = y0.
3.f ∈ C1(B(x0, σ)→ Rn) and f ′ = −(F y(x, f(x)))−1F x(x, f(x)), ∀x inB(x0, σ).
48 OLENA GONTAR AND HELMUT KÜCHENHOFF
0.5 1 1.5 2 ΣΔ
0.25
0.5
0.75
1
1.25
1.5
1.75
2
Βnaive
sixth
fourth
true
Figure 1
0.5 1 1.5 2 ΣΔ
0.25
0.5
0.75
1
1.25
1.5
1.75
2
Βnaive
square
true
Figure 2
Lemma 1. Let K be a compact set in Rn and Fn : K → Rn be nonrandom continuous
functions, F : K → Rn. Uniformly for θ in K, Fn(θ) → F (θ) as n → ∞. Suppose that
Fn(θn) = 0, and θ� is the unique solution to F (θ) = 0 on K. Then θn → θ�, as n→∞.
Bibliography
1. R.J.Carroll, H.Kuechenhoff, F.Lombard, and L.A.Stefanski, Asymptotics for the SIMEX Esti-
mator in Nonlinear Measurement Error Models, Journal of the American Statistical Association
91 (1996), no. 4, 242-250.
2. R.J.Carroll, D.Ruppert, L.A.Stefanski, and C.M.Crainiceanu,, Measurement Error in Nonlinear
Models, New York: Chapman and Hall, 2006.
3. J.R.Cook and L.A.Stefanski, Simulation-extrapolation estimation in parametric measurement
error models, Journal of the American Statistical Association 89 (1994), 1314-1328.
4. I.Fazekas, A.Kukush, and S.Zwanzig, Correction of nonlinear orthogonal regression estimator,
Ukrainian Mathematical Journal 56 (2004,), no. 8, 1308-1329.
5. A.Kukush and H.Schneeweiss, Comparing different estimators in a nonlinear measurement
error model I and II, Mathematical Method of Statistics 14 (2005), no. 1-2, 53-79, 203-223.
6. H.Schneeweiss and A.Kukush, Comparing the efficiency of structural and functional methods in
measurement error models, Discussion Paper 491, SFB 386, Ludwig - Maximilians - University
of Munich (2006).
)!� ; � � � % ���+� &�
%� $���
��
E-mail : gontaro@ukr.net; kuechenhoff@stat.uni-muenchen.de
|
| id | nasplib_isofts_kiev_ua-123456789-4534 |
| institution | Digital Library of Periodicals of National Academy of Sciences of Ukraine |
| issn | 0321-3900 |
| language | English |
| last_indexed | 2025-11-27T09:58:09Z |
| publishDate | 2008 |
| publisher | Інститут математики НАН України |
| record_format | dspace |
| spelling | Gontar, O. Kuchenhoff, H. 2009-11-25T11:02:19Z 2009-11-25T11:02:19Z 2008 The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors / O. Gontar, H. Kuchenhoff // Theory of Stochastic Processes. — 2008. — Т. 14 (30), № 1. — С. 39–48. — Бібліогр.: 6 назв.— англ. 0321-3900 https://nasplib.isofts.kiev.ua/handle/123456789/4534 519.21 The nonlinear structural errors-in-variables model is investigated. We consider a Simex estimator with polynomial extrapolation function. The expansion of a Simex estimator is based on the asymptotic expansion of a naive estimator for small measurement errors. It is shown that the Simex estimator has an asymptotic deviation from a true value of the unknown parameter which is negligible compared with a measurement error variance, while the deviation of the naive estimator is proportional to the measurement error variance. We would like to thank Prof. A. Kukush for the helpful and fruitful discussions of the Simex expansion and the proofs of theorems, as well as for his valuable comments. en Інститут математики НАН України The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors Article published earlier |
| spellingShingle | The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors Gontar, O. Kuchenhoff, H. |
| title | The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors |
| title_full | The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors |
| title_fullStr | The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors |
| title_full_unstemmed | The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors |
| title_short | The expansion of a Simex estimator in the nonlinear errors-in-variables model with small measurement errors |
| title_sort | expansion of a simex estimator in the nonlinear errors-in-variables model with small measurement errors |
| url | https://nasplib.isofts.kiev.ua/handle/123456789/4534 |
| work_keys_str_mv | AT gontaro theexpansionofasimexestimatorinthenonlinearerrorsinvariablesmodelwithsmallmeasurementerrors AT kuchenhoffh theexpansionofasimexestimatorinthenonlinearerrorsinvariablesmodelwithsmallmeasurementerrors AT gontaro expansionofasimexestimatorinthenonlinearerrorsinvariablesmodelwithsmallmeasurementerrors AT kuchenhoffh expansionofasimexestimatorinthenonlinearerrorsinvariablesmodelwithsmallmeasurementerrors |