Georgian Mathematical Journal Volume 10 (2003), Number 2, 271–288

GENERAL M-ESTIMATORS IN THE PRESENCE OF NUISANCE PARAMETERS. SKEW PROJECTION TECHNIQUE

N. LAZRIEVA AND T. TORONJADZE

Abstract. A multidimensional parametric filtered statistical model is considered, the notions of the regularity and ergodicity of the model, as well as the notion of a general M-estimator are studied. A skew projection technique is proposed, by which we construct an M-estimator of the parameter of interest in the presence of nuisance parameters, which is asymptotically normal with the zero mean and the same covariance matrix as the corresponding block of the asymptotic covariance matrix of the full parameter’s M-estimator.

2000 Mathematics Subject Classification: 65F12.

Key words and phrases: General M-estimator, nuisance parameter, projection.

1. Introduction

The well-known problem of the statistical estimation theory concerns the construction of estimators of parameter of interest in the presence of nuisance parameters. The projection technique is a frequently used for constructing efficient in the Fisher sense estimators in various observations (e.g., i.i.d., regression, etc.) schemes [1], [10], [3].

On the other hand, the key object in robust statistics is M-estimator, introduced by Huber [4] in the case of i.i.d. observations (see also [2], [8] for filtered statistical models). Hence to construct robust estimators in the presence of nuisance parameters it seems necessary to construct M-estimators in such a situation. For these purposes one needs to adopt the projection technique for M-estimators. As it will be shown below one need to use a skew projection of partial scores on the remainder terms of a full score (see (3.4)) rather than an orthogonal projection as in the case of MLE construction.

In the present paper a multidimensional parametric filtered statistical model is considered, notions of the regularity and ergodicity of models, a notion of general M-estimator are introduced (see Section 2).

In Section 3 a skew projection technique is proposed, which allows us to construct an estimator of a parameter of interest solving a d-dimensional equation (d is dimension of a parameter of interest) instead of a (d+m)-dimensional estimational equation for a M-estimator of the full (d + m)-dimensional parameter (see, e.g., (3.10) and (3.3), respectively, for d = 1).

ISSN 1072-947X / $8.00 / \begin{matrix}c \\ ◯ \end{matrix} Heldermann Verlag www.heldermann.de

In Theorem 3.1 we prove that under some regularity and ergodicity conditions the constructed estimator is asymptotically normal with the zero mean and the same covariance matrix as the corresponding (d × d)-dimensional block of the asymptotic covariance matrix of the M-estimator of the full parameter (see (3.32) for d = 1).

In Theorem 3.2 we establish asymptotic properties of the estimator of a parameter of interest constructed by the one-step approximation procedure, where the score martingale constructed by skew projection is used (see (3.33) for d = 1).

Appendix is devoted to formulation of the key Lemma which is the base of the proofs of Theorems.

2. Specification of the Model, Regularity, Ergodicity

We consider a filtered statistical model

E = (Ω, F, 𝔽 = (F_{t})_{t≥0}, {P_{θ}, θ ∈ Θ ⊂ R^{d+m}}, P ), d ≥ 1, m ≥ 1, (2.1)

where (Ω, F, 𝔽 = (F_{t})_{t≥0}, P) is a stochastic basis satisfying the usual conditions, {P_{θ}, θ ∈ Θ ⊂ R^{d+m}} is a family of probability measures on (Ω, F) such that P_{θ} ∼^{loc} P for each θ ∈ Θ, Θ is an open subset of R^{d+m}. Denote by

ρ(θ) = (ρ_{t}(θ) = \frac{d P_{θ,t}}{d P_{t}} )_{t≥0}

the density (likelihood) ratio process, where

P_{θ,t} = P_{θ} | F_{t}, P_{t} = P | F_{t}

are, respectively, the restriction of measures P_{θ} and P on the σ-algebra F_{t}. In the sequel we assume for simplicity that P_{θ,0} = P_{0} for all θ ∈ Θ. As is well-known [9], there exists M(θ) = (M_{t}(θ))_{t≥0} ∈ M_{loc}(P) such that

ρ(θ) = E(M(θ)),  (2.2)

where E(·) is the Dolean exponential, i.e.,

E_{t}(M(θ)) = exp \{M_{t}(θ) − \frac{1}{2}⟨M^{c}(θ)⟩_{t}\} ∏_{s≤t} (1 + ΔM_{s}(θ)) e^{−ΔM_{s}(θ)}

(for all unexplained notation see, e.g., [9], [5]).

Then

ln ρ(θ) = M(θ) − \frac{1}{2} ⟨M^{c}(θ)⟩ + ∑_{s≤·} ( ln(1 + ΔM_{s}(θ)) − ΔM_{s}(θ)).  (2.3)

Definition 2.1. We say that model (2.1) is regular if the following conditions are satisfied:

(1) for each (t, ω) the mapping θ ⇝ M_{t}(ω, θ) is differentiable and Ṁ(θ) ∈

GENERAL M-ESTIMATORS. SKEW PROJECTION TECHNIQUE 273

M_{loc}(P), where Ṁ(θ) = (\frac{∂}{∂θ_{1}} M(θ), …, \frac{∂}{∂θ_{d+m}} M(θ))′,^{1} (2) for each (t, ω) the mapping θ ⇝ ρ_{t}(ω, θ) is differentiable and

1In the sequel all vectors are assumed to be column-vectors, “′” is transposition sign.

\frac{∂}{∂θ} ln ρ(θ) = L (Ṁ(θ), M(θ)) ,

i.e., \frac{∂}{∂θ_{i}} ln ρ(θ) = L (\frac{∂}{∂θ_{i}} M(θ), M(θ)), where for m, M ∈ M_{loc}(P) L(m, M) is the Girsanov transformation defined as

L(m, M) : = m − (1 + ΔM)^{−1} ∘ [m, M]
= m − ⟨m^{c}, M^{c}⟩ − ∑_{s≤·} \frac{Δm_{s} ΔM_{s}}{1 + ΔM_{s}} ,  (2.4)

(3) L(Ṁ(θ), M(θ)) ∈ M_{loc}^{2}(P_{θ}).

Recall that if m ∈ M_{loc}(P), Q is some measure such that \frac{dQ}{dP} = E(M), then L(m, M) ∈ M_{loc}(Q) (see, e.g., [5]).

Define the (d + m) × (d + m)-matrix-valued Fisher information process and the Fisher information matrix as follows:

Î_{t}(θ) = ⟨L(Ṁ(θ), M(θ))⟩_{t}, t ≥ 0,
I_{t}(θ) = E_{θ} Î_{t}(θ), t ≥ 0,

where E_{θ} stands for expectation w.r.t. P_{θ}.

Remark 2.1. Consider the case where model (2.1) is associated with a kdimensional, k ≥ 1, 𝔽-adapted CADLAG process X = (X_{t})_{t≥0} in the following way: for each θ ∈ Θ P_{θ} is a unique measure on (Ω, F) such that X is a P_{θ}- semimartingale with the triplet of predictable characteristics (B(θ), C(θ), ν_{θ}) w.r.t. the standard truncation function h(x) = xI_{{|x|≤1}}, where | · | is the Euclidean norm in R^{k}.

In what follows we assume that P_{θ′} ∼^{loc} P_{θ} for each pair (θ, θ′) ∈ R^{d+m} × R^{d+m} and all measures P_{θ} coincide on F_{0}.

Fix some θ^{∗} ∈ Θ and let P := P_{θ^{∗}}, (B, C, ν) := (B(θ^{∗}), C(θ^{∗}), ν_{θ^{∗}}). Without loss of generality we can assume that there are a predictable process c = (c_{ij})_{i,j≤k} with values in the set of all symmetric nonnegative definite k × k-matrices and an increasing continuous 𝔽-adapted process A = (A_{t})_{t≥0} such that

C = c ∘ A P -a.s.

The following relationship between triplets (B(θ), C(θ), ν_{θ}) and (B, C, ν) is well known (see, e.g., [9]): there exists a P̃-measurable positive function Y (θ) = {Y (ω, t, x; θ) : (ω, t, x) ∈ Ω×R_{+} ×R^{k}} and a predictable k-dimensional process β(θ) = (β_{t}(θ))_{t≥0} with

|h(x)(Y (θ) − 1)| ∗ ν ∈ A_{loc}^{+}(P_{θ}),
(β′(θ)cβ(θ)) ∘ A ∈ A_{loc}^{+}(P_{θ})

such that (1) B(θ) = B + cβ(θ) ∘ A + h(x)(Y (θ) − 1) ∗ ν,

(2) C(θ) = C,

(3) ν_{θ} = Y (θ) · ν, and, in addition, a_{t} := ν({t}, R^{k}) = 1 ⇒ a_{t}(θ) := ν_{θ}({t}, R^{k}) = Ŷ_{t}(θ) = 1. Here for a given P̃-measurable function W, Ŵ_{t} = ∫ W (t, x)ν({t}, dx).

It is also well known (see, e.g., [6], [9]) that the P_{θ}-local martingale M(θ) in (2.2) can be writing explicitly as

M(θ) = β′(θ) · X^{c} + (Y (θ) − 1 + \frac{Ŷ(θ) − a}{1 − a} I_{(0<a<1)}) ∗ (μ − ν),  (2.5)

where X^{c} is the continuous martingale part of X under P.

Suppose that the following conditions are satisfied:

1) for all (ω, t, x) the functions β : θ ⇝ β_{t}(ω, θ) and Y : θ ⇝ Y (ω, t, x; θ) are differentiable (notation: β̇(θ) = \frac{∂}{∂θ} β(θ) = (β̇_{ij}(θ))_{i≤k, j≤d+m} with

β̇_{ij}(θ) = \frac{∂}{∂θ_{j}} β_{i}(θ), Ẏ (θ) = \frac{∂}{∂θ} Y (θ) = (Y^{(1)}(θ), …, Y^{(d+m)}(θ))′ with Y^{(j)}(θ) =

\frac{∂}{∂θ_{j}} Y (θ), 1 ≤ j ≤ d + m,

2) Ṁ(θ) := \frac{∂}{∂θ} M(θ) = β̇′(θ) · X^{c} + (Ẏ (θ) + \frac{ȧ(θ)}{1−a}) ∗ (μ − ν) ∈ M_{loc}(P),

3) \frac{∂}{∂θ} ln ρ(θ) = L (Ṁ(θ), M(θ)) ∈ M_{loc}^{2}(P_{θ}).

Then model (2.1) associated with the semimartingale X is regular in the sense of Definition 2.1.

It is not difficult to verify that

L(θ) := L (Ṁ, M(θ)) = β̇′(θ) · (X^{c} − cβ(θ) ∘ A) + Φ(θ) ∗ (μ − ν_{θ}),

where

Φ(θ) = \frac{Ẏ (θ)}{Y (θ)} + \frac{ȧ(θ)}{1 − a(θ)}

and I_{{a(θ)=1}}ȧ(θ) = 0.

Another definition of the regularity of a statistical model associated with the semimartingale X is given in [7]. Namely, suppose that the following conditions are satisfied: for each θ ∈ Θ there exist a predictable k × (d + m)-matrix-valued process β̇(θ) and a predictable (d + m)-dimensional vector function W (θ) with

|β̇′(θ) c β̇(θ)| ∘ A_{t} < ∞, |W (θ)|^{2} ∗ ν_{t} < ∞, t ≥ 0, P -a.s.

and such that for all t ≥ 0 we have

\begin{aligned}(1) & [ (β(θ′) − β(θ) − β̇(θ)(θ′ − θ))′ × \\ & c (β(θ′) − β(θ) − β̇(θ)(θ′ − θ)) ] ∘ A_{t}/ |θ′ − θ|^{2} →^{P_{θ}} 0, \\ (2) & (\sqrt{\frac{Y (θ′)}{Y (θ)}} − 1 − \frac{1}{2} W′(θ)(θ′ − θ))^{2} ∗ ν_{θ,t}/ |θ′ − θ|^{2} →^{P_{θ}} 0, \\ (3) & [ ∑_{\begin{matrix}s≤t \\ a_{s}(θ)<1 \end{matrix}} (\sqrt{1 − a_{s}(θ′)} − \sqrt{1 − a_{s}(θ)} \\ & + \frac{1}{2} \frac{Ŵ_{T}^{θ}}{\sqrt{1 − a_{s}(θ)}} (θ′ − θ))^{2}]/ |θ′ − θ| →^{P_{θ}}= 0 \end{aligned} (2.6)

as θ′ → θ, where Ŵ_{T}^{θ}(θ) = ∫ W (t, x, θ)ν_{θ}({t}, dx).

In this case as it is proved in [7] model (2.1) is regular in the following sense: at each point θ ∈ Θ the process \sqrt{\frac{ρ(θ′)}{ρ(θ)}} is locally (in time) differentiable w.r.t.

θ′ with derivative (d + m)-dimensional process L̃(θ) = (L̃_{t}(θ))_{t≥0} ∈ M_{loc}^{2}(P_{θ}) defined as follows:

L̃(θ) = β̇′(θ) · (X^{c} − cβ(θ) ∘ A) + (W (θ) + \frac{Ŵ^{θ}(θ)}{1 − a(θ)}) ∗ (μ − ν_{θ}).

The Fisher information (d + m) × (d + m)-matrix-valued process is defined as

Î(θ) := ⟨L̃(θ), L̃(θ)⟩ = (β̇′(θ)cβ(θ)) ∘ A + W (θ)W (∗θ)′ ∗ ν + ∑_{s≤·} \frac{Ŵ_{s}^{θ}(θ)Ŵ_{s}^{θ′}(θ)}{1 − a_{s}(θ)} .

Denote

Φ̃(θ) = W (θ) + \frac{Ŵ^{θ}(θ)}{1 − a(θ)} .

Then

L̃(θ) = β̇′(θ) · (X^{c} − cβ(θ) ∘ A) + Φ̃(θ) ∗ (μ − ν_{θ}).

It should be noticed that if the model is regular in both above-given senses, then

W (θ) = \frac{Ẏ (θ)}{Y (θ)} , Ŵ^{θ}(θ) = ȧ(θ), Φ(θ) = Φ̃(θ).

To preserve these notation for the second variant of regularity (regularity in the Jacod sense) let us formally define Ẏ (θ) and ȧ(θ) as

Ẏ (θ) = 2\sqrt{Y (θ)} \frac{∂}{∂θ} \sqrt{Y (θ)}, ȧ(θ) = −2\sqrt{1 − a(θ)} \frac{∂}{∂θ} \sqrt{1 − a(θ)} , (2.7)

where \frac{∂}{∂θ} \sqrt{Y (θ)} = W (θ)\sqrt{Y (θ)}, \frac{∂}{∂θ} \sqrt{1 − a(θ)} = −\frac{1}{2} \frac{Ŵ^{θ}(θ)}{\sqrt{1−a(θ)}}.

Then we have W (θ) = \frac{Ẏ (θ)}{Y (θ)}, Ŵ(θ) = ȧ(θ). Ṁ(θ) can also be defined formally as

Ṁ(θ) = β̇′(θ) · X^{c} + (Ẏ (θ) + \frac{ȧ(θ)}{1 − a(θ)}) ∗ (μ − ν),

where β̇(θ) is defined from (1) of (2.6) and Ẏ (θ) and ȧ(θ) from (2.7). In this case it is not difficult to verify that

L̃(θ) = L (Ṁ(θ), M(θ)) .

Many problems of asymptotic statistics have a natural setting in the scheme of series. We will now give the definition of the ergodicity of statistical models in terms of the scheme of series.

For this consider the scheme of series of regular statistical models

E =(E_{n})_{n≥1} =(Ω^{n}, F^{n}, 𝔽^{n} = (F_{t}^{n})_{0≤t≤T} , {P_{θ}^{n}, θ ∈ Θ ⊂ R^{d×m}}, P^{n})_{n≥1} , T >0,

where it is assumed that P_{θ}^{n} ∼ P^{n} for each n ≥ 1, d ≥ 1, m ≥ 1, and θ ∈ Θ.

Definition 2.2. We say that the series of filtered statistical models E = (E_{n})_{n≥1} is ergodic if there exists a numerical sequence (C_{n})_{n≥1} such that C_{n} > 0, lim_{n→∞} C_{n} = 0 and for each θ ∈ Θ ⊂ R^{d×m}

C_{n}^{2}Î_{T}^{n}(θ) →^{P_{θ}^{n}} I_{T} (θ), C_{n}^{2}I_{T}^{n}(θ) → I_{T} (θ) as n → ∞,

where I_{T} (θ) is a finite positive definite matrix. In the sequel the subscript “T ” will be omitted.

Denote ℓ_{j}^{n}(θ) = L(M^{(j)}^{n}(θ), M^{n}(θ)), where M^{(j)}^{n}(θ) = \frac{∂}{∂θ_{j}} M^{n}(θ), j ≤ d + m.

Obviously,

Î_{ij}^{n}(θ) = ⟨ℓ_{i}^{n}(θ), ℓ_{j}^{n}(θ)⟩, Î_{ij}^{n}(θ) = E_{θ}⟨ℓ_{i}^{n}(θ), ℓ_{j}^{n}(θ)⟩

and as n → ∞

C_{n}^{2}⟨ℓ_{i}^{n}(θ), ℓ_{j}^{n}(θ)⟩ →^{P_{θ}^{n}} I_{ij}(θ), C_{n}^{2}E_{θ}⟨ℓ_{i}^{n}(θ), ℓ_{j}^{n}(θ)⟩ → I_{ij}(θ), i, j = \overline{1, d + m}.

M-estimators. If model (2.1) is regular, the maximum likelihood (ML) equation (w.r.t. (θ)) takes the form

L_{t} (Ṁ(θ), M(θ)) = 0, t ≥ 0.

Let {m(θ), θ ∈ Θ ⊂ R^{d+m}} be a family of (d + m)-dimensional P -local martingales such that L(m(θ), M(θ)) ∈ M_{loc}^{2}(P_{θ}). Consider the following (estimational) equation w.r.t. θ:

L (m(θ), M(θ)) = 0.  (2.8)

Definition 2.3. Any solution (θ_{t})_{t≥0} of equation (2.8) is called an Mestimator.

To preserve the classical terminology we will say that the family {m(θ), θ ∈ Θ} of P -martingales define the M-estimator, and P_{θ}-martingale L(m(θ), M(θ)) is an influence (score) martingale.

We say that the family {m(θ), θ ∈ Θ} is regularly related to the regular model (2.1) if the following conditions are satisfied:

(1) for all (ω, t) the mapping θ ⇝ m(ω, t, θ) is continuously differentiable

and m^{(i)}(θ) ∈ M_{loc}(P), where m^{(i)}(θ) = \frac{∂}{∂θ_{i}} m(θ). Denote ṁ(θ) =

(m^{(1)}(θ), …, m^{(d+m)}(θ))′.

(2) for all (ω, t) the mapping θ ⇝ L_{t}(m(θ), M(θ)) is continuously differentiable and

\frac{∂}{∂θ} L(m(θ), M(θ)) = L (ṁ(θ), M(θ))
− [L(m(θ), M(θ)), L (Ṁ(θ), M(θ))] ,

where \frac{∂}{∂θ} L(m(θ), M(θ)) := (L^{(1)}(m(θ), M(θ)), …, L^{(d+m)}(m(θ), M(θ)))′ is

a (d + m) × (d + m) matrix-valued process,

L^{(i)}(m(θ), M(θ)) = \frac{∂}{∂θ_{i}} L(m(θ), M(θ))

and for M, N ∈ M_{loc}(P), [M, N] = ⟨M^{c}, N^{c}⟩ + ∑_{s≤·} ΔM_{s}ΔN_{s}, M^{c} is

the continuous martingale part of M. Note that \frac{∂}{∂θ_{i}} L(m_{j}(θ), M(θ)) =

L(m^{(i)}_{j}(θ), M(θ)) − [L(m_{j}(θ), M(θ)), L(M^{(i)}(θ), M(θ))].

3. Skew Projection Technique

In this section, for exposition simplicity, we consider the case d = 1, Θ = R^{m+1}.

Consider the series of regular statistical models

E = (E^{n})_{n≥1} = (Ω^{n}, F^{n}, 𝔽^{n} = (F_{t}^{n})_{t∈[0,T]}, {P_{θ}^{n}, θ ∈ R^{m+1}}, P^{n})_{n≥1},

m ≥ 1, T > 0.

Represent θ ∈ R^{m+1} in the form θ = (θ_{1}, \underline{θ} ′_{2})′ where θ_{1}, θ_{1} ∈ R^{1}, is a parameter of interest, \underline{θ}_{2} = (θ_{2}, …, θ_{m+1})′ is a nuisance parameter.

Let for each n ≥ 1 {m^{n}(θ), θ ∈ R^{m+1}} be a family of (m + 1)-dimensional P^{n}-martingales regularly related to the model E^{n}.

Denote

h_{i}^{n}(θ) = L (m_{i}^{n}(θ), M^{n}(θ)) , n ≥ 1, i = \overline{1, m + 1}.

In the sequel we assume that for each θ ∈ R^{m+1}, h_{i}^{n}(θ) ∈ M_{loc}^{2}(P_{θ}^{n}), i = \overline{1, m + 1}, n ≥ 1, and the following ergodicity conditions are satisfied:

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}⟨h_{i}^{n}(θ), h_{j}^{n}(θ)⟩ = Γ_{ij}(θ),  (3.1)
P_{θ}^{n}- lim_{n→∞} C_{n}^{2}⟨h_{i}^{n}(θ), ℓ_{j}^{n}(θ)⟩ = γ_{ij}(θ),  (3.2)

i, j = \overline{1, m + 1}, where Γ(θ) = (Γ_{ij}(θ)) and \frac{γ(θ)+γ′(θ)}{2} with γ(θ) = (γ_{ij}(θ)) are finite positive matrices continuous in θ.

We assume also that the Lindeberg condition L is satisfied:

L. for each ε > 0

P_{θ}^{n}- lim_{n→∞} ∫∫_{0}^{T} |x|^{2} I_{{|x|>ε}}ν_{θ}^{n}(ds, dx) = 0,

where ν_{θ}^{n} is a P_{θ}^{n}-compensator of the jump measure of the 2(m + 1)- dimensional P_{θ}^{n}-martingale C_{n}(L(Ṁ^{n}(θ), M^{n}(θ)), L(m^{n}(θ), M^{n}(θ))).

Further, consider the following estimational equation

L(m^{n}(θ), M^{n}(θ)) = 0, n ≥ 1.  (3.3)

Under suitable regularity conditions it can be proved (see Lemma A from the Appendix) that there exists a sequence (θ_{n})_{n≥1} of asymptotic solutions of equation (3.3) (M-estimators) such that

L{C_{n}^{−1}(θ_{n} − θ) | P_{θ}^{n}} ⇒ N(0, γ^{−1}(θ)Γ(θ)(γ^{−1}(θ))′) as n → ∞.

The skew projection technique proposed in this section allows us to construct an estimator (\overline{θ}_{1,n})_{n≥1} of the parameter of interest θ_{1}, solving a one-dimensional equation (see equation (3.10) below), which is asymptotically normal with zero mean and the same variance as the first component of the (m + 1)-dimensional estimator (θ_{n})_{n≥1} of θ, i.e.,

L\{C_{n}^{−1}(\overline{θ}_{1,n} − θ_{1}) | P_{θ}^{n}\} ⇒ N (0, (γ^{−1}(θ)Γ(θ) (γ^{−1}(θ))′)_{11}) .

Under the skew projection we mean the projection of h_{1}^{n}(θ) onto \underline{h} _{2}^{n}(θ) = (h_{2}^{n}(θ), …, h_{m+1}^{n}(θ))′ in the direction defined by the relations

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}⟨h_{1}^{n}(θ) − b′\underline{h} _{2}^{n}(θ), ℓ_{j}^{n}(θ)⟩ = 0, j = \overline{2, m + 1},  (3.4)

where b ∈ R^{m}.

(3.4) implies that

b′ = \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ),

where \underline{γ}_{12}(θ) and \underline{γ}_{22}(θ) correspond to the partition of γ(θ):

γ(θ) = (\begin{matrix}γ_{11}(θ) & \underline{γ}_{12}(θ) \\ \underline{γ}_{21}(θ) & \underline{γ}_{22}(θ) \end{matrix}) .

Remark 3.1. Roughly speaking, the direction defined by (3.4) is orthogonal to the linear space spanned by \underline{ℓ} _{2}^{n}(θ) = (ℓ_{2}^{n}(θ), …, ℓ_{m+1}^{n}(θ))′ rather than by \underline{h} _{2}^{n}(θ).

In the case of MLE (i.e., when h_{i}^{n}(θ) = ℓ_{i}^{n}(θ), i = \overline{1, m + 1}) we come to the well-known orthogonal projection of ℓ_{1}^{n}(θ) onto \underline{ℓ} _{2}^{n}(θ) (see, e.g., [10]) and arrive to the efficient score function for θ_{1}:

ℓ_{1}^{n}(θ) − \underline{I}_{12}(θ)\underline{I} _{22}^{−1}(θ)\underline{ℓ} _{2}^{n}(θ),

where \underline{I}_{12}(θ) and \underline{I}_{22}(θ) correspond to the partition of

I(θ) = (\begin{matrix}I_{11}(θ) & \underline{I}_{12}(θ) \\ \underline{I}_{21}(θ) & \underline{I}_{22}(θ) \end{matrix}) .

Denote

\begin{aligned}H_{n}(θ) & = h_{1}^{n}(θ) − \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ)\underline{h} _{2}^{n}(θ), \\ n^{n}(θ) & = m_{1}^{n}(θ) − \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ)\underline{m} _{2}^{n}(θ), \end{aligned}  (3.5)

where \underline{m} _{2}^{n}(θ) = (m_{2}^{n}(θ), … m_{m+1}^{2}(θ))′. Then

H_{n}(θ) = L(n^{n}(θ), M^{n}(θ)).  (3.6)

It should be noted that

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}⟨H_{n}(θ), \underline{ℓ} _{2}^{n}(θ)⟩ = 0.  (3.7)

Remark 3.2. One can consider the case when θ ∈ Θ ⊂ R^{d+m}, and assume that θ = (\underline{θ}_{1}, \underline{θ}_{2}), where \underline{θ}_{1} = (θ_{1}, …, θ_{d})′ is a parameter of interest, \underline{θ}_{2} = (θ_{d+1}, …, θ_{d+m})′ is a nuisance parameter.

In this case

H_{n}(θ) = \underline{h} _{1}^{n}(θ) − \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ)\underline{h} _{2}^{n}(θ),  (3.8)

where \underline{h} _{1}^{n}(θ) = (h_{1}^{n}(θ), …, h_{d}^{n}(θ))′, \underline{h} _{2}^{n}(θ) = (h_{d+1}^{n}(θ), …, h_{d+m}^{n}(θ))′ and \underline{γ}_{12}(θ), \underline{γ}_{22}(θ) and \underline{γ}_{21}(θ) corresponds to the partition of γ(θ) = (γ_{ij}(θ))_{i,j=\overline{1,d+m}},

γ(θ) = ( \begin{matrix}d×d & d×m \\ \underline{γ}_{11}(θ) & \underline{γ}_{12}(θ) \\ \underline{γ}_{21}(θ) & \underline{γ}_{22}(θ) \\ m×d & m×m \end{matrix} ).

Now let (\underline{θ}_{2,n})_{n≥1} be a C_{n}-consistent estimator of \underline{θ}_{2}, i.e.,

C_{n}^{−1}(\underline{θ}_{2,n} − \underline{θ}_{2}) = O_{P_{θ}^{n}}(1).  (3.9)

Consider the following estimational equation (w.r.t. θ_{1})

H_{n}(θ_{1}, \underline{θ}_{2,n}) = 0.  (3.10)

Our goal is to study the problem of solvability of equation (3.10) as well as the asymptotic properties of solutions.

Introduce the conditions:

(1) For each n ≥ 1 the family {m^{n}(θ), θ ∈ R^{m+1}} is regularly related to E^{n}.

(2) The mapping θ ⇝ \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ) is continuously differentiable.

(3) For any θ ∈ R^{m+1} and x ∈ R^{1} there exists a function Δ(x, θ) such that

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}H_{n}(x, \underline{θ}_{2,n}) = Δ(x, θ)

and the equation (w.r.t. x)

Δ(x, θ) = 0

has a unique solution θ_{1}^{∗} = b(θ) ∈ R^{1}.

(4) For each θ ∈ R^{m+1}

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}L_{n}(n^{(i)}^{n}(θ), M^{n}(θ)) = 0, i = \overline{1, m + 1},

where n^{(i)}^{n}(θ) = \frac{∂}{∂θ_{i}} n^{n}(θ).

(5) For any θ ∈ R^{m+1}, N > 0 and ρ > 0

\overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{y:|y−\underline{θ}_{2}|≤C_{n}N} C_{n}^{2}|H^{(i)}_{n}(θ_{1}, y) − H^{(i)}_{n}(θ_{1}, \underline{θ}_{2})| > ρ\} = 0, i = \overline{1, m + 1}.

(6) For any θ ∈ R^{m+1}, N > 0 and ρ > 0

\overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{\begin{matrix}x∈R^{1}, y∈R^{m} \\ |x−θ_{1}|≤r \\ |y−\underline{θ}_{2}|≤C_{n}N \end{matrix}} C_{n}^{2}|H^{(1)}_{n}(x, y) − H^{(1)}_{n}(θ_{1}, y)| > ρ\} = 0.

Remark 3.3. If L(n^{(i)}^{n}(θ), M^{n}(θ)) ∈ M_{loc}^{2}(P_{θ}^{n}), then condition (4) will be satisfied if for any θ ∈ R^{m+1}, N > 0 and ρ > 0

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}⟨L(n^{(i)}^{n}(θ), M^{n}(θ))⟩ = 0

(see, e.g., [6]).

Theorem 3.1. Let conditions (1)–(6) be satisfied. Then for each θ ∈ R^{m+1}, θ = (θ_{1}, \underline{θ} ′_{2})′, there exists a sequence (\overline{θ}_{1,n})_{n≥1} of random variables such that

I. lim_{n→∞} P_{θ}^{n}\{H_{n}(\overline{θ}_{1,n}, \underline{θ}_{2,n}) = 0\} = 1,

II. P_{θ}^{n}- lim_{n→∞} \overline{θ}_{1,n} = θ_{1},

III. if there exists another sequence θ̃_{1,n} with properties I, II, then

lim_{n→∞} P_{θ}^{n}\{θ̃_{1,n} = \overline{θ}_{1,n}\} = 1,

IV. if the sequence of distributions L{C_{n}H_{n}(θ) | P_{θ}^{n}} converges weakly to a certain distribution Φ, then

L\{γ_{p}(θ)C_{n}^{−1}(\overline{θ}_{1,n} − θ_{1}) | P_{θ}^{n}\} ⇒ Φ,

where

γ_{p}(θ) = γ_{11}(θ) − \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ)\underline{γ}_{21}(θ) > 0.  (3.11)

Proof. Keeping Remark A of Appendix in mind refer to Lemma A and put ℓ = m + 1, k = 1, Q_{θ}^{n} = P_{θ}^{n}, L_{n}(θ) = L_{n}(θ_{1}) = H_{n}(θ_{1}, \underline{θ}_{2,n}), θ = (θ_{1}, \underline{θ} ′_{2})′.

Now in order to prove the theorem we have to verify that all conditions of Lemma A are satisfied (for L_{n}(θ_{1})).

Since L_{n}(θ_{1}) = H_{n}(θ_{1}, \underline{θ}_{2,n}) the continuous differentiability of L_{n}(θ_{1}) in θ_{1} is a trivial consequence of (3.8), conditions (1) and (2). Hence condition b) of Lemma A is satisfied.

Further, let us show that for any θ ∈ R^{m+1}, θ = (θ_{1}, \underline{θ} ′_{2})′,

P_{θ}^{n}- lim_{n→∞} C_{n}(L_{n}(θ_{1}) − H_{n}(θ)) = 0.  (3.12)

Using the Taylor expansion, we obtain

C_{n}^{2}(L_{n}(θ_{1}) − H_{n}(θ)) = C_{n}(H_{n}(θ_{1}, \underline{θ}_{2,n}) − H_{n}(θ_{1}, \underline{θ}_{2}))

= C_{n}^{2}H^{∗}′_{n}(θ)C_{n}^{−1}(\underline{θ}_{2,n} − \underline{θ}_{2}) + C_{n}^{2}(H^{∗}′_{n}(θ_{1}, v_{n}) − H^{∗}′_{n}(θ_{1}, \underline{θ}_{2}))C_{n}^{−1}(\underline{θ}_{2,n} − \underline{θ}_{2}),

where H^{∗}_{n}(θ) = (H^{(2)}_{n}(θ), …, H_{n}^{(m+1)}(θ)), v_{n} = \underline{θ}_{2} + α(\underline{θ}_{2,n} − \underline{θ}_{2}), random variable

α ∈ [0, 1].

Consequently, since C_{n}^{−1}(\underline{θ}_{2,n} −\underline{θ}_{2}) = O_{P_{θ}^{n}}(1) for (3.12) it is sufficient to show that

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}H^{∗}_{n}(θ) = 0  (3.13)

and

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}(H^{∗}_{n}(θ_{1}, v_{n}) − H^{∗}(θ_{1}, \underline{θ}_{2})) = 0.  (3.14)

From (3.6) and conditions (1), (2) we have

C_{n}^{2}H^{∗}_{n}(θ) = C_{n}^{2}L(n^{∗}^{n}(θ), M^{n}(θ)) − C_{n}^{2}[H_{n}(θ), \underline{ℓ} _{2}^{n}(θ)].

Condition L and (3.7) ensure that (see, e.g., [6])

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}[H_{n}(θ), \underline{ℓ} _{2}^{n}(θ)] = P_{θ}^{n}- lim_{n→∞} C_{n}^{2}⟨H_{n}(θ), \underline{ℓ} _{2}^{n}(θ)⟩ = 0.

Now (3.13) directly follows from condition (4). As for (3.14) observe that for any N > 0 and ρ > 0

\overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{2}|H^{∗}_{n}(θ_{1}, v_{n}) − H^{∗}_{n}(θ_{1}, \underline{θ}_{2})| > ρ\}

= \overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{2}|H^{∗}_{n}(θ_{1}, v_{n}) − H^{∗}_{n}(θ_{1}, \underline{θ}_{2})| > ρ, C_{n}^{−1}|\underline{θ}_{2,n} − \underline{θ}_{2}| ≤ N\}

+ \overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{2}|H^{∗}_{n}(θ_{1}, v_{n}) − H^{∗}_{n}(θ_{1}, \underline{θ}_{2})| > ρ, C_{n}^{−1}|\underline{θ}_{2,n} − \underline{θ}_{2}| > N\}

≤ \overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{y:|y−\underline{θ}_{2}|≤C_{n}N} C_{n}^{2}|H^{∗}_{n}(θ_{1}, v_{n}) − H^{∗}_{n}(θ_{1}, \underline{θ}_{2})| > ρ\}

+ \overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{−1}|\underline{θ}_{2,n} − \underline{θ}_{2}| > N\}.

From this inequality using condition (5) we obtain

\overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{y:|y−\underline{θ}_{2}|≤C_{n}N} C_{n}^{2}|H^{∗}_{n}(θ_{1}, v_{n}) − H^{∗}(θ_{1}, \underline{θ}_{2})| > ρ\}

≤ \overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{−1}|\underline{θ}_{2,n} − \underline{θ}_{2}| > N\}.  (3.15)

Letting N → ∞ in (3.15) and taking into account (3.9) we get (3.14). Thus (3.12) is proved.

From (3.12) it follows that

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}L_{n}(θ_{1}) = P_{θ}^{n}- lim_{n→∞} C_{n}^{2}H_{n}(θ).  (3.16)

On the other hand, by using the ergodicity conditions (3.1) and (3.2) one can easily check that

P_{θ}^{n}- lim_{n→∞}⟨C_{n}H_{n}(θ)⟩ = P_{θ}^{n}- lim_{n→∞} C_{n}^{2}⟨H_{n}(θ)⟩ = Γ_{p}(θ),  (3.17)

where

Γ_{p}(θ) = Γ_{11}(θ) − 2\underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ)\underline{Γ} ′_{12}(θ)
+ \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}\underline{Γ}_{22}(θ)(\underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ))′,  (3.18)

\underline{Γ}_{12}(θ) and \underline{Γ}_{22}(θ) correspond to the partition of Γ(θ):

Γ(θ) = (\begin{matrix}Γ_{11}(θ) & \underline{Γ}_{12}(θ) \\ \underline{Γ}_{21}(θ) & \underline{Γ}_{22}(θ) \end{matrix}) .

Recall that \underline{Γ} ′_{12}(θ) = \underline{Γ}_{21}(θ).

From (3.17) it follows that

P_{θ}^{n}- lim_{n→∞}⟨C_{n}^{2}H_{n}(θ)⟩ = P_{θ}^{n}- lim_{n→∞} C_{n}^{4}⟨H_{n}(θ)⟩ = 0,

from which we obtain that (see, e.g., [9])

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}H_{n}(θ) = 0.  (3.19)

Now combining (3.12) and (3.19) we get

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}L_{n}(θ_{1}) = Δ(θ_{1}, θ) = 0

and in view of condition (3) we aventually conclude that condition c) of Lemma A is satisfied with θ_{1}^{∗}(θ) = θ_{1}.

The next step is to verify condition d) of Lemma A.

By arguments similar to those we have used in the proof of (3.12) one can show that

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}(L̇_{n}(θ_{1}) − H^{(1)}_{n}(θ)) = 0.  (3.20)

Indeed, for any ρ > 0 we have

P_{θ}^{n}{C_{n}^{2}|L̇_{n}(θ_{1}) − H^{(1)}_{n}(θ)| > ρ}

= P_{θ}^{n}\{C_{n}^{2}|H^{(1)}_{n}(θ_{1}, \underline{θ}_{2}, n) − H^{(1)}_{n}(θ_{1}, \underline{θ}_{2})| > ρ\}

≤ P_{θ}^{n}\{ sup_{|y−\underline{θ}_{2}|≤C_{n}N} C_{n}^{2}|H^{(1)}_{n}(θ_{1}, y) − H^{(1)}_{n}(θ_{1}, \underline{θ}_{2})| > ρ\}

+ P_{θ}^{n}\{C_{n}^{−1}|\underline{θ}_{2,n} − \underline{θ}_{2}| > N\}.

Therefore (3.14) follows from conditions (5) and (3.9).

At the same time, by

H^{(1)}_{n}(θ) = L(n^{(1)}^{n}(θ), M^{n}(θ)) − [H_{n}(θ), ℓ_{1}^{n}(θ)]

condition (4) together with condition L imply

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}H^{(1)}_{n}(θ) = −P_{θ}^{n}- lim_{n→∞} C_{n}^{2}[H_{n}(θ), ℓ_{1}^{n})]

= −P_{θ}^{n}- lim_{n→∞} C_{n}^{2}⟨h_{1}^{n}(θ) − \underline{γ}_{12}\underline{γ} _{22}^{−1}\underline{h} _{2}^{n}, ℓ_{1}^{n}⟩

= γ_{p}(θ),

where γ_{p}(θ) > 0 is defined by (3.11). In view of this equality we conclude that

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}L̇_{n}(θ_{1}) = −γ_{p}(θ) < 0.  (3.21)

Thus condition d) of Lemma A is satisfied.

It remains to check condition e) of Lemma A.

For any ρ > 0, r > 0 and N > 0 we have

\overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{|x_{1}−θ_{1}|≤r} C_{n}^{2}|L̇_{n}(x_{1}) − L̇_{n}(θ_{1})| > ρ\}

= \overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{|x_{1}−θ_{1}|≤r} C_{n}^{2}|H^{∗}_{n}(x_{1}, \underline{θ}_{2,n}) − H^{∗}_{n}(θ_{1}, \underline{θ}_{2,n})| > ρ\}
≤ \overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{\begin{matrix}|x_{1}−θ_{1}|≤r \\ |y−\underline{θ}_{2}|≤C_{n}N \end{matrix}} C_{n}^{2}|H^{∗}_{n}(x_{1}, y) − H^{∗}_{n}(θ_{1}, y)| > ρ\}

+ \overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{−1}|\underline{θ}_{2,n} − \underline{θ}_{2}| > N\},

which in view of (3.9) and condition (6) implies that

lim_{r→0} \overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{|x_{1}−θ_{1}|≤r} |L̇_{n}(x_{1}) − L̇_{n}(θ_{1})| > ρ\} = 0.

Thus condition e) of Lemma A is also satisfied. □

Corollary 3.1. Under the conditions of Theorem 3.1

L\{C_{n}^{−1}(\overline{θ}_{1,n} − θ_{1}) | P_{θ}^{n}\} ⇒ N(0, Γ_{p}(θ)/γ_{p}^{2}(θ)),  (3.22)

where γ_{p}(θ) and Γ_{p}(θ) are defined by (3.11) and (3.18), respectively.

Proof. Using CLT for locally square integrable martingales [6], from (3.17), condition L we have

L{C_{n}H_{n}(θ) | P_{θ}^{n}} ⇒ N(0, Γ_{p}(θ)),  (3.23)

from which, taking into account (3.12), we get

L{C_{n}L_{n}(θ_{1}) | P_{θ}^{n}} ⇒ N(0, Γ_{p}(θ)).  (3.24)

Finally, combining (3.24) with assertion IV of Theorem 3.1 we obtain the desirable convergence (3.22). □

Remark 3.4. As it has been mentioned above, if we consider system (3.3) of estimational equations

h_{i}^{n}(θ) = 0, i = \overline{1, m + 1}, n ≥ 1,

and suppose that all conditions of Lemma A with ℓ = m+1, Q_{θ}^{n} = P_{θ}^{n}, L_{n}(θ) = (h_{1}^{n}(θ), …, h_{m+1}^{n}(θ))′ are satisfied, then all assertions of Lemma A hold true, i.e., for any θ ∈ R^{m+1} there exists a sequence (θ̂_{n})_{n≥1} with properties I and II (of Lemma A) and such that

L{γ(θ)C_{n}^{−1}(θ̂_{n} − θ) | P_{θ}^{n}} ⇒ Φ as n → ∞,  (3.25)

where Φ is defined as

L{C_{n}(h_{1}^{n}(θ), …, h_{m+1}^{n}(θ))′ | P_{θ}^{n}} ⇒ Φ as n → ∞.  (3.26)

Now the ergodicity conditions, condition L and CLT for locally square integrable martingales imply that

Φ = N(0, Γ_{p}(θ)),  (3.27)

and, hence with (3.25) and (3.27) in mind it follows that

L{C_{n}^{−1}(θ̂_{n} − θ) | P_{θ}^{n}} ⇒ N(0, γ^{−1}(θ)Γ_{p}(θ)(γ^{−1}(θ))′)

as n → ∞. In particular,

L{C_{n}^{−1}(θ̂_{1,n} − θ_{1}) | P_{θ}^{n}} ⇒ N (0, (γ^{−1}(θ)Γ_{p}(θ)(γ^{−1}(θ))′)_{11}) .  (3.28)

On the other hand, from (3.25), (3.26) and (3.27) it directly follows that

L\{ξ_{1}^{n}(θ) − \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ)\underline{ξ} _{2}^{n}(θ) | P_{θ}^{n}\} ⇒ Φ̃,  (3.29)

where

ξ_{1}^{n}(θ) = γ_{11}(θ)C_{n}^{−1}(θ̂_{1,n} − θ_{1}) + \underline{γ}_{12}(θ)C_{n}^{−1}(\underline{θ̂}_{2,n} − \underline{θ}_{2}),
\underline{ξ} _{2}^{n}(θ) = \underline{γ}_{21}(θ)C_{n}^{−1}(θ̂_{1,n} − θ_{1}) + \underline{γ}_{22}(θ)C_{n}^{−1}(\underline{θ̂}_{2,n} − \underline{θ}_{2}),

and Φ̃ is a weak limit of the sequence of distributions

L\{C_{n}(h_{1}^{n}(θ) − \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ)\underline{h} _{2}^{n}(θ) | P_{θ}^{n}\} = L\{C_{n}H_{n}(θ) | P_{θ}^{n}\} as n → ∞,

i.e. (see (3.23)),

Φ̃ = N(0, Γ_{p}(θ)).  (3.30)

But

ξ_{1}^{n}(θ) − \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ)\underline{ξ} _{2}^{n}(θ) = (γ_{11}(θ) − \underline{γ}_{12}(θ)\underline{γ} _{22}^{−1}(θ)\underline{γ}_{21}(θ))C_{n}^{−1}(θ̂_{1,n} − θ_{1})
= γ_{p}(θ)C_{n}^{−1}(θ̂_{1,n} − θ_{1}).

Adding to this equality (3.29) and (3.30) we obtain

L\{C_{n}^{−1}(θ̂_{1,n} − θ_{1}) | P_{θ}^{n}\} ⇒ N(0, \frac{Γ_{p}(θ)}{(γ_{p}(θ))^{2}} ).  (3.31)

Now after comparing (3.28) and (3.31), one can conclude that

(γ^{−1}(θ)Γ(θ) (γ^{−1}(θ))′)_{11} = \frac{Γ_{p}(θ)}{(γ_{p}(θ))^{2}}  (3.32)

and thus the estimator (\overline{θ}_{1,n})_{n≥1} constructed by the skew projection technique has the same asymptotic variance as the first component (θ̂_{1,n})_{n≥1} of the estimator (θ̂_{n})_{n≥1} constructed by solving the full system (3.3).

If equation (3.10) has a unique solution \overline{θ}_{1,n}, then Theorem 3.1 establishes the asymptotic distribution of the normed sequence (C_{n}^{−1}(\overline{θ}_{1,n} − θ_{1}))_{n≥1} (see (3.22)).

To avoid the problem of the solvability of equation (3.10) one can consider the following one-step approximation procedure: let (θ_{n})_{n≥1} be a C_{n}-consistent estimator of a full parameter θ ∈ R^{m+1}. i.e., C_{n}^{−1}(θ_{n} − θ) = O_{P_{θ}^{n}}(1).

Define an estimator \overline{θ}_{1,n} of θ_{1} by the one-step procedure

\overline{θ}_{1,n} = θ_{1,n} + γ_{p}^{−1}(θ_{n})C_{n}^{2}H_{n}(θ_{n}).  (3.33)

Theorem 3.2. Let conditions (1), (2), (4) and (5) of Theorem 3.1 be satisfied as well as the condition

(6′) for any θ ∈ R^{m+1}, N > 0 and ρ > 0

\overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{\begin{matrix}x:|x−θ_{1}|≤C_{n}N \\ y:|y−\underline{θ}_{2}|≤C_{n}N \end{matrix}} C_{n}^{2}|H^{(1)}_{n}(x, y) − H^{(1)}_{n}(θ_{1}, y)| > ρ\} = 0.

Then,

L\{C_{n}^{−1}(\overline{θ}_{1,n} − θ_{1}) | P_{θ}^{n}\} ⇒ N(0, \frac{Γ_{p}(θ)}{γ_{p}^{2}(θ)})

as n → ∞.

Proof. (3.33) yields

C_{n}^{−1}(\overline{θ}_{1,n} − θ_{1}) = C_{n}^{−1}(θ_{1,n} − θ_{1}) + γ_{p}^{−1}(θ)C_{n}H_{n}(θ_{n})
+ (γ_{p}^{−1}(θ_{n}) − γ_{p}^{−1}(θ)) C_{n}H_{n}(θ_{n})
= C_{n}^{−1}(θ_{1,n} − θ_{1}) + γ_{p}^{−1}(θ)C_{n}L_{n}(θ_{1,n})
+ (γ_{p}^{−1}(θ_{n}) − γ_{p}^{−1}(θ)) C_{n}L_{n}(θ_{1,n}),  (3.34)

where L_{n}(θ_{1}) = H_{n}(θ_{1}, \underline{θ}_{2,n}).

Using the Taylor expansion, we have

L_{n}(θ_{1,n}) = L_{n}(θ) + L̇_{n}(x_{n})(θ_{1,n} − θ_{1}),  (3.35)

where L̇(θ_{1}) = \frac{∂}{∂θ} L_{n}(θ_{1}) = H^{(1)}_{n}(θ_{1}, \underline{θ}_{2,n}), x_{n} = θ_{1} − α(θ_{1,n} − θ_{1}), α ∈ [0, 1].

Substituting (3.35) in (3.34), we obtain

C_{n}^{−1}(θ_{1,n} − θ_{1}) = γ_{p}^{−1}(θ)C_{n}L_{n}(θ_{1}) + ε_{n}(x_{n}, θ) + δ(θ_{n}, θ),  (3.36)

where

ε_{n}(x_{n}, θ) = γ_{p}^{−1}(θ)(γ_{p}(θ) + C_{n}^{2}L̇(x_{n}))C_{n}^{−1}(θ_{1,n} − θ_{1}),
δ_{n}(θ_{n}, θ) = (γ_{p}^{−1}(θ_{n}) − γ_{p}^{−}(θ)) C_{n}L_{n}(θ_{1,n}).

Now if we prove that

P_{θ}^{n}- lim_{n→∞} ε_{n}(x_{n}, θ) = 0,  (3.37)
P_{θ}^{n}- lim_{n→∞} δ_{n}(θ_{n}, θ) = 0,  (3.38)

then the desirable convergence (the assertion of Theorem 3.2) follows from (3.36) and (3.24).

First we prove (3.37).

Using a standard technique we have for any ρ > 0 and N > 0

\overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{2}|L̇_{n}(x_{n}) − L̇_{n}(θ_{1})| > ρ\}

= \overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{2}|H^{(1)}_{n}(x_{n}, \underline{θ}_{2,n}) − H^{(1)}_{n}(θ_{1}, \underline{θ}_{2,n})| > ρ\}
≤ \overline{lim}_{n→∞} P_{θ}^{n}\{ sup_{\begin{matrix}x:|x−θ_{1}|≤C_{n}N \\ y:|y−\underline{θ}_{2}|≤C_{n}N \end{matrix}} C_{n}^{2}|H^{(1)}_{n}(x_{n}, y) − H^{(1)}_{n}(θ_{1}, y)| > ρ\}

+ \overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{−1} | |\underline{θ}_{2,n} − \underline{θ}_{2}| > N\}

= \overline{lim}_{n→∞} P_{θ}^{n}\{C_{n}^{−1} | |\underline{θ}_{2,n} − \underline{θ}_{2}| > N\},  (3.39)

where the last equality follows from condition (6′). Letting N → ∞ in (3.39) we obtain

P_{θ}^{n}- lim_{n→∞} C_{n}^{2}|L̇_{n}(x_{n}) − L̇_{n}(θ_{1})| = 0.  (3.40)

In view of (3.21), (3.40) implies

P_{θ}^{n}- lim_{n→∞} (γ_{p}(θ) + C_{n}^{2}L̇_{n}(x_{n})) = 0  (3.41)

from which taking into account that C_{n}^{−1}(θ_{1,n} − θ_{1}) = O_{P_{θ}^{n}}(1) we get (3.37).

As for (3.38) in view of (3.41) and (3.24) it evidently follows from (3.35) that the sequence (C_{n}L_{n}(θ_{1,n}))_{n≥1} is bounded in probability. It remains to notice that γ_{p}^{−1}(θ) is continuous in θ. Theorem is proved. □

Appendix

Let for every θ ∈ Θ ⊂ R^{ℓ}, ℓ ≥ 1, a sequence of probability measures (Q_{θ}^{n})_{n≥1}, (Q_{θ}^{n} ∼ P^{n}) and a ℓ-dimensional random vectors L_{n}(θ), n ≥ 1, as well as a sequence of positive numbers (C_{n})_{n≥1} be given on a measurable space (Ω^{n}, F_{T}^{n}).

We are interested in the of solvability of the system of equations

L_{n}(θ) = 0  (A.1)

and asymptotic behavior of solutions as n → ∞.

The following lemma is proved in [2].

Lemma A. Let the following conditions be satisfied:

a) lim_{n→∞} C_{n} = 0;

b) for each n ≥ 1 the mapping θ ⇝ L_{n}(θ) is continuously differentiable P^{n}-a.s.;

c) for each pair (θ, y), θ ∈ Θ, y ∈ Θ there exists a function Δ_{Q}(θ, y) such

that

Q_{θ}^{n}- lim_{n→∞} C_{n}^{2}L_{n}(y) = Δ_{Q}(θ, y)

and the equation (w.r.t. y)

Δ_{Q}(θ, y) = 0

has a unique solution θ^{∗} = b(θ) ∈ Θ;

d) for each θ ∈ Θ

Q_{θ}^{n}- lim_{n→∞} C_{n}^{2}L̇_{n}(θ^{∗}) = −γ_{Q}(θ),

where γ_{Q}(θ) is a positive definite matrix, L̇_{n}(θ) = (\frac{∂}{∂θ_{i}} L_{n,j}(θ))_{i,j=\overline{1,d}};

e) for each θ ∈ Θ and ρ > 0

lim_{r→0} \overline{lim}_{n→∞} Q_{θ}^{n} \{ sup_{y:|y−θ^{∗}|≤r} |L̇_{n}(y) − L̇_{n}(θ^{∗})| > ρ\} = 0.

Then for any θ ∈ Θ there exists a sequence of random vectors θ̂ = (θ̂_{n})_{n≥1} taking values in Θ such that

I. lim_{n→∞} Q_{θ}^{n}{L_{n}(θ̂_{n}) = 0} = 1,

II. Q_{θ}^{n}- lim_{n→∞} θ̂_{n} = θ^{∗},

III. if θ̃ = (θ̃_{n})_{n≥1} is another sequence with properties I and II, then

lim_{n→∞} Q_{θ}^{n}{θ̃_{n} = θ̂_{n}} = 1,

IV. if the sequence of distributions L{C_{n}L_{n}(θ^{∗}) | Q_{θ}^{n}} converges weakly to a certain distribution Φ, then

L{γ_{Q}(θ)C_{n}^{−1}(θ̂_{n} − θ^{∗}) | Q_{θ}^{n}} ⇒ Φ as n → ∞.

Remark. By the same arguments as used in the proof of Lemma A one can easily verify that all assertions of Lemma A hold true of instead of ℓdimensional function L_{n}(θ) we consider a k-dimensional, k < ℓ, function L_{n}(θ) such that L_{n}(θ) = L_{n}(θ_{1}, …, θ_{k}), θ = (θ_{1}, …, θ_{k}, …, θ_{ℓ})′, with L_{n}(θ_{1}, θ_{2}, …, θ_{k}) being continuously differentiable in each point θ_{i}, i ≤ k, and assume that all conditions of Lemma A are satisfied with k-dimensional function Δ_{Q}(θ, y) = Δ_{Q}(θ, y_{1}, y_{2}, …, y_{k}), y = (y_{1}, …, y_{k}, …, y_{ℓ}) ∈ Θ, and

L̇(θ) = (\frac{∂}{∂θ_{i}} L_{n,j}(θ_{1}, …, θ_{k}))_{i,j≤k} .

It should be noticed that a function L_{n}(θ) is defined on projection of set Θ ⊂ R^{k} × R^{ℓ−k} onto R^{k}.

Acknowledgement

The research was supported by INTAS Grant No. 99–00559 and by Grant of Georgian Academy of Sciences No. 10.1.

References

1. P. J. Bickel, C. A. J. Klaassen, Y. Ritov, and J. A. Wellner, Efficient and adaptive estimation for semiparametric models. Springer, New York, 1998.

2. R. J. Chitachvili, N. L. Lazrieva, and T. A. Toronjadze, Asymptotic theory of M-estimators in general statistical models, Parts I, II. Centrum voor Wiskunde en Informatica, Reporis BS–R9019, BS–R9020, June, Amsterdam, 1990.

3. J. H. Friedman and W. Stuetzle, Projection pursuit regression. J. Amer. Statist. Assoc. 76(1981), No. 376, 817–823.

4. P. Huber, Robust statistics. Willey, New York, 1981.

5. J. Jacod, Calcul stochastique et problémes de martingales. Lecture Notes in Mathematics, 714. Springer, Berlin, 1979.

6. J. Jacod and A. N. Shiryaev, Limit theorems for stochastic processes. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], 288. Springer-Verlag, Berlin, 1987.

7. J. Jacod, Regularity, partial regularity, partial information process for a filtered statistical model. Probab. Theory Related Fields 86(1990), No. 3, 305–335.

8. N. Lazrieva and T. Toronjadze, Robust estimators in discrete time statistical models. Contiguous alternatives. Proc. A. Razmadze Math. Inst. 115(1997), 59–97.

9. R. S. Liptser and A. N. Shiryaev, Theory of martingales. (Russian) Nauka, Moscow, 1986.

10. J. A. Wellner, Semiparametric models: progress and problems. Centrum voor Wiskunde en Informatica, Report MS–R8614, November, Amsterdam, 1986.

(Received 10.01.2003)

Authors’ address:

A. Razmadze Mathematical Institute Georgian Academy of Sciences 1, M. Aleksidze St., Tbilisi 0193, Georgia