Georgian Mathematical Journal Volume 8 (2001), Number 2, 319–322

ON NORMAL APPROXIMATION OF LARGE PRODUCTS OF FUNCTIONS: A REFINEMENT OF BLACKWELL’S RESULT

ALBERT Y. LO AND V. V. SAZONOV

Abstract. An asymptotic expansion for the approximation of standardized products of large numbers of smooth positive functions by exp(−x^{2}/2) is given. This result is closely related to the Bernstein–von Mises theorem on the normal approximation of posterior distributions.

2000 Mathematics Subject Classification: 62F15.

Key words and phrases: Approximation of large products of functions, Bernstein–von Mises theorem.

1. Introduction

In [1] D. Blackwell, using a simple elementary way, proved that under some conditions standardized products of large numbers of smooth positive functions are close to exp(−x^{2}/2). Moreover, he gave an estimation of this closeness. In the present note we observe that, under additional smoothness conditions, a little more effort provides an asymptotic expansion making this closeness still more precise.

As was pointed out by D. Blackwell, the result he proved is closely related to the Bernstein–von Mises theorem stating the approximability of posterior distributions by normal ones. More precisely, he noted that if one considers the product of smooth positive functions as a likelihood function, one obtains that properly normalized it is often nearly normal, but under wide conditions the likelihood function is close to the posterior density of the parameter. The Bernstein–von Mises theorem was deeply studied by Le Cam [5], [6], Le Cam and Yang [7] (see Historical Remarks therein), De Groot [2] and others, with refinements made by Lindley [8], Johnson [4], Ghost et. al. [3].

2. Approximation of Sums

Let F be a class of real functions defined on an interval (a, b). Assume that the functions in F are t + 3 times continuously differentiable, where t is an integer ≥ 0 and there are positive constants M_{i}, i = 3, …, t + 3, and m such that

\begin{matrix}sup \{|f^{(k)}(x)| : x ∈ (a, b), f ∈ F \} ≤ M_{k}, k = 3, …, t + 3, \\ sup \{f^{(2)}(x) : x ∈ (a, b), f ∈ F \} ≤ −m. \end{matrix}  (1)

ISSN 1072-947X / $8.00 / \begin{matrix}c \\ ◯ \end{matrix} Heldermann Verlag www.heldermann.de

Let a sum S of n functions from F satisfy S^{(1)}(x_{0}) = 0 at a point x_{0} ∈ (a + d, b − d), d > 0. Denote σ^{2} = −1/S^{(2)}(x_{0}). Then, by the Taylor formula

S(x_{0} + σy) = S(x_{0}) − \frac{y^{2}}{2} + ∑_{k=3}^{t+2} \frac{1}{k!} S^{(k)}(x_{0})(σy)^{k}
+ \frac{1}{(t + 3)!} S^{(t+3)}(x_{0} + θσy)(σy)^{t+3},  (2)

where 0 < θ < 1. Obviously condition (1) implies

σ ≤ (mn)^{−1/2}, |S^{(k)}(x_{0})σ^{k}| ≤ M_{k}m^{−k/2}n^{1−k/2}, k = 3, …, t + 3. (3)

Moreover, if y is bounded, say |y| ≤ A, then

x_{0} + σθy ∈ (a, b),  (4)

if σ|y| ≤ (mn)^{−1/2}A < d, i.e., if

n > A^{2}m^{−1}d^{−2}.  (5)

When (4) is satisfied we have as in (3)

|S^{(t+3)}(x_{0} + θσy)σ^{t+3}| ≤ M_{t+3}m^{−(t+3)/2}n^{−(t+1)/2}.  (6)

Thus we have proved the following

Proposition. If a sum S of n functions from F has a maximum at a point x_{0} ∈ (a + d, b − d), d > 0, and condition (5) is satisfied, then expansion (2) is true with the terms in it satisfying (3), (6).

3. Approximation of Products

Now we will use the above proposition to construct an approximation of products.

Consider a product Q = ∏_{i=1}^{n} g_{i} of n positive functions defined on (a, b), and

assume that Q^{(1)}(x_{0}) = 0, where x_{0} ∈ (a + d, b − d), d > 0, and that f_{i} = log g_{i}, i = 1, …, n, satisfy condition (1). Furthermore, let

S = log Q, σ^{2} = −Q(x_{0})/Q^{(2)}(x_{0}) = −1/S^{(2)}(x_{0})

and denote

Q_{S} = Q(x_{0} + σy)/Q(x_{0})

the standardized form of Q. Then for |y| ≤ A, if condition (5) is satisfied, the above proposition implies

Q_{S}(y) = exp ( log Q(x_{0} + σy) − log Q(x_{0}))
= exp (S(x_{0} + σy) − S(x_{0})) = exp(−y^{2}/2)I,

ON NORMAL APPROXIMATION OF LARGE PRODUCTS OF FUNCTIONS 321

where

I = exp ( ∑_{k=3}^{t+2} \frac{1}{k!} S^{(k)}(x_{0})(σy)^{k} + \frac{1}{(t + 3)!} S^{(t+3)}(x_{0} + θσy)(σy)^{t+3}). (7)

Next, using the expansion e^{u} = 1 + u + ⋯ + \frac{u^{t}}{t!} + \frac{u^{t+1}}{(t+1)!} e^{λu}, 0 < λ < 1, we can write

I = 1 + ∑_{s=3}^{t+2} ∑_{r=1}^{s} ∑′(k_{1}! … k_{r}!)^{−1}S^{(k_{1})}(x_{0}) … S^{(k_{r})}(x_{0})(σy)^{2r+s} + R,

where the summation ∑′ is over all integers k_{1}, …, k_{r} such that k_{j} ≥ 3, j = 1, …, r, k_{1} + ⋯ k_{r} = 2r + s. By (1) we have (3), and hence

|S^{(k_{1})}(x_{0}) … S^{(k_{r})}(x_{0})σ^{2r+3}| ≤ M_{k_{1}} ⋯ M_{k_{r}}m^{−(2r+s)/2}n^{−s/2}.

It is straightforward to check that when condition (5) is also satisfied

|R| ≤ c(t, m, M_{3}, …, M_{t+3}, A)n^{−(t+1)/2}.

4. Examples

In the examples that follow we take for simplicity t = 2.

a. Let 0 < a < b < 1 and consider on (a, b) the product Q(x) = x^{l}(1 − x)^{n−l}, where n is an integer and 0 ≤ l ≤ n; Q is a product of n functions each equal to x or 1 − x.

Considered on (0, 1), the function Q attains its maximum at x_{0} = l/n. We will assume that a+d ≤ l/n ≤ b−d for some d > 0. Now an elementary computation shows that for the derivatives of S(x) = log Q(x) = l log x + (n − l) log(1 − x) we have

S^{(2)}(x_{0}) = −1/σ^{2} = −\frac{n^{3}}{l(n − l)},
S^{(3)}(x_{0})|S^{(2)}(x_{0})|^{−3/2} = \frac{2(n − 2l)}{(l(n − l)n)^{1/2}} ,
S^{(4)}(x_{0})(S^{(2)}(x_{0}))^{−2} = −6 \frac{3l^{2} − 3ln + n^{2}}{l(n − l)n} .

Thus in the case we consider

\frac{Q(x_{0} + σy)}{Q(x_{0})} = exp(−y^{2}/2)Σ + R,  (8)

where

Σ = 1 + \frac{1}{3} \frac{n − 2l}{(l(n − l)n)^{1/2}} y^{3} − \frac{1}{4} \frac{3l^{2} − 3ln + n^{2}}{l(n − l)n} y^{4} + \frac{1}{16} \frac{(n − 2l)^{2}}{l(n − l)n} y^{6},

and when |y| ≤ A and (5) is satisfied

|R| ≤ cn^{−3/2}  (9)

with c depending only on a, b, A.

b. Next consider Q(x) = x^{n}e^{−x}, x > 0. As is pointed out in [1], the case of this function reduces to the case of Q_{1}(x) = (xe^{−x})^{n} since, as is easy to check, they have the same standardized forms:

Q_{S}(x) = Q_{1S}(x) = (1 + n^{−1/2}y)^{n} exp(−n^{1/2}y).

For Q_{1} we have S_{1} = log Q_{1} = nf, f = log x − x, and f^{(1)}(x) = (1/x) − 1, and f^{(k)}(x) = (−1)^{k−1}(k − 1)!x^{−k}, k ≥ 2. Thus x_{0} = 1, σ = 1 and the general result of Secion 2 is applicable with 0 < a < 1 < b < ∞, d = min(|a − 1|, |b − 1|), m = b^{−2}. Namely, we have

Q_{S}(y) = Q_{1S}(y) = exp(−y^{2}/2)(1 + \frac{1}{3} y^{3}n^{−1/2} − (\frac{1}{4} y^{4} − \frac{1}{16} y^{6})n^{−1}) + R,

and when |y| ≤ A and condition (5) is satisfied |R| ≤ cn^{−3/2} with c depending only on a, b, A.

Acknowledgement

The work of the second author was partially supported by Russian Foundation for Basic Research, Project 00-01-00406.

References

1. D. Blackwell, Approximate normality of large products. Technical Report No. 54, Dept. of Statistics, University of California, Berkeley, 1985.

2. M. H. De Groot, Optimal statistical decisions. Wiley, New York, 1970.

3. J. K. Ghost, B. Sinha, and S. N. Joshi, Expansions for posterior probability and integrated Bayes risk. Statistical Decision Theory and Related Topics (S. S. Gupta and J. O. Berger, eds) 3, 1, 403–456, Academic Press, New York, 1982.

4. R. A. Johnson, Asymptotic expansions associated with posterior distributions. Ann. Math. Statist. 41(1970), 851–864.

5. L. Le Cam, On some asymptotic properties of maximum likelihood estimates and related Bayes’ estimates. Univ.California Publ. Statist. 1(1953), 277–330.

6. L. Le Cam, Likelihood functions for large numbers of independent observations. Research Papers in Statistics. Festschr. J. Neyman (F. N. David, ed.), 167–187, Wiley, New York, 1966.

7. L. Le Cam and G. L. Yang, Asymptotics in statistics: some basic concepts. Springer, New York, 1990.

8. D. V. Lindley, The use of prior probability distributions in statistical inference and decision. Proc. 4th Berkeley Symp. on Math. Stat. and Prob. 453–468, Univ. California Press, Berkeley, 1961.

(Received 25.12.2000)

Authors’ addresses:

Albert. Y. Lo Hong Kong University of Sciences and Technology Hong Kong E-mail: imaylo@ust.hk V. V. Sazonov Steklov Mathematical Institute 8, Gubkin str., 117966, Moscow GSP-1 Russia E-mail: sazonov@mi.ras.ru