Georgian Mathematical Journal Volume 8 (2001), Number 2, 363–376

REPRESENTATIONS OF CONDITIONAL MEANS

M. M. RAO

Dedicated to Nicholas N. Vakhania on his 70th birthday

Abstract. A unified treatment of integral representations of conditional expectations, in terms of Bochner integrals, with specializations to Gaussian conditioning, is given. The integrals in the latter have deterministic kernels. Difficulties in actual calculations of (even regular) conditional probabilities, and the inadequacy of methods related to L’Hospital’s procedures, are exemplified. Limitations of the methods used and some new problems for investigation are indicated.

2000 Mathematics Subject Classification: Primary: 60B11, 60H99. Secondary: 20C20. Key words and phrases: Bochner’s integral representation, deterministic kernels, Kac–Slepian paradox, Bochner’s boundedness principle.

1. Introduction

A conditional mean is a linear “smoothing” or averaging operation on a topological vector space of measurable functions based on a probability space (Ω, Σ, P), although P here can be replaced by a σ-finite measure without much difficulty. It is then of interest to define and represent it as an integral, generally using vector measures. Different types of these means, related to conditioning, will be presented in what follows. For convenience, all the (probability) measures are taken to be complete throughout.

Thus if L^{1}(P) is the Lebesgue space of real integrable functions on (Ω, Σ, P) and f ∈ L^{1}(P), consider ν_{f} : B ↦ ∫_{B} f dP, B ∈ B, a σ-subalgebra of Σ. Evidently ν_{f} is σ-additive and P_{B}-continuous where P_{B} = P |B, the restriction which is finite. Hence by the classical (scalar) Radon–Nikodým theorem, there is a P_{B}-unique B-measurable g_{f} = \frac{dν_{f}}{dP_{B}} ∈ L^{1}(P_{B}). The mapping E^{B} : f ↦ g_{f} is a well-defined positive linear operator, called conditional mean or expectation. It satisfies the set of equations:

∫_{B} f dP = ∫_{B} g_{f} dP_{B} = ∫_{B} E^{B}(f) dP_{B}, B ∈ B, f ∈ L^{1}(Σ).  (1)

Letting f = χ_{A}, A ∈ Σ, and P^{B}(A) = E^{B}(χ_{A}), one has P^{B} : Σ → L^{1}(Ω, B, P_{B}) (= L^{1}(P_{B}) to be a vector measure, i.e., P^{B} is a (non-negative) σ-additive func-

ISSN 1072-947X / $8.00 / \begin{matrix}c \\ ◯ \end{matrix} Heldermann Verlag www.heldermann.de

tion in the norm topology of L^{1}(P_{B}). However, P^{B}(·)(ω) : Σ → ℝ^{+}, ω ∈ Ω, is not necessarily a measure, in that it can fail to be σ-additive. [For an example of this pathology, see Loève [8], p. 370.] It is called a regular conditional probability measure if P^{B}(·)(ω) : Σ → ℝ^{+} is σ-additive and P^{B}(A)(·) : Ω → ℝ is B-measurable, A ∈ Σ. However, one can define, for a simple function h = ∑_{i=1}^{n} a_{i}χ_{A_{i}}, A_{i} ∈ Σ, disjoint, an integral as:

E^{B}(h) = ∑_{i=1}^{n} a_{i}P^{B}(A_{i}) = ∫_{Ω} h dP^{B}.  (2)

A standard computation shows that the integral here is well-defined for all simple functions, and it does not depend on the representation of such h. Next an extension for all measurable non-negative functions and then for all elements of L^{1}(P) is easily obtained, using the Dunford–Schwartz (or DS)-integral. This and an integral representation of E^{B} on L^{1}(P), relative to the scalar measure P, as a Bochner integral, is given in the next section. It will then be specialized and refined further. As an application, a vector Radon–Nikodým theorem is presented for a pair of conditional measures on Σ. Difficulties for evaluating conditional expectations and measures of such operators are also illustrated, with the resulting paradoxes. This and some specialized integral representations for Gaussian processes in terms of deterministic kernels relative to (Gaussian) martingale measures form the content of Section 3. Moreover, limitations of the special method employed are noted.

2. An Integral Representation of Conditional Means

Recall that for a vector measure ν : Σ → X, a Banach space, there is a positive finite dominating measure μ in terms of which integrals of scalar functions relative to ν can be defined following the procedure of Dunford and Schwartz ([3], Sec. IV.10). If X = L^{1}(P) and ν = P^{B}, one can explicitly calculate the desired μ. This is used in the following work, and is stated here as:

Lemma 2.1. The variation of P^{B} : Σ → L^{1}(P_{B}) is P.

Proof. By definition of variation, for each A ∈ Σ, if Σ(A) = {A ∩ B : B ∈ Σ}, then

|P^{B}|(A) = sup { ∑_{i=1}^{n} ∥P^{B}(A_{i})∥_{1} : A_{i} ∈ Σ(A), disjoint}

= sup { ∑_{i=1}^{n} ∫_{Ω} E^{B}(χ_{A_{i}}) dP_{B} : A_{i} as above}

= P (A).  (1)

Thus its variation measure in the Banach space L^{1}(P_{B})is P. This can also be obtained from ([3], III.2.20, p. 114) easily.

It follows that P^{B} is P -continuous and let the DS-integral for simple functions be as in (2) of the preceding section. If f ∈ L^{1}(P) is arbitrary, then there exists a sequence of simple functions {f_{n}, n ≥ 1} such that f_{n} → f pointwise (or in P -measure), and if {∫_{A} f_{n} dP^{B}, n ≥ 1} ⊂ L^{1}(P_{B}) is Cauchy for each A ∈ Σ, the limit is P -uniquely defined. It is denoted as ∫_{A} f dP^{B}, so that

∫_{A} f dP^{B} = lim_{n} ∫_{A} f_{n} dP^{B}, A ∈ Σ, f ∈ L^{1}(P).  (2)

One can verify that the DS-integral above is defined for all elements of L^{1}(P), and the dominated convergence statements are also shown in [3] to hold for these integrals. From this and (2) of Section 1 one has the representation:

E^{B}(f) = ∫_{Ω} f dP^{B}, f ∈ L^{1}(P).  (3)

The next result yields a useful refinement of (3), implying that the DS-integral may be replaced by the more familiar Bochner integral. First a technical condition will be restated. Let Σ_{+}(A) = {B ⊂ A : B ∈ Σ_{+}} where Σ_{+} = {A ∈ Σ : P (A) > 0}. Then the mean direction set of a vector measure ν : Σ → X, a Banach space, on A ∈ Σ_{+} is the collection A_{ν}(A) = \{\frac{ν(B)}{P (B)} : B ∈ Σ_{+}(A)\} ⊂ X. Motivated by a corresponding property in the classical fundamental theorem of calculus, and following Kelley and Srinivasan [6], one says that a vector measure ν : Σ → X of finite variation |ν|(·) has locally small mean direction sets if for each ε > 0 there is an A(= A_{ε}) ∈ Σ_{+} such that the diameter diam(A_{ν}(A)) ≤ ε.

This condition abstracts a relative weak compactness hypothesis, originally formulated by Phillips [14], which was generalized and perfected by Rieffel [20] and refined further by Kupka [8]. In the context of conditional measures, all these generalizations coincide. The important consequence, when X is an arbitrary Banach space, is that a vector measure ν : Σ → X of finite variation has a Bochner integrable RN-derivative iff ν satisfies the above diameter condition. This is of use for us since P^{B} : Σ → L^{1}(P_{B}) and the range space does not generally have the so-called Radon–Nikodým property. The above condition on P^{B} thus implies (and in fact is equivalent to having) the Bochner integrable bounded density. Let us state this specialization for P^{B}, without repeating the details, and then discuss several consequences and applications of interest in the present work.

Theorem 2.2. For any σ-algebra B ⊂ Σ of (Ω, Σ, P), let P^{B} : Σ → L^{1}(P_{B}) be a conditional measure. Suppose that P^{B} satisfies the locally small mean diameter condition introduced above. Then there is a P -unique strongly measurable and P -integrable g : Ω → L^{1}(P_{B}) such that

P^{B}(A) = ∫_{A} g dP, A ∈ Σ,  (4)

where the integral of the (vector valued) function g is in Bochner’s sense.

Discussion of the result. The traditional (relative) weak compactness condition, used by Phillips, for ν into L^{1}(P) of finite variation is equivalent to (relative) compactness in our case by ([3], VI.8.13, on p. 510), since P^{B}(A) = E^{B}(χ_{A}), A ∈ Σ and the operator E^{B} is a projection. Since the mean diameter condition is not easy to verify in applications, the following classes of measures will help understand its significance better. In case (Ω, Σ, P) is a standard Borel space, or a Polish (probability) space, then for each B ⊂ Σ, P^{B} is a regular conditional measure (cf., e.g., [18], Sec. 5.3), and taking B as a σ-subalgebra of Σ generated by a random variable or vector, one concludes that the representation (4) above is valid (and thus the diameter condition is automatically satisfied as is the case for scalar measures). If (Ω, Σ, P) is a perfect probability space then Sazonov [20] shows that P^{B} is regular and also perfect, admitting special methods. In fact Doob ([2], Example 2.7, on p. 617) has directly established the representation (4), using a classical construction from (scalar) differentiation theory, when Ω = ℝ^{2}. Several applications with extensions of the (saclar) case have also been discussed by Meyer [12]. An interesting aspect of the abstract theory is that one can clearly see the underlying structure of the problem, and the regularity of P^{B} is not postulated at the outset. The significance of the representation (4) is revealed if one considers (Ω, Σ, P) as the Lebesgue unit interval and B = Σ, so that P^{B}(A) = χ_{A}, A ∈ Σ, and it is known that for this case there is no Bochner integrable g for which (4) is valid. This case and the ramifications of the abstract Banach space as well as more general locally convex spaces were given in [20], [8], and [11], with some elegant simplifications in [13]. Also an aspect of the above, using Phillips’ criterion with boundedness and uniform integrability, was indicated in [15] without a detailed discussion as here. It must be noted, however, that in applications a verification of an abstract condition such as the locally small mean diameter above, is not easy and sometimes one has to find methods amounting to establishing the result individually.

As a consequence of the above theorem, the following refinement of a result due to Loève ([8], p. 344), who termed it a “relative conditional expectation”, can be obtained:

Theorem 2.3. Let B ⊂ Σ be a σ-algebra from (Ω, Σ, P) and 0 ≤ Z ∈ L^{1}(P). If μ_{Z} : A ↦ ∫_{A} Z dP, A ∈ Σ, (so μ_{Z} is a finite measure), and X is a μ_{Z}- integrable random variable, then the (new) conditional measure μ_{Z}^{B} : Σ → L^{1}(P_{B}) admits representations as:

∫_{Ω} f dμ_{Z}^{B} = \frac{E_{P}^{B}(fZ)}{E_{P}^{B}(Z)}, ∀f ∈ L^{∞}(P),  (5)

= \frac{∫_{Ω} fgZ dP}{∫_{Ω} gZ dP} , ∀f ∈ L^{∞}(P),  (6)

where the left side (and (5)) is the DS (vector) integral, and the right side of (6), when P^{B} satisfies the locally small mean diameter condition, is a standard Bochner integral for a P -unique g : Ω → L^{∞}(P). In particular, if Z = 1 = f a.e., then (5) and (6) reduce to (4).

Remark. This extension of Theorem 2.2 is of interest in filtering theory as seen from [12]. [In the proof, let μ stand for μ_{Z} for simplicity.]

Proof. By definition, μ is a finite measure and dμ = Z dP. So μ_{B} = μ|B is given by

μ_{B}(A) = ∫_{A} Z dP = ∫_{A} E_{P}^{B}(Z) dP_{B}, A ∈ B.  (7)

Hence dμ_{B} = E_{P}^{B}(Z) dP_{B}, and for the signed measure ν : A ↦ ∫_{A} X dμ, A ∈ Σ, with E_{μ}^{B} denoting the conditional expectation operator for μ, one has similarly

ν_{B}(B) = ∫_{B} X dμ = ∫_{B} E_{μ}^{B}(X) dμ_{B}, B ∈ B,
= ∫_{B} E_{μ}^{B}(X)E_{P}^{B}(Z) dP_{B}, by (7).  (8)

Combining (7) and (8), it follows that

∫_{A} E_{μ}^{B}(X)E_{P}^{B}(Z) dP_{B} = ν_{B}(A) = ∫_{A} XZ dP
= ∫_{A} E_{P}^{B}(XZ) dP_{B}, A ∈ B.  (9)

The extreme integrands are B-measurable and so can be identified. Thus

E_{μ}^{B}(X) = \frac{E_{P}^{B}(XZ)}{E_{P}^{B}(Z)}.  (10)

Since the P -null sets are contained in the class of those of μ (because μ ≪ P), this is well defined, and by (2) of Section 1, (5) follows.

Finally, under the additional condition on P^{B} one can use Theorem 2.2. So there is a strongly measurable g : Ω → L^{∞}(P) such that (10) is representable as:

E_{μ}^{B}(X) = \frac{∫_{Ω} XZg dP}{∫_{Ω} Zg dP} , X ∈ L^{1}(P)),  (11)

and if Z = 1, since E_{P}^{B}(1) = 1, a.e., this becomes (4).

To emphasize the present point of view, let us include a Radon–Nikodým type derivative for a pair of conditional (hence vector) measures, as an interesting application of (4). The following abstract concept, already employed in [15], will be needed here to state the general result (cf., e.g., [21] for the relevant Banach space results specifically adapted to this subject).

Definition 2.4. (1) Let (Ω, Σ) be a measurable space, X , Y be Banach spaces and μ : Σ → X , ν : Σ → Y be σ-additive functions. Then ν is termed absolutely continuous relative to μ (ν ≪ μ) if for A ∈ Σ, |μ|(A) = 0, implies ν(A) is the zero element of Y; and ν is singular for μ (ν ⊥ μ) if there is a B_{0} ∈ Σ, |μ|(B_{0}) = 0 and ν(B) = ν(B ∩ B_{0}), B ∈ Σ, where |μ|(·) is again the variation measure of μ.

(2) If f : Ω → X is a strongly measurable Bochner integrable function on (Ω, Σ, λ) where λ is a finite measure, and T : Ω → B(X , Y), [the vector space of bounded linear operators] is a strongly measurable operator function such that T f : Ω → Y is Bochner integrable relative to λ, then the vector measure μ_{f} defined by μ_{f} : A ↦ ∫_{A} f dλ, A ∈ Σ, unambiguously gives a bilinear vector integral by the equation:

∫_{A} (T f) dλ = ∫_{A} T dμ_{f}, A ∈ Σ.  (12)

With these concepts, applied to a pair of (quite general) conditional measures P_{i}^{B_{i}}, i = 1, 2, one has the following comprehensive Radon–Nikodým type theorem as a consequence of the representation (6). The result will then be specialized to obtain a better understanding of the structure of conditional measures:

Theorem 2.5. Let (Ω, Σ, μ_{i}), i = 1, 2, be probability spaces, B_{i} ⊂ Σ, i = 1, 2, any σ-algebras, and P_{i}^{B_{i}} the conditional measures on Σ into L^{1}(B_{i}, μ_{i}), i = 1, 2, each satisfying a local mean diameter condition. Then there exists a (usually nonunique) strongly measurable operator function T : Ω → B(L^{1}(B_{1}, μ_{1}), L^{1}(B_{2}, μ_{2})) and a set A_{0} ∈ Σ such that

P_{1}^{B_{1}}(A) = ∫_{A} T (ω) P_{2}^{B_{2}}(dω) + P_{1}^{B_{1}}(A ∩ A_{0}), A ∈ Σ,  (13)

where |P_{1}^{B_{1}}|(A_{0}) = (μ_{1}(A_{0}) =)0, and the first term on the right is a bilinear (Bochner–Bartle) vector integral.

If μ_{1} ≪ μ_{2}, then the second term in the decomposition (13) disappears, and T (·) becomes a scalar (measurable) function when B_{1} = B_{2} in addition.

Proof. Since P_{i}^{B_{i}} : Σ → L^{1}(B_{i}, μ_{i}), i = 1, 2, are (norm) σ-additive (cf., e.g., [18, Prop. 2.3.1(v) on p. 44), these are vector measures with variations |P_{i}^{B_{i}}|(·) = μ_{i}(·). Now by Theorem 2.2 above, dP^{B_{i}} = g_{i} dμ_{i}, i = 1, 2, so that the hypothesis of [15, Theorem 3.8(I)] is satisfied. So by the latter, there exists a strongly measurable operator function T satisfying (13), the second term being a consequence of the Lebesgue type decomposition for vector measures. [The existence of such a T was again established by a different argument in [16], Theorem 2.] Further the nonuniqueness in the general case was illustrated with an example in [15]. If μ_{1} ≪ μ_{2} so that P_{1}^{B_{1}} ≪ P_{2}^{B_{2}} by Definition 2.4(1), it follows that the second term on the right of (15) vanishes.

It remains to establish the last part when μ_{1} ≪ μ_{2}. Let h = \frac{dμ_{1}}{dμ_{2}} , the RNderivative. Now using a change of variables in the Bochner integral, (4) can be written as:

(a) dP_{1}^{B_{1}} = g_{1} dμ_{1} = g_{1}h dμ_{2}; (b) dP_{2}^{B_{2}} = g_{2} dμ_{2}, a.e. [μ_{2}].  (14)

Hence for each bounded measurable f one has when B = B_{i}, i = 1, 2,

∫_{A} f dP_{1}^{B} = ∫_{A} fg_{1}h dμ_{2}, A ∈ B,

= ∫_{A} E_{2}^{B}(fg_{1}h) dμ_{2B}, E_{2}^{B} is c. exp. for μ_{2},

= ∫_{A} E_{2}^{B}(fg_{1}h)\frac{1}{g_{2}} dP_{2}^{B}, by (14)(b),

= ∫_{A} E_{2}^{B}(\frac{E^{B}(fg_{1}h)}{g_{2}}) dP_{2}^{B}.

This holds since the support of g_{1} is contained in that of g_{2}. Taking f = χ_{A}, A ∈ B, one obtains from the above

P_{1}^{B}(A) = ∫_{A} E_{2}^{B}( ) dP_{2}^{B} = ∫_{A} T dP_{2}^{B}, A ∈ B,  (15)

defining T by this equation. So the integrand in (15) is a nonnegative measurable function giving the RN-derivative as a scalar function in this case.

Remarks. 1. If μ_{1} = μ_{2} so that h = 1 a.e., and a related result (reciprocal to (15)) was obtained in ([16], p. 311) under similar conditions for a slightly restricted case.

2. Since P_{i}^{B_{i}}, i = 1, 2 are order preserving vector measures, one can consider a generalization if L^{1}(B, μ) is replaced by a Stone algebra of continuous functions and the conditional measures are replaced by certain “modular” vector measures when the σ-additivity is given using order topology. This generalization and the resulting RN-theorem with a suitable order integration were developed by Wright [23]. He also applied that theory for certain averaging operators and obtained a related martingale convergence theorem.

The last stated aspect will not be discussed here, since it needs several other preliminaries. In the following section another representation especially for Gaussian processes, and their conditional means will be considered.

3. Conditional Means and Kernel Representations for Gaussian Processes

It is appropriate and relevant here to observe a serious difficulty in the evaluation of conditional expectations, (even) for Gaussian processes. This problem also arises in many applications. To illustrate, let {X_{t}^{i}, t ∈ [−a, a]}, i = 1, 2, be a pair of Gaussian processes such that the X_{t}^{1}-process is ergodic and has a pointwise derivative at each time point t, and let X_{t}^{2}-process denote the derived process, i.e., (X_{t}^{1})′ = X_{t}^{2} (for simplicity, let E(X_{t}^{1}) = 0, ∀t). Such a process X_{t}^{1} exists on a suitable canonical probability space, using the classical Kolmogorov representation of a stochastic function [7]. The problem now is to find the conditional probability of the tangent X_{t}^{2} at t = 0 given that X_{0}^{1} = a. This means that one has to calculate P [X_{0}^{2} < y|X_{0}^{1} = a], and since the conditioning event has zero probability, the usual (L’Hospital’s type) approximation to evaluate this probability leads to ambiguities. But by the Kolmogorov definition, this is given (a.e. uniquely) by

h(a) = P [X_{0}^{2} < y|σ(X_{0}^{1})](a) = E^{σ(X_{0}^{1})}(χ_{[X_{0}^{2}<y]})(a),

and the last quantity is defined by the system of functional equations:

∫_{A} χ_{[X_{0}^{2}<y]} dP = ∫_{A} E^{σ(X_{0}^{1})} dP_{σ(X_{0}^{1})}, A ∈ σ(X_{0}^{1}).  (1)

However there is no algorithm to calculate the conditional expectation h(·), whose value at ‘a′ is the desired result. Kac and Slepian [5] have discussed several approximations, usually employed in the literature, for this calculation, and obtained many distinct solutions for the same problem resulting in a paradox. This anomaly will be briefly illustrated here because of its significance.

To use a standard computation of the asserted probability, consider the event A_{δ}^{m} defined for each δ > 0, m ∈ ℝ, by

A_{δ}^{m} = \{ω : X_{t}^{1}(ω) = a + mt, for some 0 ≤ t ≤ \frac{δ}{\sqrt{1 + m^{2}}} \},

so that P (A_{δ}^{m}) > 0, and one needs to find P [B_{y}|A_{0}^{m}] where B_{y} = [X_{0}^{2} < y] for any m. With the usual formula P (B|A) = \frac{P (A∩B)}{P (A)} for P (A) > 0, one has the result

P (B_{y}|A_{0}^{m}) = lim_{δ↘0} \frac{P [X_{0}^{2} < y, A_{δ}^{m}, X_{0}^{2} > m] + P [X_{0}^{2} < y, A_{δ}^{m}, X_{0}^{2} ≤ m]}{P [A_{0}^{m}, X_{0}^{2} > m] + P [A_{δ}^{m}, X_{0}^{2} ≤ m]}. (2)

An elementary but tedious calculation of the right side, carried out in [5] and detailed in [18], gives, with A_{0}^{m} = [X_{0}^{1} = a], for any m ∈ ℝ:

P [X_{0}^{2} < y|X_{0}^{1} = a] = ∫_{−∞}^{y} \frac{|v − m|e^{− \frac{v^{2}}{2α^{2}}}}{2α^{2}e^{− \frac{m^{2}}{2α^{2}}} + m ∫_{−m}^{m} e^{\frac{u^{2}}{2α^{2}}} du} dv

= ∫_{−∞}^{y} f_{X_{0}^{2}}^{m}(v|a) dv (say),  (3)

where α^{2} > 0, is the variance of X_{0}^{2}. Here one uses the fact that X_{0}^{1} and X_{0}^{2} are independent of each other, due to ergodicity, and hence in (3) the end result does not involve a. But the density f_{X_{0}^{2}}^{m}(·|a) which depends on the auxiliary number m ∈ ℝ, gives infinitely many values for the left side probability. Letting m → ∞, so that the limit is taken vertically, called the “vertical window or v.w. solution”, one finds

f(v|a)|_{v.w.} = \frac{e^{− \frac{v^{2}}{2α^{2}}}}{\sqrt{2πα^{2}}} .  (4)

If, on the other hand, m → 0, so that the limit is through the horizontal direction, called the “horizontal window or h.w. solution”, it is seen to be

f(v|a)|_{h.w.} = \frac{|v|}{2α^{2}} e^{− \frac{v^{2}}{2α^{2}}} .  (5)

Similarly the other values of m give different functions f(·|a). Alternatively, if A_{δ}^{m} above is replaced by B_{δ} = {(x − a)^{2} + t^{2} < δ^{2}, ∃t, 0 < t < δ}, then the same procedure, termed “circular window or c.w. solution” is found to give a value:

f(v|a)|_{c.w} = \frac{\sqrt{1 + v^{2}}e^{− \frac{v^{2}}{2α^{2}}}}{∫_{ℝ} \sqrt{1 + v^{2}}e^{− \frac{v^{2}}{2α^{2}}} dv} .  (6)

The difficulty is that the conditioning σ-algebras B^{m} = σ(A_{δ}^{m}, δ > 0) and B̃ = σ(B_{δ}, δ > 0) are all different from the correct B = σ(X_{0}^{1}) of (1). Note that all these solutions do not involve a as noted earlier.

The problem in the above considerations is that of differentiation of (probability) measures, and the systems of sets {A_{δ}^{m}, B_{δ}; δ > 0, m ∈ ℝ} used above do not satisfy the necessary conditions in that theory which is a serious part of (Lebesgue’s) differentiation. This deeper aspect of analysis (needed here) is not well-recognized or discussed in the literature. That question and some general methods for correct solutions are treated in the author’s book ([18], Sections 3.4 and 7.5). In case densities relative to Lebesgue measure exist then it is shown (nontrivially) that f_{X|Y} (x|y) = \frac{f_{X,Y} (x,y)}{f_{Y} (y)} will also give a correct solution under some regularity conditions (cf., e.g., [18], pp. 52–53).

The next result illuminates such a method in some cases of Gaussian measures. It is based on a special property, and is an extension of certain works in the literature which is analogous to the “v.w.” definition considered above. To motivate this, let X, Y be jointly normally distributed vectors. Since the distributions have densities, using the recalled ratio definition, it is shown classically that the conditional mean vector E(X|Y) is a linear function of Y with a constant covariance matrix (cf., [1], p. 24, or [9], Chapter 13). Since it plays a rôle below, its basic details will be sketched.

Let E(X) = μ_{x}, E(Y) = μ_{y} and R_{xy}, R_{yy} be covariance matrices of (X, Y) and (Y, Y) respectively. For simplicity, suppose that R_{yy} is non-singular, and consider the “Gram–Schmidt orthogonalization” of X − μ_{x} and Y − μ_{y}. This entails choosing a suitable matrix A such that Y − μ_{x} and Z = (X − μ_{x}) − A(Y − μ_{y}) are orthogonal, so that E(Z(Y − μ_{y})^{∗}) = 0. Here ( )^{∗} denotes the conjugate transpose of a vector (or a matrix). Let us simplify this equation and determine A as:

0 = E(Z(Y − μ_{y})^{∗})
= E((X − μ_{x})(Y − μ_{y})^{∗}) − AE((Y − μ_{y})(Y − μ_{y})^{∗})
= R_{xy} − AR_{yy},  (7)

so that A = R_{xy}R_{yy}^{−1}. Thus with this A, Y − μ_{y} and Z are orthogonal and, being centered Gaussian vectors, they are also independent. This key property implies [by the “v.w. definition”] that E(Z|Y) = E(Z) = 0. A simplification of this equation gives:

0 = E(Z|Y) = E(X|Y) − μ_{x} − A(Y − μ_{y}),

or that

E(X|Y) = μ_{x} + A(Y − μ_{y}),  (8)

showing the linearity of the conditional expectation (also termed “regression function”), and then Z = X − E(X|Y). The conditional covariance of X given Y is now immediately obtained:

E((X − E(X|Y))(X − E(X|Y)^{∗}) = E(ZZ^{∗}|Y)

= E(ZZ^{∗}), since Y and Z are independent,

= R_{xx} − R_{xy}R_{yy}^{−1}R_{xy}^{∗},  (9)

which is a constant. It should be observed that these statements are still valid even if R_{yy} is singular, if R_{yy}^{−1} is interpreted as the generalized (or the Moore-Penrose) inverse of a (singular) matrix, but this will not be needed here. The above result may be recorded for reference and use.

Proposition 3.1. If (X, Y) is jointly normal where X = (X_{1}, …, X_{m}) and Y = (Y_{1}, …, Y_{n}) with means μ_{x}, μ_{y}, and covariances R_{xy}, R_{yy}, then the regression function E(X|Y) is linear, which is given by (8), and the conditional covariance of X −E(X|Y) is a constant (does not involve the values of Y) given by (9).

This finite vector statement admits a generalization for all continuous Gaussian martingale processes, of interest in applications, which will now be derived:

Proposition 3.2. Let Y = {Y_{t}, F_{t}, t ≥ 0} be a real continuous Gaussian martingale (not necessarily a diffusion, or adapted to the Brownian filtration), and X be a random variable which is a function of Y in the sense that for each finite set (t_{1}, …, t_{n}), of time points, the vector (X, Y_{t_{1}}, …, Y_{t_{n}}) is Gaussian distributed. Then the regression function E(X|Y_{s}, s ≤ t) is representable a.e. as

E(X|Y_{s}, s ≤ t) = E(X|Y_{0}) + ∫_{0+}^{t} K(t, s) dY_{s},  (10)

where the kernel K(·, ·) is a nonstochastic jointly measurable and locally integrable function, and the stochastic integral in (10) is well-defined, (by the Bochner L^{2,2}-boundedness principle, as in [17], p. 170).

Proof. The following proof is similar to that of Lemma 10.1 of [9], extended to the general Gaussian martingales. This is an extension of the simple case given in the preceding proposition. It was established in ([9], p. 353), if Y is a diffusion process, using properties of the Brownian Motion, but the present assertion needs crucial modifications in the stochastic integration for the asserted generalization. Thus consider the dyadic partition of [0, t] for any t > 0 as follows: 0 = t_{0}^{n} < t_{1}^{n} < ⋯ < t_{2^{n}}^{n} = t where t_{k}^{n} = \frac{kt}{2^{n}} , k = 0, 1, …, 2^{n}, so that there is a linear ordering (in the refinement) by n. If F_{t}^{n} = σ(Y_{t_{k}^{n}}, 0 ≤ k ≤ 2^{n}), then by the (path) continuity of the process, F_{t}^{n} ↑ F_{t} = σ(Y_{s}, 0 ≤ s ≤ t), as n → ∞. Let X_{t}^{n} = E(X|F_{t}^{n}), and observe that {X_{t}^{n}, F_{t}^{n}, n ≥ 1} is an L^{2}(P)-bounded martingale. Indeed,

E(X_{t}^{n+1}|F_{t}^{n}) = E[E(X|F_{t}^{n+1})|F_{t}^{n}]
= E(X|F_{t}^{n}) = X_{t}^{n}, a.e.,

and by the conditional Jensen inequality

E((X_{t}^{n})^{2}) ≤ E(X^{2}) < ∞, n ≥ 1.

Hence the X_{t}^{n}-process is uniformly integrable and by a standard martingale convergence theorem X_{t}^{n} → E(X|F_{t}), a.e., as well as in L^{2}(P)-mean.

Since by definition, F_{t}^{n} = σ(Y_{t_{k}^{n}} −Y_{t_{k−1}^{n}}, k = 1, …, 2^{n} −1) is true, Proposition 3.1 can be applied to the pair {X, (Y_{0}, Y_{t_{1}^{n}} − Y_{t_{0}^{n}}, …, Y_{t_{2^{n}}^{n}} − Y_{t_{2^{n}−1}^{n}})}, (now A is a vector cf., (8))

E(X|F_{t}^{n}) = E(X|Y_{0}) + ∑_{j=0}^{2^{n}−1} L_{n}(t, t_{j}^{n})(Y_{t_{j+1}^{n}} − Y_{t_{j}^{n}}),  (11)

where L_{n}(·, ·) is a certain product moment function and is continuous in both variables jointly, by the continuity of the Y -process. It is to be shown that (11) tends to (10) as n → ∞, using the fact that (Y_{t_{k}^{n}} − Y_{t_{k−1}^{n}}) are the L^{2}(P)- martingale differences and the right side sum in (11) approximates the desired martingale integral. First consider a simple function K_{n}(·, ·) as:

K_{n}(t, s) = ∑_{j=1}^{2^{n}−1} L_{n}(t, t_{j}^{n})χ_{[t_{j}^{n},t_{j+1}^{n})}(s),

which is deterministic. Note that by definition of K_{n}(·, ·), it is a sum of products of jointly measurable functions and hence is jointly measurable, as well as integrable on each compact rectangle in ℝ^{2}, so that it is locally integrable. Now (11) can be expressed unambiguously as:

E(X|F_{t}^{n}) = E(X|Y_{0}) + ∫_{0+}^{t} K_{n}(t, s) dY_{s}, a.e.  (12)

But for any simple function f = ∑_{j=1}^{n} a_{j}χ_{[t_{j},t_{j+1})×A_{j}}, A_{j} ∈ F_{t_{j}}, 0 ≤ t_{0} < ⋯ < t_{n+1} ≤ t and a square integrable martingale {Y_{s}, F_{s}, s ≥ 0} one has (in [9] Gaussian martingales adapted to a Brownian filtration are shown to have deterministic quadratic variations, but here our filtration is more general and the variations may be stochastic; so a short argument depending on the Doléans-Dade measure [20, p. 383], without using a Brownian connection, is included):

E(| ∫_{0}^{t} f(s) dY_{s}|^{2}) = ∑_{j=0}^{n} a_{j}^{2}E(χ_{A_{j}}E^{F_{j}}(Y_{t_{j+1}} − Y_{t_{j}})^{2})

+ 2 ∑_{0≤j<k≤n} a_{j}a_{k}E[χ_{A_{j}∩A_{k}}(Y_{t_{j+1}} − Y_{t_{j}})E^{F_{t_{k}}} (Y_{t_{k+1}} − Y_{t_{k}})]

= ∑_{j=0}^{n} a_{j}^{2}E[χ_{A_{j}}(Y_{t_{j+1}} − Y_{t_{j}})^{2}] + 0

= ∑_{j=0}^{n} a_{j}^{2}μ[(t_{j}, t_{j+1}] × A_{j}]

= ∫_{[0,t]×Ω} |f|^{2} dμ,  (13)

where μ is the Doléans-Dade measure determined by the L^{1}(P)-bounded submartingale {Y_{t}^{2}, F_{t}, t ≥ 0}. The isometry given in (13) can be used to extend the integral to all locally bounded measurable functions f. Hence (12) and (13) imply (replacing f with K_{n}(t, ·) and without demanding that the quadratic variation be nonstochastic)

E(X_{t}^{n} − X_{t}^{m})^{2} = E[ ∫_{0+}^{t} (K_{n}(t, s) − K_{m}(t, s)) dY_{s}]^{2}

= ∫_{[0,t]×Ω} [K_{n}(t, s) − K_{m}(t, s)]^{2} dμ(s, ω),  (14)

where μ is the corresponding measure as in (13). If μ̃(0, s] = μ((0, s] × Ω), the marginal measure, then (14) becomes for each t > 0:

E(X_{t}^{n} − X_{t}^{m})^{2} = ∫_{0+}^{t} (K_{n}(t, s) − K_{m}(t, s))^{2} dμ̃(s).  (15)

But by the first part, {X_{t}^{n}, n ≥ 1} is Cauchy and hence K_{n}(t, ·) ∈ L^{2}(ℝ^{+}, μ̃), n ≥ 1, is also a Cauchy sequence so that it converges to an element K(t, ·) ∈ L^{2}(μ̃) for each t, and the joint measurability is similar to that of Lemma 4.9 of [9]. This and (13) imply that as n → ∞ the integral in (12) indeed converges to that in (10), giving the desired result.

Remarks. 1. It should be noted that the isomorphism appearing in (14) as well as (13) are critical for the above proof. Note that the Y_{t}-process being continuous it has measurable modifications which are used above; but this property is also available for a right continuous process. However this is not enough for other computations above. In general, for L^{2,2}-bounded processes there is only a bounding inequality, and then the same is true in (15). This does not therefore imply that {K_{n}(t, ·), n ≥ 1} is Cauchy which is needed for the representation (10). So the method fails here, and it is not clear that the result extends to non Gaussian systems perhaps based on another procedure. The contrast between the general (vector) representation of conditional measures of Section 2 and that of the above special process case should be recognized.

2. There are cases where conditional expectations can be exactly calculated using proper differentiation procedures. A wide class of processes for which this is possible is detailed in ([18], Section 7.5), and some with Fourier analysis methods is also in ([18], Section 5.6). It should perhaps be observed in passing that an axiomatic approach, such as the one proposed by Rényi, could not resolve the paradoxes discussed early in this section. [For a more detailed analysis of this “computational problem”, the reader is referred to [18], Chapter 4.]

References

1. T. W. Anderson, An introduction to multivariate statistical analysis. 2nd ed. Wiley, New York, 1984.

2. J. L. Doob, Stochastic processes. Wiley, New York, 1953.

3. N. Dunford and J. T. Schwartz, Linear operators. Part I: General theory. Wiley-Interscience, New York, 1958.

4. A. Ionescu Tulcea and C. Ionescu Tulcea, Topics in the theory of lifting. Springer, New York, 1969.

5. M. Kac and D. Slepian, Large excursions of Gaussian processes. Ann. Math. Statist. 30(1959), 1215–1228.

6. J. L. Kelley and T. P. Srinivasan, On the Bochner integral. Proc. Conf. Vector and Operator Valued Measures and Applications, 165–174, Academic Press, New York, 1973.

7. A. N. Kolmogorov, Foundations of the theory of probability. Chelsea, New York, 1933/1956. (This is a 1956 English translation, and Chelsea publication, of the original 1933 German edition published by Springer, Berlin.)

8. J. Kupka, Radon–Nikodým theorems for vector valued measures. Trans. Amer. Math. Soc. 169(1972), 197–217.

9. R. S. Liptser and A. N. Shiryayev, Statistics of random processes. I, II. Springer, New York, 1977.

10. M. Loève, Probability theory. D. Van Nostrand, Princeton, N. J., 1955.

11. H. B. Maynard, A Radon–Nikodým theorem for operator-valued measures. Trans. Amer. Math. Soc. 173(1972), 449–463.

12. P. A. Meyer, Sur une probleme de filtration. Sem. d. Proba.-VII, Lecture Notes in Math. 321(1973), 223–247.

13. S. Moedomo and J. J. Uhl, jr., Radon–Nikodým theorems for the Bochner and Pettis integrals. Pacific J. Math. 38(1971), 531–536.

14. R. S. Phillips, On weakly compact subsets of a Banach space. Amer. J. Math. 65(1943), 108–136.

15. M. M. Rao, Abstract Lebesgue–Radon–Nikodým theorems. Ann. Mat. Pura Appl. 76(1967), 107–132.

16. M. M. Rao, Remarks on a Radon–Nikodým theorem for vector measures. Proc. Conf. Vector and Operator Valued Measures and Applications, 303–317, Academic Press, New York, 1973.

17. M. M. Rao, Stochastic processes: inference theory. Kluwer Academic, Dordrect, The Netherlands, 2000.

18. M. M. Rao, Conditional measures and applications. Marcel Dekker, New York, 1993.

19. M. M. Rao, Stochastic processes and integration. Sijthoff and Noordhoff, Alphen aan den Rijn, The Netherlands, 1979.

20. M. A. Rieffel, The Radon–Nikodým theorem for the Bochner integral. Trans. Amer. Math. Soc. 131(1968), 466–487.

21. V. V. Sazonov, On perfect measures. Izv. Akad. Nauk SSSR, Ser. Mat. 26(1962), 391– 414; English translation: Amer. Math. Soc. 48(1962), 229–254.

22. N. N. Vakhania, V. I. Tarieladze, and S. A. Chobanyan, Probability distributions on Banach spaces. D. Reidel Publishing Co., Dordrecht, 1987.

23. J. D. M. Wright, A Radon–Nikodým theorem for Stone algebra valued measures. Trans. Amer. Math. Soc. 139(1969), 75–94.

(Received 4.04.2000)

Author’s address:

Department of Mathematics University of California Riverside, CA 92521 U.S.A. E-mail: rao@math.ucr.edu