Journal of Convex Analysis Volume 3 (1996), No.1, 97–116

Favorable Classes of Mappings and Multimappings In Nonlinear Analysis and Optimization

Jean-Paul Penot

Département de Mathématiques Appliquéees, CNRS URA 1204, Université de Pau, 64000 Pau, France. e-mail: jean-paul.penot@univ-pau.fr

Received August 5, 1994 Revised manuscript received September 23, 1995

Dedicated to R. T. Rockafellar on his 60th Birthday

A generalization of the class of lower C^{k}−functions introduced by R.T. Rockafellar [28] called lower T^{k}−functions is proposed in the infinite dimensional case. Mappings of class T^{k} are studied for themselves as they seem to deserve some attention. Other classes of functions such as subconvex functions, subsmooth functions, semismooth functions are either introduced or extended to the case of an infinite dimensional Banach space.

R.T. Rockafellar has pointed out in [28] some classes of functions on an open subset W of some Euclidean space which are important from the point of view of nonsmooth analysis. It is our purpose here to extend his study to the infinite dimensional situation (see section 3) and to delineate some notions close to the class of semismooth functions introduced by Mifflin [20] (see section 4). We also deal with generalizations of the class of submonotone multimappings considered by Spingarn [32] (see section 2). The lack of local compactness of the space leads us to consider directional convergence, as in [8], [20], [32], rather than ordinary convergence. We also consider (in section 1) a class of Gâteaux differentiable mappings whose derivatives satisfy a mild continuity property. We call it the class of T^{k}-mappings because its definition involves the tangent functor of differential geometry. In finite dimensional spaces it coincides with the class of C^{k}-mappings. It seems to play an important role in nonlinear analysis; in particular it fits well the case of superposition operators (or Nemitskii operators) between L_{p} spaces. This class is used to define a generalization of the notion of lower-C^{k} function introduced in [28]; we call it the class of lower-T^{k} functions. In turn, we show that any lower-T^{k} function is semismooth for a large class of subdifferentials. For the applicability to algorithms of the notions considered here, we refer the reader to [21] for instance.

ISSN 0944-6532 / $ 2.50 ◯^{c} Heldermann Verlag

98 J.-P. Penot / Favorable classes of mappings and multimappings

1. Mappings of class T^{k}

In the sequel, by t.v.s. we mean a Hausdorff topological vector space ; the reader may suppose all t.v.s. considered here are normed vector spaces (n.v.s.). Even when X and Y are n.v.s. the space L (X, Y) of continuous linear maps from X to Y can be given various interesting topologies ; the same is true for the space L^{n} (X_{1}, ..., X_{n}; Y) of continuous n-linear maps from a product X_{1} × ... × X_{n} of t.v.s. into Y or for the space L^{n} (X; Y) := L^{n} (X × ... × X; Y). The following definition allows one to avoid considering any topology or convergence on L (X, Y) or L^{n} (X_{1}, ..., X_{n}; Y) ; in fact it is linked with what is called continuous convergence [12], [15].

Definition 1.1. Given a topological space W and t.v.s. X_{1}, ..., X_{n}, Y, a mapping

f : W → L^{n} (X_{1}, ..., X_{n}; Y)

is said to be of class T^{0} if the associated mapping f̂ = ev ∘ (f × I_{X_{1}×...×X_{n}}) : W × X_{1} × ... × X_{n} → Y given by

f̂ (w, v_{1}, ..., v_{n}) = f (w) (v_{1}, ..., v_{n})  (1)

is continuous.

Here ev : L^{n} (X_{1}, ..., X_{n}; Y) × X_{1} × ... × X_{n} → Y is the evaluation mapping given by ev (u, v_{1}, ..., v_{n}) = u (v_{1}, ..., v_{n}) for u ∈ L^{n} (X_{1}, ..., X_{n}; Y) , v_{i} ∈ X_{i}, i = 1, ..., n. The preceding notion is not only simple ; it also enjoys interesting functorial properties which make it usable. By this we mean, as in category theory, that crucial composition properties hold .

Proposition 1.2. Let W be a topological space and let X, Y, Z, X_{1}, ..., X_{n}, Y_{1}, ..., Y_{p} be t.v.s. Given f : W → L (X, Y) , g : W → L (Y, Z) of class T^{0} the mapping h : W → L (X, Z) defined by h (w) = g (w) ∘ f (w) is of class T^{0}.

More generally, given n_{1}, ..., n_{p} ∈ IN\{0} with n_{1}+...+n_{p} = n, f_{i} : W → L^{n_{i}}(X_{m_{i}+1}× ...× X_{m_{i}+n_{i}}; Y_{i}) with m_{1} = 0, m_{i} = n_{1} + ... + n_{i−1} for i = 2, ..., p, g : W → L^{p} (Y_{1}, ..., Y_{p}; Z) , the mapping h : W → L^{n} (X_{1}, ..., X_{n}; Z) given by

h (w) = g (w) ∘ (f_{1} (w) × ... × f_{p} (w)) ,

is of class T^{0}.

Proof. The two assertions are immediate consequences of the formulae

ĥ (w, v) = ĝ (w, f̂ (w, v))

ĥ (w, v_{1}, ..., v_{n}) = ĝ (w, f̂_{1} (w, v_{1}, ..., v_{n_{1}}) , ..., f̂_{p} (w, v_{m_{p}+1}, ..., v_{n})) .

It follows from the preceding proposition that the notion of mapping of class T^{0} can be used to define a category of vector bundles. We shall not pursue this route here but instead introduce a notion of weaken continuous differentiability useful for applications. Let us first present a useful characterization of the class T^{0}.

When X and Y are n.v.s. and L (X, Y) is given the topology induced by its usual norm, any continuous mapping f : W → L (X, Y) is of class T^{0}, the converse being false in general, as the following characterization shows (take Y = IR and W = IN ∪ {∞} and consider a sequence in X^{∗} which is weakly^{∗} convergent but not strongly convergent).

Proposition 1.3. Let W be a topological space, let X, Y be n.v.s. and let f : W → L (X, Y) . Then the following assertions (a) and (b) are equivalent; if moreover W is metrizable and X is complete these assertions are equivalent to assertion (c):

(a) f is of class T^{0};

(b) f is locally bounded and continuous when L (X, Y) is given the topology of pointwise convergence;

(c) for each v ∈ X the mapping w ↦ f (w) v := f(w)(v) is continuous.

Proof. Clearly (a) implies (c) i.e. the second part of assertion (b). Moreover for each w ∈ W, f̂(w, 0) = 0 and as f̂ is continuous at (w, 0) there exists r > 0 and a neighborhood U of w in W such that

f̂ (U × B (0, r)) ⊂ B (0, 1) .

Thus ∥f (u)∥ ≤ r^{−1} for u ∈ U and (b) holds. Conversely let us suppose (b) holds. Then for each (w, x) ∈ W × X there exists m ∈ IR_{+} and a neighborhood U of w in W such that ∥f (u)∥ ≤ m for u ∈ U. Given ε > 0 let r > 0 be such that mr < ε/2 and let U′ be a neighborhood of w contained in U such that ∥f (u) x − f (w) x∥ < ε/2 for u ∈ U′. Then, for (u, v) ∈ U′ × B (x, r) one has

∥f̂ (u, v) − f̂ (w, x) ∥ ≤ ∥f (u) (v − x) ∥ + ∥f (u) x − f (w) x∥ ≤ ε,

so that f̂ is continuous at (w, x) . Finally the implication (c) ⟹ (a) is a consequence of [7] Th 3, chap. III, paragraph 3, number 6, p. 28.

For more general assumptions on the involved spaces we refer to [12]. In the sequel W is an open subset of a t.v.s. X and Y is another t.v.s. .

Definition 1.4. A mapping f : W → Y is said to be of class T^{1} if it is continuous, radially differentiable and if f′ : W → L (X, Y) is of class T^{0} (or, equivalently, if df : W × X → Y, given by

df (w, v) = f′ (w) (v) for (w, v) ∈ W × X  (2)

is continuous on W × X).

Here f is said to be radially differentiable (or Gâteaux-differentiable) at x if for each v ∈ X the limit

f′(x, v) = lim_{t→0_{+}} \frac{1}{t} (f (x + tv) − f (x))

exists and is linear and continuous as a function of v, so that we denote it by f′(x)(v) or df(x, v). In fact, we often have a stronger differentiability property known as directional differentiability or Hadamard differentiability (at least when X and Y are n.v.s.) or MB-differentiability [4] or differentiability in the full limit sense [29].

Proposition 1.5. If f : W → Y is of class T^{1} and if Y is a locally convex t.v.s. then f is directionally differentiable on X : for each (x, v) ∈ W × X one has

f′(x)v = lim_{{t↘0 \atop u→v}} \frac{1}{t}(f(x + tu) − f(x)).

It is easy to give examples showing that in general one does not have Fréchet differentiability.

Proof. Let us set for s ∈ [0, 1] , t > 0, x ∈ W, u, v ∈ X

g_{t,u} (s) = t^{−1}f (x + tv + st (u − v)) .

Since

gt,u^{′} (s) = f′ (x + tv + st (u − v)) (u − v) → 0

uniformly in s ∈ [0, 1] as (t, u) → (0_{+}, v) , the Mean Value Theorem (which can also be called the Mean Value Inequality) entails that for each continuous seminorm q on Y one has, with a proof similar to the one in the case of a norm,

\begin{aligned}q (t^{−1} (f (x + tu) − f (x + tv))) & = q (g_{t,u} (1) − g_{t,u} (0)) \\ & ≤ sup_{s∈[0,1]} q (gt,u^{′} (s)) → 0 \end{aligned}

as (t, u) → (0_{+}, v) , hence the result.

Any Gâteaux differentiable continuous convex function f : W → IR is of class T^{1}. In fact for any (x, v) ∈ W × X and any net or sequence ((x_{n}, v_{n})) → (x, v) one has

lim sup_{n} f′ (x_{n}, v_{n}) ≤ f′ (x, v)

since for each r > f′ (x, v) we can find s > 0 with s^{−1} (f (x + sv) − f (x)) < r so that

f′ (x_{n}, v_{n}) = inf_{t>0} t^{−1} (f (x_{n} + tv_{n}) − f (x_{n})) ≤ s^{−1} (f (x_{n} + sv_{n}) − f (x_{n})) < r

for n large enough. Changing v_{n} into −v_{n} we get that (f′ (x_{n}, v_{n})) → f′ (x, v).

The following observation is an immediate consequence of the implication (a)⟹(b) of Proposition 1.3 and of the Mean Value Inequality.

Corollary 1.6. If X and Y are n.v.s., any mapping f : W → Y of class T^{1} is locally Lipschitzian.

The terminology we adopt stems from the following observation : f : W → Y is of class T^{1} iff its tangent mapping

T f : W × X → Y × Y

given by

T f (w, v) = (f (w) , f′ (w) v)  (3)

is continuous. Since this operation is functorial, i.e. T (g ∘ f) = T g ∘ T f we get the following consequence (which can also be derived from Proposition 1.3).

Corollary 1.7. If f : V → W, g : W → Z where V (resp. W) is an open subset of X (resp. Y) are of class T^{1}, then g ∘ f : V → Z is of class T^{1}. We are ready to give the following definition.

Definition 1.8. A mapping f : W → Y is of class T^{k} for k ≥ 2 if f is radially differentiable and if f and T f : W × X → Y × Y given by T f(w, x) := (f(w), df(w, x)) are of class T^{k−1}. Thus f is of class T^{k} (k ≥ 1) iff f is of class T^{1} and if df : X × X → Y is of class T^{k−1}. An immediate induction using the relation T (g ∘ f) = T g ∘ T f shows the following composition property.

Proposition 1.9. The composition of two mappings of class T^{k} is of class T^{k}.

Corollary 1.10. Any mapping of class C^{k} is of class T^{k}. If f : W → Y is of class T^{k}, for any finite dimensional vector subspace V of X, f | W ∩ V is of class C^{k}. If f : W → Y is of class T^{k}, the k^{th}-derivative of f defined inductively by

f^{(k)} (x) (v_{1}, ..., v_{k}) = dg_{v_{1},...,v_{k−1}} (x, v_{k})  (4)

with g_{v_{1},...,v_{k−1}} (w) = f^{(k−1)} (w) (v_{1}, ..., v_{k−1}) is well defined, symmetric in v_{1}, ..., v_{k} and continuous in x, v_{1}, ..., v_{k}.

Proof. The first assertion is an immediate consequence of Proposition 1.3; the second one follows from the corresponding property for the restriction of f to W ∩ V, where V is the vector subspace generated by x, v_{1}, ..., v_{k}. In fact the last result yields a characterization.

Proposition 1.11. The mapping f : W → Y is of class T^{k} with k ≥ 1 iff it is continuous and its j^{th}-derivatives f^{(j)} : W → L^{j} (X; Y) are well defined for j = 1, ..., k and are of class T^{0} iff it is continuous and its k^{th}-derivative is well defined and of class T^{0}.

Proof. It is enough to prove the last sufficient condition. For k = 1 the assertion follows from the definitions since df (w, v) = f′ (w) (v) . Suppose the result holds for k − 1 with k ≥ 2. One has to prove that df : W × X → Y is of class T^{k−1}. Using the induction hypothesis it is enough to show that (df)^{(k−1)} : W × X → L^{k−1} (X^{2}; Y ) is well-defined and of class T^{0}. Now an easy induction shows that

\begin{aligned}(df)^{(k−1)} (w, x) (u_{1}, v_{1}) ... (u_{k−1}, v_{k−1}) & = f^{(k)} (w) (x, u_{1}, ..., u_{k−1}) \\ +f^{(k−1)} (w) (v_{1}, u_{2}, ..., u_{k−1}) + ... & + f^{(k−1)} (w) (u_{1}, ..., u_{j−1}, v_{j}, u_{j+1}, ..., u_{k−1}) \\ & + f^{(k−1)} (w) (u_{1}, ..., u_{k−2}, v_{k−1}) . \end{aligned} (5)

It follows that (df)^{(k−1)} is of class T^{0}.

An interesting connexion with the class of C^{k}-mappings can be pointed out.

Proposition 1.12. Suppose X and Y are n.v.s. and f : W → Y is of class T^{k} with k ≥ 2. Then f is of class C^{k−1,1}, i.e. f has Fréchet derivatives of order j ≤ k − 1 which are locally Lipschitzian.

Proof. Let us first suppose k = 2. Then df is of class T^{1} hence (by Corollary 1.6) is locally Lipschitzian. In particular, for any (w, x) ∈ W × X there exists c ∈ IR_{+}, r > 0 such that for (z, x) ∈ B (w, r) × B (0, r)

∥df (w, x) − df (z, x)∥ ≤ c ∥w − z∥

hence

∥f′ (w) − f′ (z)∥ ≤ \frac{c}{r} ∥w − z∥ ,

and f′ is locally Lipschitzian. Since f′ is continuous, the Gâteaux derivative is a Fréchet derivative and hence f is of class C^{1}. Now suppose the result has been proved for k ≤ r, with r > 2 and let us prove it for k = r + 1. By definition df is of class T^{r}, hence has Fréchet derivatives of order j ≤ r − 1 which are locally Lipschitzian. Since by formula (5)

f^{(r)} (w) (x, u_{1}, ..., u_{k−1}) = (df)^{(r−1)} (w, x) (u_{1}, 0) ... (u_{r−1}, 0)

we get the local Lipschitzian behavior of f^{(r)} (.) by an argument similar to the one used in the beginning of the proof.

The following immediate result will be used to obtain Taylor expansions for mappings of class T^{k}.

Lemma 1.13. Let g : [0, 1] × V → L^{n} (X, Y) be of class T^{0}, where V is an open subset of some n.v.s. Then f : V → L^{n} (X, Y) given by

f (v) (x_{1}, ..., x_{n}) = ∫_{0}^{1} g (t, v) (x_{1}, ..., x_{n}) dt

is of class T^{0}.

Theorem 1.14. If W is an open subset of the n.v.s. X and if Y is a Banach space, f : W → Y is of class T^{k}, k ≥ 1 iff there exists mappings f_{j} : W → L^{j} (X; Y) and r : V → L^{k} (X; Y) of class T^{0}, with

V = {(w, x) ∈ W^{2} : ∀t ∈ [0, 1] (1 − t) w + tx ∈ W }

such that r (w, w) = 0 for each w ∈ W and

f (x) − f (w) = ∑_{j=1}^{k} f_{j} (w) (x − w)^{j} + r (w, x) (x − w)^{k} .

Proof. If f is of class T^{k}, its derivatives of order j ≤ k exists and for any w, x ∈ W the restriction of f to W ∩ V, where V is the vector subspace generated by w and x is of class C^{k}. Thus the preceding formula holds with f_{j} (w) = f^{(j)} (w) , r (w, x) =

∫_{0}^{1} \frac{(1−t)^{k−1}}{(k−1)!} (f^{(k)} (w + t (x − w)) − f^{(k)} (w)) dt, so that f_{j} and r are of class T^{0} by the

preceding lemma. Conversely, suppose f satisfies the preceding expansion for some T^{0}-mappings f_{j}, r. It follows from the converse to Taylor’s theorem [1] p.6 that for each finite dimensional vector subspace V of X the restriction of f to W ∩ V is of class C^{k} and

f^{(j)} (w) = f_{j} (w) for j = 1, ..., k, w ∈ W.

The result thus follows from Proposition 1.11.

Let us close this section by an example of fundamental importance : the case of Nemitskii operators. We leave its extension to the case of superposition operators in Sobolev spaces to the reader.

Example 1.15. ([23]) Given p ∈ ]1, ∞[ , separable Banach spaces E, E′ and a finite measured space (T, T , τ), let F : T × E → E′ be measurable and such that (a) for each t ∈ T F_{t} : e ↦ F (t, e) is of class C^{1}, (b) t ↦ F (t, 0) is in L_{p} (T, E′) , (c) there exists

some c ∈ IR_{+} and some b ∈ L_{q} (T, IR) with q = (1 − p^{−1})^{−1} such that for (t, e) ∈ T × E

∥DF_{t} (e)∥ ≤ b (t) + c ∥e∥^{\frac{p}{q}} .

Then, for X = L_{p} (T, E) , Y = L_{1} (T, E′) , the mapping f : X → Y given by f(x) = F (., x (.)) is of class T^{1}. In fact f is well defined, hence continuous by Krasnoselskii’s theorem (see [2] p. 20 for instance). Moreover df exists and is given by

df (x, v) : t ↦ DF_{t} (x (t)) v (t) ,

so that df maps L_{p} (T, E) × L_{p} (T, E) into L_{1} (T, E′), hence is continuous. If p = 1 and if the growth condition on F is replaced with

∥DF_{t} (e)∥ ≤ b (t)

for some b ∈ L_{∞} (T, IR), then again f is of class T^{1} but it is not of class C^{1} in general. Similarly, if p ∈ ]1, ∞[ and conditions (a), (b), (c) are replaced with (a’) for each t ∈ T the mapping F_{t} : e ↦ F (t, e) is of class C^{2}, (b’) t ↦ F (t, 0) and t ↦ DF_{t} (0) are in L_{p} (T, E′) and L_{q} (T, L (E, E′)) respectively ; (c’) there exists c ∈ IR_{+} such that ∥D^{2}F_{t} (e)∥ ≤ c for each (t, e) ∈ T × E, then f is of class T^{2} but it is not of class C^{2} in general (see [2] p. 24-26). This fact has been noticed by Skripnik [30], [31] when examining the famous Palais-Smale theory.

2. Directional Submonotonicity

From now on X is a Banach space with dual space X^{∗} and M denotes the class of weak^{∗} closed -valued multimappings M : X\overrightarrow{→}X^{∗} which are locally bounded in this sense that for each x ∈ X there exists r > 0 and m > 0 such that ∥v^{∗}∥ ≤ m for each (v, v^{∗}) ∈ M with v ∈ B (x, r) . Here we identify M and its graph. Let us recall the concept of submonotonicity introduced by J.E. Spingarn [32] : M ∈ M is said to be submonotone at x ∈ X if for each y ∈ M (x)

lim inf_{{x′→x,x′≠x \atop y′∈M(x′)}}⟨y′ − y, \frac{x′ − x}{∥x′ − x∥}⟩ ≥ 0,

so that M is trivially submonotone at x if x ∉ domM = M^{−1} (X^{∗}) . Any monotone multimapping is obviously submonotone. In order to introduce a weakening of the preceding notion let us say that a sequence (x_{n}) of X converges to x in the direction u ∈ X_{0} := X\{0} (or equivalently u ∈ S_{X}, the unit sphere of X) if (x_{n}) → x, x_{n} ≠ x for n large enough and

(∥x_{n} − x∥^{−1} (x_{n} − x)) converges to ∥u∥^{−1} u. We write (x_{n}) →_{u} x. Equivalently, (x_{n})→_{u} x

iff there exist sequences (t_{n}) → 0_{+} and (u_{n}) → u such that x_{n} = x + t_{n}u_{n} for each n ∈IN. We say that (x_{n}) converges directionally to x if there is some u ∈ X_{0} := X \ {0} such that (x_{n})→_{u} x. Our definition slightly differs from the one in [32] which does not discard the case u = 0. Our definition can be related to the notion of sponge introduced by J. Treiman [35] : a subset S of X is called a sponge at x if for any u ∈ X_{0} there exists ε > 0 and a neighborhood V of u in X such that x + [0, ε] V ⊂ S. Clearly, any neighborhood of x is a sponge but when X is infinite dimensional the converse may not hold. The connection is as follows; the proof is easy and left to the reader.

Lemma 2.1. A subset S of X is a sponge at x iff for any sequence which converges directionally to x one has x_{n} ∈ S eventually.

From this characterization one gets that a mapping f : X → Y into a topological space Y is spongiously continuous at x (i.e. for each neighborhood V of f (x) there exists a sponge S at x such that f (S) ⊂ V) iff f is directionally continuous at x in the sense : for any u ∈ X_{0} and any (x_{n})→_{u} x one has (f (x_{n})) → f (x). We observe that the results of this section would be valid if M is replaced with the class M_{S} of weak^{∗}-closed convex-valued multimappings M : X\overrightarrow{→}X^{∗} which are spongiously bounded in this sense that for each x ∈ X there exists a sponge S at x such that M (S) is bounded. It is easy to see that the following notion coincides with the notion of submonotonicity in finite dimensional spaces. In infinite dimensional spaces the larger class of directional submonotone multifunctions seems to be more suitable.

Definition 2.2. A multimapping M : X\overrightarrow{→}X^{∗} is said to be directionally submonotone at x ∈ X if for each y ∈ M (x) and any u ∈ X_{0} = X \ {0} , (x_{n})→_{u} x, y_{n} ∈ M (x_{n}) one has

lim inf_{u}⟨y_{n} − y, \frac{x_{n} − x}{∥x_{n} − x∥}⟩ ≥ 0.  (7)

It is directionally submonotone if it is directionally submonotone at each x ∈ X.

Let us first relate this condition to a special kind of directional upper semicontinuity we call directional exposability in order to avoid confusions with what would correspond to genuine directional upper continuity. Let us say that M is directionally exposable at x if for any u ∈ X_{0}, (x_{n})→_{u} x, y_{n} ∈ M (x_{n}) and any weak^{∗}-cluster point y_{∞} of (y_{n}) one has y_{∞} ∈ M (x)_{u} with

M (x)_{u} = {y ∈ M (x) : ⟨y, u⟩ = h (M (x) , u)} ,

where h_{C} = h (C, .) is the Hörmander’s support function of the subset C of X^{∗} :

h (C, u) = sup {⟨y, u⟩ : y ∈ C} .

Let us say that M is directionally closed at x if for any u ∈ X_{0}, (x_{n})→_{u} x, y_{n} ∈ M (x_{n}) any weak^{∗}-cluster point of (y_{n}) belongs to M (x). This property is satisfied when M(x) is weak^{∗} closed and convex and M is directionally scalarly upper semicontinuous in the sense that

lim sup_{n} h (M (x_{n}) , v) ≤ h (M (x) , v)

for each v ∈ X and each sequence (x_{n}) which converges directionally to x. The converse holds when M is locally bounded around x. In [8] M is said to be weak directionally closed (WDC) at x when M satisfies the weaker property

lim sup_{n} h (M (x_{n}) , u) ≤ h (M (x) , u)  (8)

whenever u ∈ X_{0}, (x_{n})→_{u} x. We would prefer to call this property pseudo-directional closedness because (8) is valid only for sequences (x_{n}) such that (x_{n}) →_{u} x and not for any directionally convergent sequence, but we keep the terminology of [8] for the reader’s convenience. The proof of the following result is as in [8] Prop. 7 ; as in the assertions of the preceding paragraph it uses the fact that when C is a weak^{∗} closed convex subset of X^{∗} one has

C = {y ∈ X^{∗} : ⟨y, u⟩ ≤ h (C, u) ∀u ∈ X} .

Proposition 2.3. If M is a weak^{∗} closed-valued multimapping from X to X^{∗} which is locally bounded (i.e. M ∈ M) and if M is directionally submonotone and WDC then it is maximal directionally submonotone i.e. not strictly contained in some directionally submonotone M′ ∈ M.

The following characterization is an easy extension of [32] Thm 1.

Proposition 2.4. The multimapping M ∈ M is directionally submonotone and directionally closed at x iff it is directionally exposable at x.

Proof. Suppose M is directionally submonotone at x and directionally closed at x. Let u ∈ X_{0}, (x_{n})→_{u} x, y_{n} ∈ M (x_{n}) and let y_{∞} be a weak^{∗}-cluster point of (y_{n}). We have y_{∞} ∈ M (x) by directional closedness. Moreover, since (y_{n}) is bounded, for each y ∈ M (x) we have

⟨y_{∞} − y, u⟩ ≥ lim inf_{u}⟨y_{n} − y, \frac{x_{n} − x}{∥x_{n} − x∥}⟩ ≥ 0,

hence ⟨y_{∞}, u⟩ ≥ h (M (x) , u) and M is directionally exposable at x. Conversely, suppose M is directionally exposable at x. Given u ∈ X_{0}, x ∈ X, (x_{n})→_{u} x, y_{n} ∈ M (x_{n}) , y ∈ M (x) , we take an infinite subset P of IN such that

lim_{n∈P}⟨y_{n} − y, \frac{x_{n} − x}{∥x_{n} − x∥}⟩ = lim inf_{n}⟨y_{n} − y, \frac{x_{n} − x}{∥x_{n} − x∥}⟩.

Since (y_{n})_{n∈P} is bounded we can find a weak^{∗}-convergent subnet (y_{n_{i}})_{i∈I}. Its limit y_{∞} satisfies ⟨y_{∞}, u⟩ ≥ h (M (x) , u) ≥ ⟨y, u⟩, hence

lim_{n∈P}⟨y_{n}, \frac{x_{n} − x}{∥x_{n} − x∥}⟩ = ⟨y_{∞}, u⟩ ≥ ⟨y, u⟩ = lim_{n∈P}⟨y, \frac{x_{n} − x}{∥x_{n} − x∥}⟩

so that (7) holds. Moreover M is directionally closed as any directionally exposable multimapping.

Now let us present a characterization of directional submonotonicity itself.

Proposition 2.5. Let F : X\overrightarrow{→}X^{∗} be bounded on a neighborhood of x . The multimapping F : X\overrightarrow{→}X^{∗} is directionally submonotone at x iff it is scalarly directionally submonotone at x in the following sense: for any u ∈ X_{0} and any (x_{n}) →_{u} x one has

lim inf_{n} −h (F (x_{n}) , −u) ≥ h (F (x) , u) .  (9)

It follows easily from this characterization that the sum of two directionally submonotone multimappings is directionally submonotone. This property is also an easy consequence of Definition 2.2.

Proof. If this relation does not hold for some u ∈ X_{0} and some (x_{n}) →_{u} x we can find ε > 0 and some infinite subset N of IN such that

−h (F (x_{n}) , −u) < h (F (x) , u) − ε

for each n ∈ N ; thus there exists y_{n} ∈ F (x_{n}) such that

⟨y_{n}, u⟩ < h (F (x) , u) − ε.

Let y ∈ F (x) be such that ⟨y, u⟩ > h (F (x) , u) − \frac{ε}{2}. Then, as (y_{n}) is bounded and as (\frac{x_{n}−x}{∥x_{n}−x∥}) converges to \frac{u}{∥u∥} ,

lim inf_{n}⟨y_{n} − y, \frac{x_{n} − x}{∥x_{n} − x∥}⟩ ≤ −\frac{ε}{2 ∥u∥} < 0

and F is not directionally submonotone. Conversely if relation (9) holds, for any u ∈ X_{0}, for any (x_{n}) →_{u} x and for any y_{n} ∈ F (x_{n}) , y ∈ F (x) we have

lim inf_{n}⟨y_{n} − y, u⟩ ≥ lim inf_{n} −h (F (x_{n}) , −u) − h (F (x) , u) ≥ 0

and since (y_{n} − y) is bounded and ∥x_{n} − x∥^{−1} (x_{n} − x) → ∥u∥^{−1} u, we get (7) and F is directionally submonotone.

In the following statement we call M ∈ M pseudo-directionally scalarly continuous (PDSC) at x if

lim_{n} h (M (x_{n}) , u) = h (M (x) , u)

whenever u ∈ X_{0} and (x_{n}) →_{u} x ; it would be called directionally scalarly continuous if the preceding relation would hold whenever u ∈ X_{0} and (x_{n}) converges directionally to x. Obviously, any scalarly continuous multimapping M (i.e. any multimapping M such that h(M(·), u) is continuous for any u ∈ X) is directionally scalarly continuous, hence PDSC. For directionally submonotone multimappings, a much weaker condition suffices.

Corollary 2.6. If the multimapping M ∈ M is directionally submonotone at x then it is PDSC at x iff it is WDC at x.

Proof. The preceding proposition and the inequality

−h (M (w) , −u) ≤ h (M (w) , u)

for any w ∈ X and any u ∈ X_{0} show that h_{u}(t, v) := h(M(x + tv), u) is l.s.c. at (0_{+}, u) when M is directionally submonotone at x.Therefore, it is continuous at (0_{+}, u) iff it is u.s.c. at (0_{+}, u) .

The preceding circle of ideas can be completed by using the following new concept.

Definition 2.7. The multimapping F : X\overrightarrow{→}X^{∗} is said to be thin at x if for any u ∈ X_{0}, (u_{n}) → u, (t_{n}) → 0_{+} , (y_{n}) with y_{n} ∈ F (x_{n}), x_{n} := x + t_{n}u_{n} the sequence (⟨y_{n}, u⟩) has a limit.

For instance, if f : X → X^{∗} is of class T^{0}, then the multimapping F whose graph is the graph of f is thin at each point of W.

Proposition 2.8. Let M ∈ M . Then M is thin and PDSC at x iff it is directionally submonotone and WDC at x.

Proof. The necessary condition stems from the fact that for any u ∈ X_{0}, (u_{n}) → u, (t_{n}) → 0_{+} , one can find y_{n} ∈ M(x_{n}), x_{n} := x + t_{n}u_{n} with ⟨y_{n}, u⟩ = h(M(x_{n}), u). Now we observe that the limit of (⟨y_{n}, u⟩) does not depend on the choice of y_{n} in M(x_{n}) since with another choice z_{n} we can duplicate the sequence (x_{n}) and take alternatively y_{n} and z_{n}. Therefore

lim_{n}⟨y_{n}, u⟩ = lim_{n} h(M(x_{n}), u) = h(M(x), u)

as M is PDSC. Thus M is directionally submonotone, and obviously WDC. The sufficient condition follows from Corollary 2.6 and the fact that for any u ∈ X_{0}, (u_{n}) → u, (t_{n}) → 0_{+} , (y_{n}) with y_{n} ∈ M(x_{n}), x_{n} := x + t_{n}u_{n} one has

−h (M (x_{n}) , −u) ≤ ⟨y_{n}, u⟩ ≤ h (M (x_{n}) , u)

and we can use (8) and the fact that M is scalarly directionally submonotone to obtain that M is thin at x.

3. Lower-C^{k} and lower-T^{k} functions

The following definition incorporates a notion introduced by R.T. Rockafellar [28]. In the sequel W is an open subset of a normed vector space X and k ∈ IN, k ≥ 1.

Definition 3.1. A function f : W → IR is said to be lower-T^{k} (resp. lower-C^{k}) if for each w ∈ W there exists an open neighborhood V of w in W, a compact topological space S and a function F : S × V → IR which has radial derivatives of order j ≤ k as a function of its second variable which are of class T^{0} (resp. C^{0}) (with respect to both variables) and such that

f (x) = sup_{s∈S} F (s, x) for x ∈ V.

We set F_{s} (x) = F (s, x) for (s, x) ∈ S × V and for j = 1, ..., k

D_{X}^{j}F (s, x) = F_{s}^{(j)} (x) .

We say that f is lower-T^{∞} (resp. lower-C^{∞}) if f is lower-T^{k} (resp. lower-C^{k}) for each k ∈ IN \ {0}. We denote by LT^{k} (X) (resp. LC^{k} (W)) the space of lower-T^{k} (resp. lower-C^{k}) functions on W. Other notations are introduced in the following statement which also uses the index of nonconvexity of f : W → IR at w ∈ W introduced in [26] as the infimum c (f, w) of the set of nonnegative real numbers c such that f + \frac{1}{2}c ∥·∥^{2} is convex on some convex neighborhood of w, with the usual convention inf ∅ = +∞.

Theorem 3.2. Let W be an open subset of a Hilbert space X. The following assertions on a function f : W → IR are equivalent :

(a) f is lower-T^{∞} : f ∈ LT^{∞} (W) ;

(b) f is lower-T^{2} : f ∈ LT^{2} (W) ;

(c) for each w ∈ W one has c (f, w) < ∞ ;

(d) f ∈ NΓ_{loc} (W): locally f = g − \frac{1}{2}c ∥·∥^{2} with g convex l.s.c., c ∈ IR_{+} ;

(e) f ∈ QΓ_{loc} (W): locally f = g − h with g convex l.s.c., h continuous quadratic ;

(f) f ∈ C^{∞}Γ_{loc} (W): locally f = g − h with g convex l.s.c., h of class C^{∞} ;

(g) f ∈ C^{2}Γ_{loc} (W): locally f = g − h with g convex l.s.c., h of class C^{2} ;

(h) f ∈ T^{2}Γ_{loc} (W): locally f = g − h with g convex l.s.c., h of class T^{2};

(i) f is lower C^{∞} : f ∈ LC^{∞} (W) .

Proof. The implications (a) ⟹ (b),(c) ⟺ (d), (d)⟹ (e)⟹ (f)⟹ (g)⟹ (h),(i)⟹ (a) are obvious; the implication (a)⟹ (i) can be proved as in Proposition 1.12.We intend to prove that (b)⟹ (d),(h)⟹ (b) and (f)⟹ (a). We closely follow some arguments of [28], especially for the implications (b)⟹ (d), and (f)⟹ (a).

(b) ⟹ (d) Given w ∈ W, let F : S × V → IR yield a representation of f as in Definition 3.1 , F being of class T^{2} in x. Since the mapping (s, x, u) ↦ F′′_{s} (x) uu is continuous on S × V × X there exists r > 0 and ρ > 0 such that

|F′′_{s} (x) uu| ≤ 1

for (s, x, u) ∈ S × B (w, ρ) × B (0, r). It follows easily that ∥F′′_{s} (x)∥ ≤ r^{−2} for (s, x) ∈ S × B (w, ρ). Then, for c ≥ r^{−2} and for each s ∈ S the function G_{s} : B (w, ρ) → IR given by

G_{s} (x) = F_{s} (x) + \frac{1}{2}c ∥x∥^{2}

satisfies G′′_{s} (x) ≥ 0 for each x ∈ B (w, ρ), hence is convex. Therefore g = sup_{s∈S} G_{s} is convex and for x ∈ B (w, ρ) one has

f (x) = sup_{s∈S} F_{s} (x) = g (x) − \frac{1}{2}c ∥x∥^{2} .

(f)⟹ (a) (resp. (h)⟹ (b)). Given w ∈ W let U be an open convex neighborhood of w in W such that f |U = g − h with g : U → IR convex l.s.c. and h : U → IR of class C^{∞} (resp. T^{2}). By a well known consequence of the Baire category theorem g is continuous. Let V be a bounded closed convex neighborhood of w contained in U on which g and ∂g are bounded (it is known that ∂g is locally bounded [27]; that also follows from the local Lipschitz property of g). Let B be a closed ball in X^{∗} containing ∂g (x) for each x ∈ V and let

S = {(y, t) ∈ B × [−m, m] : t ≥ g^{∗} (y)} ,

where m = sup {|⟨y, v⟩| + |g (v)| : v ∈ V, y ∈ B} and g^{∗} is the conjugate function of g. Since g^{∗} is l.s.c. when X^{∗} is endowed with the weak^{∗} topology, S is closed in B ×[−m, m], hence compact. Now for x ∈ V

g (x) = sup {⟨y, x⟩ − t : (y, t) ∈ S}

since g (x) = sup {⟨y, x⟩ − g^{∗} (y) : y ∈ X^{∗}}, the supremum being attained for y ∈ ∂g (x) ⊂ B, with

|g^{∗} (y)| ≤| ⟨y, x⟩ | + |g (x)| ≤ m,

so that S is nonempty. Finally, for x ∈ V one has

f (x) = sup {F (s, x) : s ∈ S}

with F (s, x) = G (s, x) − h (x) , G (s, x) = ⟨y, x⟩ − t for s = (y, t) in S. Since Gs^{′} (x) v =

⟨y, v⟩ and G′′_{s} (x) = 0 we see that G is of class T^{∞}. Therefore F is of class T^{∞} (resp. T^{2}) in the variable x when h is of class T^{∞} (resp. T^{2}).

Let us give a characterization of the lineality space of the convex cone LT^{∞} (W), i.e. the greatest vector subspace contained in LT^{∞} (W). Its proof is inspired by [13].

Proposition 3.3. Let W be an open subset of a Hilbert space. The lineality space of LT^{∞} (W) is the space C^{1,1} (W) of functions of class C^{1} on W with locally Lipschitzian derivatives :

LT^{∞} (W) ∩ (−LT^{∞} (W)) = C^{1,1} (W) .

Proof. Let f ∈ C^{1,1} (W). Then for any w ∈ W we can find a convex neighborhood U of w and c ∈ IR_{+} with

|∇f (x) − ∇f (y)| ≤ c ∥x − y∥ x, y ∈ U.

Then by the Cauchy-Schwarz inequality we get that x ↦ ∇f (x)+cx and x ↦ −∇f (x)+ cx are monotone so that f + \frac{1}{2}c ∥·∥^{2} and −f + \frac{1}{2}c ∥·∥^{2} are convex on U. Therefore f ∈ LT^{∞} (W) and −f ∈ LT^{∞} (W) . Conversely, let f ∈ LT^{∞} (W) ∩ (−LT^{∞} (W)). Given w ∈ W we can find c ∈ IR_{+} and an open convex subset U of W with w ∈ U such that g := f+\frac{1}{2}c ∥·∥^{2} and h := −f+\frac{1}{2}c ∥·∥^{2} are convex on U. Then, for each x ∈ U, v ↦ f′ (x, v) = g′ (x, v)−c (x |v) and v ↦ −f′ (x, v) = h′ (x, v)−c (x |v) are sublinear and l.s.c.. Thus f′ (x, .) is linear and continuous. Moreover

|(∇f (u) − ∇f (v) | u − v)| ≤ c ∥u − v∥^{2} for u, v ∈ U

by the monotonicity of ∇g and ∇h. Let us show that this entails

∥∇f (u) − ∇f(v)∥ ≤ c ∥u − v∥

for u, v in U. Given u and v in U it suffices to show that for any finite dimensional vector subspace Z of X containing u and v the restriction f_{Z} of f to U ∩ Z has a Lipschitzian gradient ∇f_{Z} with rate c. Now, by a famous result of Alexandroff, g, hence f, is twice differentiable on a subset D of U ∩ Z with U ∩ Z \ D negligible. For u ∈ D, z ∈ Z

|f′′_{Z} (u) zz| = lim_{t→0} |t^{−2}(∇f (u + tz) − ∇f (u) | tz)| ≤ c ∥z∥^{2}

so that ∥f′′_{Z} (u)∥ ≤ c. Therefore we can find sequences (u_{n}) , (v_{n}) in U with limits u and v respectively and (1 − t) u_{n} + tv_{n} ∈ D for a.e. t ∈ [0, 1] and

\begin{aligned}∥∇f_{Z} (u) − ∇f_{Z} (v)∥ & = lim ∥∇f_{Z} (u_{n}) − ∇f_{Z} (v_{n})∥ \\ & = lim |∫_{0}^{1} f′′_{Z} ((1 − t) u_{n} + tv_{n}) (u_{n} − v_{n}) dt| ≤ c ∥u − v∥ . \end{aligned}

Let us conclude the present section by observing that although we proved here that a function f : W → IR is of lower-T^{k} (k ≥ 2) iff it is lower-C^{∞} when W is an open subset of a Hilbert space X, we have not proved such an identification for a general n.v.s.. Moreover we have not considered the case of vector-valued mappings.

4. Semismoothness and Subsmoothness

In this section we deal with a subdifferential operator ∂ considered as a mapping ∂ : L → M, where L := L(W) is the class of locally Lipschitzian functions on W. A priory, we do not impose any condition on ∂. However, some of our statements will require some of the following natural properties.

Definition 4.1. The subdifferential ∂ is said to be localizable if for each open subset V of W one has ∂f(v) = ∂g(v) for each v ∈ V whenever f and g coincide on V; symmetric if ∂(−f) = −∂f for each f ∈ L; contingently consistent if ∂f(x) ⊇ ∂^{!}f(x) for each f ∈ L and each x ∈ W, where ∂^{!} is the contingent subdifferential given by:

∂^{!}f(x) := {y ∈ X^{∗} : ∀v ∈ X ⟨y, v⟩ ≤ f^{!}(x, v)} ,

where

f^{!}(x, v) := lim inf_{(t,u)→(0_{+},v)} \frac{1}{t}(f(x + tu) − f(x))

is the contingent (or lower epi-derivative) of f at x in the direction v.

Most known subdifferentials satisfy the latter condition (but the Fréchet subdifferential ∂^{−} and the proximal subdifferential ∂^{π} do not ). The symmetry condition is satisfied by the Clarke subdifferential [9] and the moderate subdifferential ∂ of Michel-Penot [5], [6], [18], [19], but it is not satisfied by the contingent subdifferential nor the limiting subdifferential of Mordukhovich, Kruger and Ioffe ([14], [16], [22], for instance). Another property will be useful; it is a form of the Mean Value Theorem. In the case of the Clarke-Rockafellar subdifferential ∂^{↑} it has been devised by D. Zagrodny [36] following some previous work of J.-P. Penot [24] dealing with the contingent derivatives and introducing the idea of approximate Mean Value Inequalities; see also [37] in this connection.

Definition 4.2. The subdifferential ∂ is said to be valuable on X (or, alternatively, the space X is said to be ∂-valuable) if for any f in L and for any a, b ∈ X with [a, b] ⊂ W, a ≠ b, there exist c ∈ [a, b] , c ≠ a and sequences (c_{n}) → c, (c′_{n}) with c′_{n} ∈ ∂f(c_{n}) for each n such that

f(a) − f(b) ≤ lim inf_{n}⟨c′_{n}, a − b⟩.

From the recent works of several authors (see [3], [17], [25], [34], [36], [37] and their references for instance) it follows that ∂ is valuable whenever ∂ = ∂^{↑}, the Clarke subdifferential; it is also the case for ∂ = ∂^{−}, the Fréchet subdifferential, when X is an Asplund space, or ∂ = ∂^{!} when X is a reliable space ([25]), in particular when X has a Lipschitzian bump function of class T^{1}. Applications of the preceding form of the Mean Value Inequality can be found in the preceding references; for related results about characterizations of Lipschitzness and monotony see also [10]. The following definition will be convenient; in it we write ∂- with parenthesis as we drop the reference to ∂ if there is no risk of confusion.

Definition 4.3. The function f ∈ L is said to be (∂-)regular if the directional derivative f′ of f exists and if for any x ∈ W, u ∈ X one has f′(x, u) = d^{∂}f(x, u) where

d^{∂}f(x, u) := h(∂f(x), u).

It is said to be (directionally) (∂-)subconvex if ∂f is (directionally) submonotone. It is said to be (∂-)subsmooth if ∂f is PDSC. Thus, f is subsmooth if for any u ∈ X_{0} and any sequences (u_{n}) → u, (t_{n}) → 0_{+} one has d^{∂}f(x, u) = lim_{n} d^{∂}f(x + t_{n}u_{n}, u).

Examples. If for a function f of class T^{1} one has ∂f(x) = {f′(x)} , which is the case for ∂ = ∂^{!} or ∂ = ∂ or ∂ = ∂^{↑}, as easily seen, one gets that f is ∂-subsmooth and ∂-subconvex. If f = max_{i∈I} f_{i}, where I is a finite set and f_{i} is continuous and if ∂f(w) = \overline{co}(∪_{i∈I(w)} ∂f_{i}(w)) for each w ∈ W, where \overline{co}(A) denotes the weak^{∗} closed convex hull of A and I(w) := {i ∈ I : f_{i}(w) = f(w)} then f is ∂-regular (resp. ∂-subconvex, resp. ∂-subsmooth) whenever each f_{i} has this property.

Taking into account [20] Lemma 2, ∂^{↑}−subsmoothness (i.e. subsmoothness with respect to ∂^{↑}) in the following definition coincides with the original definition of semismoothness by Mifflin ([20]). Here we follow the choices of [11] (see also [8]) and we use the terminology ”directional differentiability” for the concept of Hadamard differentiability, for the sake of consistency with the preceding notions. Note that for elements of L directional differentiability coincides with radial (or Gâteaux-) differentiability.

Definition 4.4. The function f ∈ L is said to be semismooth at x ∈ W if f is directionally differentiable at x and if ∂f is thin at x with f′(x, u) = lim_{n}⟨y_{n}, u⟩ for each (x_{n}) →_{u} x and each y_{n} ∈ ∂f(x_{n}).

Proposition 4.5. Suppose ∂ is valuable on X. Then any directionally subconvex function f in L is subregular in the following sense: for any (x, u) ∈ W × X one has

d^{∂}f(x, u) ≤ f^{!}(x, u),  (10)

hence

∂f(x) ⊂ ∂^{!}f(x).

In particular, if ∂ is valuable and contingently consistent, for any directionally subconvex function f one has ∂f = ∂^{!}f.

Proof. Let (x, u) ∈ W × X and let (t_{n}) → 0_{+}, (u_{n}) → u be such that

f^{!}(x, u) = lim_{n} t_{n}^{−1}(f(x + t_{n} u_{n}) − f(x)).

Since ∂ is valuable one can find θ_{n} ∈ ]0, 1] , x_{n} ∈ W, y_{n} ∈ ∂f(x_{n}) such that ∥x + θ_{n}t_{n}u_{n} − x_{n}∥ < θ_{n}t_{n}^{2} and

f(x) − f(x + t_{n}u_{n}) ≤ ⟨y_{n}, −t_{n}u_{n}⟩ + t_{n}^{2}.

It follows that (x_{n}) →_{u} x and

lim_{n} t_{n}^{−1}(f(x + t_{n} u_{n}) − f(x)) ≥ lim_{n} inf⟨y_{n}, u_{n}⟩ − t_{n} ≥ h(∂f(x), u),

by Preposition 2.5 and the directional submonotonicity of ∂f, and inequality (10) is proved.

The proof of a reverse inequality is more subtle.

Proposition 4.6. Suppose ∂ is valuable on X. Then any function f in L such that ∂f is WDC is super-regular in the following sense: for any (x, u) ∈ W × X one has

d^{∂}f(x, u) ≥ f^{♯}(x, u) := lim_{{v→u \atop t↘0}} sup t^{−1}(f(x + t v) − f(x)) = −(−f)^{!}(x, u).  (11)

Proof. Let (x, u) ∈ W × X and let (t_{n}) → 0_{+}, (u_{n}) → u be such that

f^{♯}(x, u) = lim_{n} t_{n}^{−1}(f(x + t_{n} u_{n}) − f(x)).

In view of the Lipschitzian character of f we could take u_{n} = u, but this choice would not allow us to avoid the following trick.Let us introduce w_{n} = x+r_{n}u_{n} with r_{n} ∈ ]0, t_{n}^{2}] such that | f(w_{n}) − f(x) |< t_{n}^{2}. As ∂ is valuable one can find θ_{n} ∈ [0, 1), x_{n} ∈ W, y_{n} ∈ ∂f(x_{n}) such that ∥w_{n} + θ_{n}(t_{n} − r_{n})u_{n} − x_{n}∥ < r_{n}t_{n}^{2} and

f(x + t_{n}u_{n}) − f(w_{n}) ≤ ⟨y_{n}, (t_{n} − r_{n})u_{n}⟩ + t_{n}^{2}.

Then (x_{n}) →_{u} x since

∥(θ_{n}t_{n} + (1 − θ_{n})r_{n})^{−1}(x_{n} − x) − u∥
≤ (θ_{n}t_{n} + (1 − θ_{n})r_{n})^{−1}r_{n}t_{n}^{2} + ∥(θ_{n}t_{n} + r_{n})^{−1}(w_{n} + θ_{n}t_{n}u_{n} − x) − u_{n}∥ + ∥u_{n} − u∥
≤ t_{n}^{2} + ∥u_{n} − u∥.

Since (⟨y_{n}, u_{n}⟩) is bounded and since ∂f is WDC it follows that

lim_{n} t_{n}^{−1}(f(x + t_{n} u_{n}) − f(x)) ≤ lim_{n} sup⟨y_{n}, u_{n}⟩ − t_{n}^{−1}r_{n}⟨y_{n}, u_{n}⟩ + 2t_{n} ≤ h(∂f(x), u).

Corollary 4.7. Suppose ∂ is valuable on X. Then any directionally subconvex function f in L such that ∂f is WDC is regular and semismooth.

Proof. Using the obvious inequality −(−f)^{!}(x, u) ≥ f^{!}(x, u) and relations (10), (11) we get

−(−f)^{!}(x, u) ≥ f^{!}(x, u) ≥ d^{∂}f(x, u) ≥ −(−f)^{!}(x, u),

so that f is directionally differentiable and ∂-regular. Then we apply Proposition 2.8.

Corollary 4.8. Suppose ∂ is valuable on X. Then any semismooth and subsmooth function in L is regular and directionally subconvex.

Proof. For such a function f, the multimapping ∂f is both PDSC and thin, hence WDC and directionally submonotone by Proposition 2.8. The proofs of the preceding two propositions show the following useful criterion. As mentioned above, it applies in particular for the Clarke’s subdifferential ∂^{↑}. Proposition 4.9. Suppose ∂ is valuable and ∂f is WDC. Then f is semismooth iff ∂f is thin. Taking Proposition 2.8 and Corollary 4.7 into account we get the following complementary result. Proposition 4.10. Suppose ∂ is valuable and ∂f is PDSC. If ∂f is thin then f is semismooth and ∂-regular : for each (x, u) ∈ W × X one has

f′(x, u) = h(∂f(x), u).

The following result generalizes [11] Thm 5.1 in this sense that it applies to a wide class of subdifferentials.

Theorem 4.11. Suppose ∂ is valuable on X. Let f ∈ L be directionally differentiable on W and such that ∂f ⊂ ∂^{↑}f. Then f is ∂-semismooth at x ∈ W iff for each u ∈ X_{0} the function (t, v) ⟼ f′(x + tv, u) is continuous at (0_{+}, u).

It follows that the notion of ∂-semismoothness is independent of ∂ for any ∂ contained in ∂^{↑} which is valuable on X.

Proof. Suppose f is ∂-semismooth at x and let ((t_{n}, u_{n})) be a sequence with limit (0_{+}, u). Given a sequence (ε_{n}) with limit 0_{+} let us choose s_{n} > 0 such that | s_{n}^{−1}(f(x + t_{n}u_{n} + s_{n}u)−f(x+t_{n}u_{n}))−f′(x+t_{n}u_{n}, u) |≤ ε_{n}. Using the fact that ∂ is valuable as in the proofs of Propositions 4.5 and 4.6 we get sequences (w_{n}), (x_{n}), (y_{n}), (z_{n}) such that (w_{n}) →_{u} x, (x_{n}) →_{u} x, y_{n} ∈ ∂f(x_{n}), z_{n} ∈ ∂f(w_{n}) for each n and

lim inf_{n}⟨y_{n}, u⟩ ≤ lim inf_{n} s_{n}^{−1}(f(x + t_{n}u_{n} + s_{n}u) − f(x + t_{n}u_{n}))

≤ lim sup_{n} s_{n}^{−1}(f(x + t_{n}u_{n} + s_{n}u) − f(x + t_{n}u_{n})) ≤ lim sup_{n}⟨z_{n}, u⟩.

As f is ∂-semismooth at x we conclude that each term of this string of inequalities converges to f′(x, u), so that (f′(x + t_{n}u_{n})) converges to f′(x, u) too. Conversely, if (t, v) ⟼ f′(x+tv, u) is continuous at (0_{+}, u) for each u ∈ X_{0} we get that f is ∂^{↑}-semismooth at x, hence is also ∂-semismooth at x, by our containment assumption.

Theorem 4.12. Suppose ∂ is valuable on X and contained in ∂^{↑}. Then any lower T^{1}-function f on W is ∂-semismooth.

Proof. One sees easily that the proof of [20] Thm 2 can be adapted to show that f is ∂^{↑}-subsmooth. Using the remark following the preceding statement we get that f is also ∂-semismooth.

Acknowledgment. We are grateful to the referees for their precious criticisms.

References

[1] R. Abraham, J. Robbin: Transversal mappings and flows, W.A. Benjamin, New York, Amsterdam, 1967.

[2] J.-P. Aubin, I. Ekeland: Applied nonlinear analysis, John Wiley, New York, 1984.

[3] D. Aussel, J.-N. Corvellec and M. Lassonde: Mean Value Theorem and subdifferential criteria for lower semicontinuous functions, preprint, Univ. Clermont, April 1994.

[4] V.T. Averbuh, O.G. Smolyanov: Different definitions of derivatives in linear topological spaces, Uspehi Mat. Nauk 23(4) (1968) (162) 67–116.

[5] J. Birge, L. Qi: Semiregularity and generalized subdifferentials with applications to optimization, Math. Oper. Research 18 (4) (1993) 982–1005.

J.-P. Penot / Favorable classes of mappings and multimappings 115

[6] J. Borwein, S.P. Fitzpatrick and J. R. Giles: The differentiability of real functions on normed linear spaces using generalized subdifferentials, J. Math. Anal. Appl. 128 (1987) 512–538.

[7] N. Bourbaki: Eléments de mathématiques, espaces vectoriels topologiques, chap. III-V, Hermann, Paris, 1955.

[8] J.V. Burke, L. Qi: Weak directional closedness and generalized subdifferentials, J. Math. Anal. Appl. 159 (2) (1991) 485–499.

[9] F.H. Clarke: Optimization and nonsmooth analysis, Wiley Interscience, New York, 1983.

[10] F.H. Clarke, R.J. Stern and P.R. Wolinski: Subgradient criteria for monotone and Lipschitz behavior, Canad. J. Math. 45 (1993) 1167–1183.

[11] R. Correa, A. Jofre: Tangentially continuous directional derivatives in nonsmooth analysis, J. Optim. Theory and Appl. 61 (1989) 1–21.

[12] J. Danes: On continuous convergence of sequences of linear mappings, Rev. Roum. Math. Pures et Appl. 18 (7) (1973) 1045–1049.

[13] J.-B. Hiriart, Ph. Plazanet: Moreau’s theorem revisited, Analyse nonlinéaire (Perpignan, 1986) Ann. Inst. H. Poincaré 6 (1989) suppl., 325–338.

[14] A.D. Ioffe: Proximal analysis and approximate subdifferentials, J. London Math. Soc. (2) 41 (1990) 175–192.

[15] J.L. Kelley: General topology, Van Nostrand, Princeton, 1955.

[16] A. Kruger: Properties of generalized subdifferentials, Siberian J. Math. 26 (1985) 822–832.

[17] Ph. Loewen: A Mean Value Theorem for Fréchet subgradients, preprint, Nonlinear Anal. Th. Meth. Appl. 23 (11) (1994) 1365–1382.

[18] Ph. Michel, J.-P. Penot: Calcul sous-différentiel pour des fonctions lipschitziennes et non lipschitziennes, C. R. Acad. Sc. Paris, 298 (1984) 269–272.

[19] Ph. Michel, J.-P. Penot: A generalized derivative for calm and stable functions, Diff. Int. Equations, 5(2) (1992) 433–454.

[20] R. Mifflin: Semismooth and semiconvex functions in constrained optimization, SIAM J. Control and Opt. 15(6) (1977) 959–972.

[21] R. Mifflin: An algorithm for constrained optimization with semismooth functions, Math. Oper. Research, 2 (2) (1977) 191–207.

[22] B. Sh. Mordukhovich: Approximation methods in problems of optimization and control, Nauka, Moscow, 1988 (Russian; English translation to appear in Wiley-Interscience).

[23] J.-P. Penot: Continuité et différentiabilité des opérateurs de Nemytskii, Publications mathématiques de Pau (1976) VIII 1–45.

[24] J.-P. Penot: On the Mean Value Theorem, Optimization 19 (1988) 147–156.

[25] J.-P. Penot: A Mean Value Theorem with small subdifferentials, preprint, Univ. of Pau, Jan. 1994, revised July 1994.

[26] J.-P. Penot, M.L. Bougeard: Approximation and decomposition properties of some classes of locally d.c. functions, Math. Prog. 41 (1988) 195- 227.

[27] R.T. Rockafellar: Local boundedness of nonlinear monotone operators, Michigan Math. J. 16 (1970) 397- 407.

[28] R.T. Rockafellar: Favorable classes of Lipschitz continuous functions in subgradient optimization, in “Progress in nondifferentiable optimization”, E. Nurminski, ed. IIASA Collaborative Proc. Series, Intern. Institute of Applied Analysis, Laxenburg, Austria, 1982, 125–144.

[29] R.T. Rockafellar: First- and second-order epi-differentiability in nonlinear programming, Trans. Amer. Math. Soc. 307 (1) (1988) 75–108.

[30] I.V. Skripnik: The application of Morse’s methods to nonlinear elliptic equations, (Russian) Dokl. Akad. Nauk SSSR 202 (1972) 769–771, MR 45# 1206.

[31] I.V. Skripnik: The differentiability of integral functionals, (Ukrainian) Dopividi Akad. Nauk Ukrain RSR ser. A (1972) 1086–1089. MR 2749.

[32] J.E. Spingarn: Submonotone subdifferentials of Lipschitz functions, Trans. Amer. Math. Soc. 264 (1) (1981) 77–89.

[33] J.E. Spingarn: Submonotone mappings and the proximal point algorithm, Num. Funct. Anal. Opt. 4(2) (1981–1982) 123–150.

[34] L. Thibault, D. Zagrodny: Integration of subdifferentials of lower semicontinuous functions on Banach spaces, to appear J. Math. Anal. Appl.

[35] J. Treiman: Clarke’s generalized gradients and ε-subgradients in Banach spaces, Trans. Amer. Math. Soc. 294 (1986) 65–78.

[36] D. Zagrodny: Approximate Mean Value Theorem for upper subderivatives, Nonlinear Anal., Th., Methods, Appl. 12 (1988) 1413–1428.

[37] D. Zagrodny: A note on the equivalence between the Mean Value Theorem for the Dini derivative and the Clarke-Rockafellar derivative, Optimization 21 (1990) 179–183.