Some Open Problems in Dynamical Systems

Michael Herman^{1}

”Ce que nous savons est peu de choses ; ce que nous ignorons est immense”. P. S. Laplace

1 Siegel singular disks.

Let P_{α}(z) = e^{2iπα}z + z^{2}, α ∈ ℝ − ℚ, α is a Bruno number and we denote by S_{α} the maximal connected open set containing 0 on which P_{α} is linearizable (see [Y_{2}]) (the Siegel singular disk of P_{α} ; we use the word singular to avoid any confusion with Siegel domains).

Questions.

1. Does there exist α_{1} a Bruno number such that ∂S_{α_{1}} (the boundary of S_{α_{1}}) is a C^{∞} submanifold of ℂ (a circle) ? (see R. Pérez-Marco) [P]).

2. Does there exist α_{2} a Bruno number such that ∂S_{α_{2}} (the boundary of S_{α_{2}}) is not a Jordan curve ? (see A. Douady) [D]).

I conjecture that the answers are positive, but for question 2 one might need to consider general polynomials.

2 Invariant tori.

We suppose α ∈ 𝕋^{n} satisfies a diophantine condition (and write α ∈ DC).

∃ γ > 0 , ∃β ≥ 0 , ∀ k ∈ ℤ^{n} − \{0\} , |||⟨k, α⟩||| ≥ \frac{γ}{∥k∥^{n+β}} ,

where ⟨k, α⟩ = ∑_{j} k_{j}α_{j}, ||| ||| denotes the distance to nearest integer and ∥k∥ =

∑_{j} |k_{j}|.

1C.N.R.S.

Documenta Mathematica · Extra Volume ICM 1998 · II · 797–808

Let C_{0}^{∞}(𝕋^{n}) the space of C^{∞} functions on 𝕋^{n} of Haar integral 0. To each φ ∈ C_{0}^{∞}(𝕋^{n}) we associate the C^{∞} exact symplectic diffeomorphisms on T^{∗}𝕋^{n} ≅ 𝕋^{n} × ℝ^{n} with coordinates (θ, r) ∈ 𝕋^{n} × ℝ^{n},

F_{φ}(θ, r) = (θ + r, r + \frac{∂φ}{∂θ} (θ + r)) , \frac{∂}{∂θ} = (\frac{∂}{∂θ_{1}} , …, \frac{∂}{∂θ_{n}} )

(if φ ∈ C^{k}, F is C^{k−1}).

For α ∈ DC let U_{α} = {φ ∈ C_{0}^{∞}(𝕋^{n}), F_{φ} leaves invariant a torus T_{α,φ} graph of ψ ∈ C^{∞}(𝕋^{n}, ℝ^{n}) on which F_{φ} is C^{∞}-conjugated to R_{α} : θ → θ + α}. Using KAM, 0 ∈ U_{α} and U_{α} is C^{∞} open (cf. [H_{8}]). The open set is not all C_{0}^{∞}(𝕋^{n}) since (cf. [H_{3}] and [H_{4}]) if φ ∈ U_{α} the symmetric matrix I + 1/2D^{2}φ(θ) must be strictly positive definite for all θ ∈ 𝕋^{n} (I denotes the unit n × n matrix) : if one remplaces φ by λφ, supposing that φ ≢ 0 (is not identically zero), the above condition will be violated when λ is large.

2.1 - Question.

Is the boundary ∂U_{α} of U_{α} in C^{∞}(𝕋^{n}) a locally flat C^{k}-submanifold (for some k ≥ 0) ? Nothing is known (e.g. is every point x ∈ ∂U_{α} accessible from U_{α} and the exterior of U_{α} ?).

2.2 - Let ⋃_{α∈DC} U_{α} = U ; what is the minimal number k ∈ ℝ_{+}^{∗} for which U is open near 0, in the C^{k}-topology ? (One shows, [H_{3}], that k ≥ n+2). For many reasons it is not unnatural to work in Sobolev W^{k,p} topology, k ∈ ℝ_{+}^{∗}, p ≥ 1 : the topology defined by asking that φ has [k] derivatives in the distributional sense in L^{p}(𝕋^{n}, dθ) and t ∈ 𝕋^{n} ↦ D^{[k]}φ ∘ R_{t} ∈ L^{p} satisfies a Hölder condition of exponent k − [k]. One easily shows, [H_{3}], that U is not open (near 0) in the W^{2n+2−ε,1} topology (∀ ε > 0), and the author showed, [H_{2}], that, when n = 1 and α is of constant type, (∃ γ > 0, ∀ p/q, |α − p/q| ≥ \frac{γ}{q^{2}} ) , U_{α} is

open near 0, in the W^{4,p}, p > 1, topology.

The author has been unable to decide if this is also true when p = 1 !

One proves, using [S], that U_{α} is open, near 0, in the C^{k}-topology when k ≥ 2(n + β) + 2 + ε, ∀ ε > 0, and is not open in the C^{k}-topology, near 0, when k ≤ 2(n + β) + 2 − ε, ∀ ε > 0 (when n = 1, this is proved in [H_{1}] and if n ≥ 2, it is really not difficult to adapt the examples, with n = 1, of [H_{1}] to this case).

Remark. There is a difference between n = 1 and n ≥ 2. When n = 1, the global conjugacies theorems of C^{∞}-diffeomorphisms of the circle to diophantine rotations (cf. [Y_{1}]) imply that ψ has to loose its smoothness when φ ∈ ∂U_{α}, see [H_{2}, p.46-49]. If n ≥ 2, there are no global conjugacy theorems to diophantine translations of 𝕋^{n} (cf. [H_{5}]) which makes question 2.1 harder since one has to solve 2 problems.

3 Questions on measure preserving diffeomorphims of 2 manifolds.

The following question would be to give a counterexample to a conjecture (or question) of Birkhoff [B_{1}].

3.1 - Question.

Does there exist a C^{ω}(= ℝ-analytic)-diffeomorphism, homotopic to the identity, Lebesgue measure-preserving, of 𝕋^{1}×[−1, 1] or S^{2}, with a finite number of periodic orbits and a dense orbit ?

The answer is positive for C^{∞}-diffeomorphisms (Anosov, Katok [AK]).

The problem would be to create methods to prove similar results as in [AK] for C^{ω}-diffeomorphisms ; as the group of C^{ω}-diffeomorphisms of a compact C^{ω}- manifold with the natural C^{ω}-topology is not a Baire space, one has to work in the complex ; which requires new methods (cf. [Y][PY]).

3.2 - Question. Let f be a C^{∞}-diffeomorphism preserving the Lebesgue measure of 𝕋^{1} × [−1, 1], homotopic to the identity, that has a finite number of periodic points (in fact, by a result of Franks [ICM 94] also obtained independly by P. Le Calvez, no periodic points) and is such that the rotation number ρ(f_{|𝕋^{1}×{−1}}) = α satisfies a diophantine condition. Is f C^{∞}-conjugated to R_{α}(θ, r) = (θ + α, r) ?

I would expect a counter-example, but there is some evidence in the opposite direction.

We will show elsewhere this is the case if f is C^{∞}-close to R_{α} and f is always C^{∞}-conjugated to R_{α} near 𝕋^{1} × {±1}.

When α is a Liouville number a negative answer to the question can happen in the exotic examples constructed by M. Handel [HA] (see also [H_{9}]).

We can ask the same question replacing the condition that f is Lebesgue measure preserving by the intersection property (this could make a counter-example easier).

We also ask the similar question for 𝔻^{2} = {z ∈ ℂ, |z| ≤ 1}. When f is Lebesgue measure preserving and f has a finite number of periodic points is f C^{∞}-conjugated to r_{α} ? This is always the case near 0 and ∂𝔻^{2} and globally when f is C^{∞} near r_{α}, r_{α}z = e^{2iπα}z.

3.3 - Let f : z ∈ ℝ^{2n} → Az + O(z^{2}) ∈ ℝ^{2n} be a germ of symplectic diffeomorphisms such that A ∈ Sp(2n, ℝ) is conjugated in Sp(2n, ℝ) to r_{α_{1}} × ⋯ × r_{α_{n}}, α = (α_{1}, …, α_{n}) ∈ DC.

3.4 - Conjecture. If f is real analytic, then f leaves invariant, in any small neighbourhood of 0, a set of positive Lebesgue measure of Lagrangian tori.

3.5 - Remarks.

By Moser’s theorem and a theorem of Rüssmann [R], the conjecture is true when n = 1. We insist that we ask for a set of positive measure of invariant tori in the conjecture. If we ask the conjecture when f is only C^{∞}, the conjecture, as we will show elsewhere is correct if n = 1, unknown if n = 2, and false if n ≥ 3. There are many cases when the conjecture is true.

After Birkhoff normal form in polar symplectic coordinates x_{j} = \sqrt{r_{j}} cos 2πθ_{j} y_{j} = −\sqrt{r_{j}} sin 2πθ_{j} , α ∈ DC, f(θ, r) = (θ+α+ℓ(r), r)+(ℓ(0) = 0), ℓ ∈ (ℝ[[r_{1}, …, r_{n}]])^{n} (i.e. formal power series).

The conjecture is true when the components of ℓ are independant over ℝ as formal power series (this follows from a theorem of Rüssmann (cf. [BHS]) and with this hypothesis, the conjecture is true even when f is C^{∞}) or ℓ = 0 [R].

4 Entropy and exponents.

4.1 - Let M^{n} be a compact connected C^{∞}-manifold, f : M^{n} → M^{n} a C^{∞}- diffeomorphism that leaves invariant a probability measure μ on M^{n}. We denote by Df(x) the tangent of f at x ∈ M. Let

λ_{+}(f, μ) = lim_{k→+∞} \frac{1}{k} ∫_{M} log ∥Df^{k}(x)∥_{x} dμ(x) = inf_{k≥1} \frac{1}{k} ∫_{M} log ∥Df^{k}(x)∥_{x} dμ(x)

where Df^{k} is tangent map for f^{k}, ∥ ∥_{x} is the operator norm induced by a Riemannian metric on T M of Df^{k}(x) : T_{x}M → T_{f^{k}(x)}M. When μ is a C^{∞} probability measure (in every chart μ = φdx_{1} ⊗ ⋯ ⊗ dx_{n} , φ ∈ C^{∞} , φ > 0) by Pesin’s formula the entropy of f for the measure μ satisfies nλ_{+}(f, μ) ≥ h_{μ}(f) ≥ λ_{+}(f, μ) so that λ_{+}(f, μ) = 0 ⇒ h_{μ}(f) = 0 (also is true if f is only C^{1} by Margulis-Ruelle). G_{μ}^{∞}(𝔻^{2}) = {f ∈ Diff^{∞}(𝔻^{2}), support(f) ⊂ 𝔻^{2}, f_{∗}μ = μ = Lebesgue measure normalised}. 4.2 - Conjecture. For every C^{∞}-neighbourhood W of the identity in G_{μ}^{∞} there exists g ∈ W such that h_{μ}(g) > 0.

See [K_{1}] for an example of f ∈ G_{μ}^{∞}(𝔻^{2}) with h_{μ}(f) > 0.

Let us remark that since f → λ_{+}(f, μ) ≥ 0 is upper semicontinuous, the set G = {f, λ_{+}(f, μ) = 0} is a G_{δ} in G_{μ}^{∞}(𝔻^{2}) for the C^{∞}-topology. 4.3 - One of the 2 exclusive possibilities is true by Baire category :

a) The set G is a dense G_{δ}.

b) There exists U ⊂ G_{μ}^{∞}, a C^{∞} open set, U ≠ ∅, for every g ∈ U one has h_{μ}(g) ≥ δ, for some δ > 0. Question. Is a) or b) true ? Mañé claims [ICM 82] that, for G_{μ}^{∞}(M^{2}) where (M^{2}, μ) is a compact 2-manifold without boundary and μ is C^{∞} probability measure, then a) is true for G a dense G_{δ} in the C^{1} topology on G_{μ}^{∞}(M^{2}) − {f, f is an Anosov diffeomorphism}. For an outline of a possible proof, see [M].

A similar question can be asked for smooth measure preserving diffeomorphisms on (M^{n}, μ), any compact manifold of dimension n ≥ 2, or for symplectic diffeomorphisms on symplectic manifolds (M^{2n}, w), n ≥ 1, without boundaries. 4.4 - There are cases when b) has a positive answer : when f is partially hyperbolic (a C^{1} open property) : the tangent map of f, Df, leaves invariant T M = E^{s} ⊕ E^{c} ⊕ E^{u}, 3 continuous bundles and there exists C_{j} > 0, 1 ≤ j ≤ 4, such that 0, for every k ∈ ℕ, ∥Df^{k}v∥_{f^{k}(x)} ≥ C_{1} λ_{1}^{k}∥v∥_{x}, λ_{1} > 1, v ∈ E_{x}^{u} , ∥Df^{k}v∥_{f^{k}(x)} ≤ C_{2} λ_{2}^{k}∥v∥_{x}, λ_{2} < 1, v ∈ E_{x}^{s} , C_{4} λ_{4}^{k}∥v∥_{x} ≤ ∥Df^{k}v∥_{f^{k}(x)} ≤ C_{3} λ_{3}^{k}∥v∥_{x}, v ∈ E_{x}^{c} , 0 < λ_{2} < λ_{4} ≤ 1 ≤ λ_{3} < λ_{1} . 4.5 - More generally and in particular in the case volume preserving, there is the notion of dominated splitting : T M = E^{s} ⊕ E^{u}, continuous invariant bundles, invariant by Df, such that : ∥Df_{|E_{x}^{s}}^{k}∥_{x} ∥Df_{|E_{f^{k}(x)}^{u}}^{−k}∥_{f^{k}(x)}∥ ≤ C λ^{k}, 0 < λ < 1, C >

0 . 4.6 - The notion of stably exponents = SE_{∗}^{∞}. Let f ∈ G_{∗}^{∞}(M) where ∗ = μ for smooth measure preserving diffeomorphisms, or ∗ = w for symplectic form. 4.7 - Definition. Let f ∈ G_{∗}^{∞}(M). We say that f ∈ SE_{∗}^{∞} if there exists δ > 0 and there exists V a C^{1}-neighbourhood of f such that every g ∈ V, and for every periodic orbit of g, 0_{p/q} = {g^{j}(x_{0}), 0 ≤ j ≤ q − 1, g^{q}(x_{0}) = x_{0}} ,

λ_{+}(g, μ_{p/q}) ≥ δ > 0 where μ_{p/q} = \frac{1}{q} ∑_{0}^{q−1} δ_{g^{j}(x_{0})}, x_{0} ∈ 0_{p/q}

and δ_{y} denotes the Dirac mass at y ∈ M^{n}.

We take the C^{1}-topology only in order to use closing lemma arguments. Using the ergodic closing lemma of Mañé ([A_{2}]), f ∈ SE_{∗}^{∞} implies for every

υ probability measure invariant by f satisfies λ_{+}(f, υ) ≥ δ. Remark. When f is a general diffeomorphism we ask in the definition of stable exponents that inf(λ_{+}(g, μ_{p/q}) , λ_{+}(g^{−1}, μ_{p/q})) ≥ δ. 4.8 - Remarks. If f ∉ SE_{∗}^{∞}, there exists a C^{1}-perturbation g of f, g ∈ G_{∗}^{∞}(M), such that g has a totally elliptic periodic orbit : there exists x_{0}, g^{q}(x_{0}) = x_{0} and all the eigenvalues of Dg^{q}(x_{0}) are on the unit circle. Then by a C^{1}-perturbation we can suppose that g is periodic on a small neighbourhood of x_{0}. After a C^{∞}-perturbation one falls in the KAM world (symplectic, or for volume preserving the existence of codimension 1 periodic invariant diophantine tori : Cheng Sun [C], Yoccoz [Y], Xia [X], and by the similar proof as in Yoccoz [Y], see also [BHS]). The above results do not violate that C^{∞}-generically in G_{μ}^{∞}(M^{n}), when n ≥ 3, all the periodic points are hyperbolic.

When f ∈ SE^{∞} is not measure preserving, using the definition in the remark of 4.7, then by a C^{1}-perturbation, one can create a sink.

Of course, by definition SE_{∗} is C^{1}-open in G_{∗}^{∞}(M). 4.9 - Questions.

a) Is the set {f ∈ G_{∗}^{∞}(M), f has a dominated splitting on M} C^{1}-dense in SE_{∗}^{∞} ? (it is C^{1}-open).

For many reasons it seems reasonable to conjecture that the answer is positive (see ([DPU]) on 3-manifolds for general diffeomorphisms ”stably transitive”).

It follows from Newhouse [N] that if f ∈ G_{w}^{∞}(M^{2n}) is such that all periodic points are C^{1}-stably hyperbolic, then f is an Anosov diffeomorphism.

b) Does there exist a connected and simply connected manifold M^{n} with a partially hyperbolic diffeomorphism ? The answer is not even known if the dimension of M^{n} is n = 3. If E^{c} ⊕ E^{u} or E^{c} ⊕ E^{s} defines a C^{0}- foliation, it would follow that it is negative by the C^{0}- Novikov theorem, but it is an open problem to prove, when n = 3, that E^{c} ⊕ E^{u} defines a C^{0}-foliation !

c) On G_{∗}^{∞}(M) − SE_{∗}^{∞}, which of the two possibilities of 4.3 is true ? 4.10 - Let M^{n} be compact connected manifold and f a strictly ergodic C^{∞}- diffeomorphism (minimal (i.e. every orbit is dense) and uniquely ergodic). Conjecture. The topological entropy of f equals 0. The conjecture is true when n = 2, Katok [K] ([ICM 82]) ; A.B. Katok remarked the minimal diffeomorphisms with positive topological entropy constructed in [H_{6}] are not uniquely ergodic.

5 Existence of periodic orbits.

There is some evidence, [G][H_{7}], that the closing lemma of Pugh will not generalize to the C^{∞}-topology. So we will ask some precise questions, hoping they might lead to some better understanding of the problem. 5.1 - Let Diff_{o}^{∞}(𝕋^{n}), the group of diffeomorphisms of 𝕋^{n}, C^{∞}-isotopic to the identity endowed, with the C^{∞}-topology. Let G = \{f ∈ Diff_{0}^{∞}(𝕋^{n}), f has no periodic points\} (that is a G_{δ}) and \overline{G} the C^{∞}-closure of G Diff_{0}^{∞}(𝕋^{n}). 5.2 - Question. Does the closed set \overline{G} have no interior (n ≥ 2) in Diff_{0}^{∞}(𝕋^{n}) ?

One can ask the above for volume-preserving diffeomorphisms when n ≥ 3. In counterpart, it follows from Conley-Zehnder [HZ] (see also Zehnder [ICM 86]), for Diff_{0,w}^{∞}(𝕋^{2n}) when w is a constant symplectic form and 0, w denotes that the isotopy is symplectic, that \overline{G_{w}} has no interior in Diff_{0,w}^{∞}(𝕋^{2n}), where G_{w} = G ∩ Diff_{0,w}^{∞}(𝕋^{2n}). 5.3 - Along the lines of Poincaré [P_{1}] (see [H_{7}]), we ask :

Let H_{0}(r) be a function defined on T^{∗}𝕋^{n} = 𝕋^{n} × ℝ^{n}, n ≥ 2. We suppose H_{0} is C^{ω}-convex with superlinear growth when ∥r∥ → +∞, H_{0}(r)/∥r∥ → +∞), and \frac{∂H_{0}}{∂r} (0) = 0 . Question. Can one find a C^{ω}-family H_{ε}(θ, r) = H_{0}(r)+εH_{0}(θ, r, ε), every thing being ℝ-analytic, such that for some ε_{0} small on H_{ε_{0}}^{−1}(1), the periodic orbits of the Hamiltonian flow of H_{ε_{0}} are not dense ? The only real restrictions we ask for an example is that for every θ, r → H_{ε_{o}}(θ, r) is strictly convex and H_{ε_{o}}^{−1}(] − ∞, 1[) countains the zero section. One easily constructs C^{∞}-counter examples, but this is not the question asked !

5.4 - Recently V. Ginzburg, [G_{1}], and the author constructed M^{2n−1} ⊂ ℝ^{2n}, n ≥ 4, compact connected, C^{∞}-hypersurface such that characteristic flow on M^{2n−1} has no periodic orbits (for C^{∞}-compact connected examples when n = 3, see [G_{2}]).

Let us write M^{2n−1} = H^{−1}(1), where H is a C^{∞}-function ans 1 is a regular value. A theorem of Hofer and Zehnder [HZ] says that for ε_{n} → 1, the Hamiltonian flow of H on H^{−1}(ε_{n}) has at least one periodic orbit P_{ε_{n}} and one wonders how bad P_{ε_{n}} behaves, when ε_{n} → 1, in the space of compact sets on H_{ε_{n}}^{−1}. 5.5 - Questions.

1. When n ≥ 2 can one find M^{2n−1} ⊂ ℝ^{2n}, a C^{∞}-compact connected hypersurface, such that on M^{2n−1} the characteristic flow is minimal (every orbit is dense) ?

2. Can the characteristic flow on M^{2n−1} be an Anosov flow ?

5.6 - Let M^{n} be a compact connected oriented manifold of dimension n ≥ 3 and with Euler-Poincaré characteristic χ(M^{n}) = 0. We consider C^{∞}, Ω-divergence free vectors-fields where Ω is a C^{∞}-volume form on M^{n}. We suppose X(x) ≠ 0, ∀ x ∈ M^{n}. We denote by f_{t}^{X} the flow of the vector-field X. 5.7 - Questions.

a) Does there exist vector fields as above (in every homotopy class of non-zero vector fields) such that the flow of f_{t}^{X} has no periodic orbit ?

b) Does there exist vector-fields as above such that the flow f_{t}^{X} is Ω-ergodic ?

(Anosov [A] constructed an ergodic flows f_{t}^{X} on every (M^{n}, Ω) orientable manifold (n ≥ 3) but the vector field in Anosov’s construction has zeros (which of course will be the case when χ(M^{n}) ≠ 0). For n = 2, Anosov’s result is also true except, by Poincaré-Bendixon’s theorem, on the 2-sphere (we suppose that M^{n} is orientable otherwise, one has to replace Ω by a smooth measure and Anosov’s theorem is still true except for the projective plane or the Klein bottle). I certainly believe the answers are positive when n is large. The case n = 3 is much more delicate (in the C^{1} case, for question a), see [K_{2}]). 5.8 - Let (M^{2n}, w) be a C^{∞}-compact symplectic manifold. Given a closed 1-form, v we define a symplectic vector-field X by i_{X}w = v. The following statements are equivalent :

• The vector-field X has no zeros ;

• v has no zeros ;

• M^{2n} fibers of S^{1} (Tischler [T]). Claim : There exists a symplectic manifold M^{2n} with χ(M^{2n}) = 0, but M^{2n} does not fiber over S^{1}.

To see this, we consider M^{4} = (ℙ_{1}(ℂ) × M_{g})♯_{k}\overline{ℙ}_{2}(ℂ) where M_{g} is a compact surface of genus g ≥ 2 and we blow up k points of ℙ_{1}(ℂ) × M_{g}. We have χ(M^{4}) = 2(2 − 2g) + k = 0 when k = 2(−2 + 2g). The manifold M^{4} does not fiber over S^{1} since the fiber, if it existed, would have a non finitely generated fundamental group.

Conjecture.

Every symplectic diffeomorphism homotopic to the identity of (M^{2n}, w), where χ(M^{2n}) = 0 and M^{2n} does not fiber over S^{1}, has a fixed point (in fact a number of fixed points ≥ minimal number of zeros of any symplectic vector-field). When f is C^{1}-near the identity, the conjecture is easy and true.

6 Instabilities of Hamiltonian flows on T^{∗}𝕋^{n}, n ≥ 3, and the problem of topological stability.

6.1 - A part from the results of Arnold [A_{3}], R. Douady and P. Le Calvez [DL_{2}][D_{2}], and the beautiful work of John Mather for twist maps [M_{1}] (in [LC_{2}] P. Le Calvez proves some of Mather’s results by topological methods following in part Birkhoff) and partial generalizations to higher dimensions by Mather [MA_{2}] [MA_{3}] and a survey [MA_{4}], the subject lacks any non-trivial result. 6.2 - Questions.

1. Can one find an example of a C^{∞}-Hamiltonian H in a small C^{k}- neighbourhood k ≥ 2 of H_{0}(r) = 1/2∥r∥^{2} such that, on H^{−1}(1), the Hamiltonian flow of H has one dense orbit ?

See the example of Donnay Liverani [DL] of a C^{∞}-Riemannian metric on 𝕋^{2} with an ergodic geodesic flow.

Many people believe that examples as above do exist and are C^{∞}-generic (cf. P. and T. Ehrenfest [E], G.D Birkhoff [B_{2}], V.I. Arnold [ICM 66]) and these questions have been called by the author the quasi-ergodic hypothesis, following [E] (but in this reference, no clear distinction is made between ”every orbit is dense” and ”one orbit is dense”, and we choose ”one orbit is dense”). The negation of the quasi-ergodic hypothesis is topological stability.

In the following questions I suppose that H is in a C^{2}-neighbourhood of H_{0}(r), that is convex, of super linear growth, and such that \frac{∂H_{0}}{∂r} (0) = 0.

(What we really want is that H^{−1}(1) be connected and that the set H^{−1}(] − ∞, 1[) contains a Lagrangian manifold, that is the graph over 𝕋^{n}).

2. For the C^{∞}-generic H and the generic ergodic minimal measure μ of Mather [MA_{2}], is the flow f_{t|supp(μ)}^{H} on H^{−1}(1) a hyperbolic flow ? (For the twist map case, see Le Calvez [LC_{1}]).

3. For the C^{∞}-generic H on H^{−1}(1) are the probability measures defined by periodic orbits, dense (in the weak topology on probability measures) in the ergodic minimal measures ? (For monotone twist maps the result is known by Aubry Mather’s theory [MA_{4}]. The higher dimensional case is much more delicate (cf. [A_{1}], see also [H_{4}, §3.3, p. 53]).

4. Can one find C^{∞}-generically a closed connected set F, invariant by the Hamiltonian flow f_{t}^{H} of H, F ⊂ H^{−1}(1), such that F contains all the supports of the ergodic minimal measures, in F the orbits asymptotic to the locally minimal orbits (cf. [LC_{2}]) are dense in F, and on F, f_{t|F}^{H} has a dense orbit ? The above questions are precise, compatible with what J. Mather proved for the dynamics of monotone twist maps in Birkhoff zones of instabilities (the existence of a dense orbit is an open question). The author conjectures the answer of the question 4 is positive and the methods of Mather, specially variational methods, make that the question is not desperate and furtheremore no hypothesis of small C^{∞}-perturbations of completely integrable systems is needed. In counterpart, as we asked in the question that the orbits asymptotic to the locally minimal orbits (minimizing with constraints) are dense in F (condition that J. Mather uses in most proofs (cf. [MA_{4}],p. 162-169)), C^{∞}-generically the locally minimal orbit (that is a closed set) are nowhere dense in H^{−1}(1) (so the question is not really related to the quasi-ergodic hypothesis), but would show how to move around frequencies (rotation vectors of the ergodic minimizing measures). For some examples, when n = 2 and large perturbations, see V. Bangert [BA,9.12].

7 The oldest open question in dynamical systems.

7.1 - I. Newton, [N_{1}], certainly believed that the n-body problem, n ≥ 3, (n particles moving under universal gravitation) is topological instable and, to paraphrase Laplace, makes the hypothesis that God solves the problem and controls the instabilities (hypothesis criticized by Leibniz and all the enlighted XVIII^{th} century). The question we will ask for the n-body problem is a special case when the energy surface is not compact and the volume form on the energy surface is not finite (hypotheses at infinity are of course necessary), but we will not formulate any general question.

7.2 - For the n-body problem in space, we will suppose n ≥ 3.

• The center of mass is fixed at 0.

• On the energy surface we C^{∞}-reparametrize the flow by a C^{∞} function φ_{e} (after reduction of the center of mass) such that the flow is complete : we replace H by φ_{e}(H −e) = H_{e} so that the new flow takes an infinite time to go to collisions (φ_{e} > 0 is a C^{ω} function outside collisions).

7.3 - Following G.D. Birkhoff [B_{3}] (who only considers the case n = 3 and the angular momentum ≠ 0) (see also A.N. Kolmogorov [ICM 54]), we ask :

Question. Is for every e the non wandering set of the Hamiltonian flow of H_{e} on H_{e}^{−1}(0) nowhere dense in H_{e}^{−1}(0) ? In particular, this would imply that the bounded orbits are nowhere dense and no topological stability occurs. It follows from the identity of Jacobi-Lagrange that, when e ≥ 0, every point such that its orbit is defined for all times, is wandering. The only thing known is that, even when e < 0, wandering sets do exist (Birkhoff and Chazy, see V.M. Alexeyev [ICM 70]).

The fact that the bounded orbits have positive Lebesgue-measure when the masses belong to a non empty open set, is a remarkable result announced by V.I. Arnold [A_{4}] (Arnold only gives a proof for planar 3-body problem and if the author is not mistaken, Arnold’s claim is correct). In some respect Arnold’s claim proves that Lagrange and Laplace, against Newton, are correct in the sense of measure theory and that in the sense of topolgy, the above question, in some respect, could show Newton is correct. For some soft 806 Michael Herman

(almost explicit) examples of dissipative nature of Hamiltonian on non-compact energy surfaces with infinite volume we refer the reader to [H_{8}]. What seems not an unreasonable question to ask (and possibly prove in a finite time with a lot of technical details) is that :

In one of the masses m_{0} = 1 and the other masses m_{j} = εM_{j}, 1 ≤ j ≤ n−1, M_{j} > 0, ε > 0, then in any neighbourhodd of fixed different circulat orbits arounds m_{0} moving in the same direction in a plane, when ε is small, there are wandering domains. But in many respects this is not the question asked that is more global.

In Herman [ICM 78] (that refers to the article of M.R. Herman in the Proceedings of the ICM held in 1978) the reader will find other open problems some of which are still insolved. For other problems on Siegel’s linearization theorem (most are still open) we refer to [H_{10}].

References

[A] D.V. Anosov, Existence of smooth ergodic flows on smooth manifolds, Math. U.R.S.S. Izvestija 8 (1974), p. 525-552.

[AK] D.V. Anosov, A.B. Katok, New examples in smooth ergodic theory, Trans. Moscow Math. Soc. 23 (1970), p. 1-35.

[A_{1}] Marie-Claude Arnaud, Type des points fixes des difféomorphismes symplectiques de 𝕋^{n} × ℝ^{n}, Mémoire (nouvelle série) n^{o} 48, Suppl. Bull. Soc. Math. Fr. t.120, Fasc. 1, (1992).

[A_{2}] Marie-Claude Arnaud, Le ”closing Lemma” en topologie C^{1}, to appear in

Mémoires de Soc. Math. Fr. [A_{3}] V.I. Arnold, Instability of dynamical systems with several degrees of freedom, Sov.

Math. Dok 5 (1964), p. 581-585. [A_{4}] V.I. Arnold, Small denominators and problems of stability of motion in classical

and celestial mechanics, Russ. Math. Surveys 18,6 (1963), p. 85-191.

[BA] V. Bangert, Mather sets for twist maps and geodesics on tori, Dynamics reported, Vol.1, Wiley & Sons (1988), p. 1-56.

[BHS] H.S. Broer, G.B. Huitema, M.B. Sevryuk, Quasi-periodic motions in families of dynamical systems, Springer Lect. Notes in Math. n^{o} 1645 (1996).

[B_{1}] G.D. Birkhoff, Collected Math. Papers, Vol.2, Dover, p. 712. [B_{2}] G.D. Birkhoff, Collected Math. Papers, Vol.2, Dover, p. 462-465. [B_{3}] G.D. Birkhoff, Dynamical Systems, Amer. Math. Soc. Colloquium Pub. Vol. IX

(1966), p. 290.

[C] Ch.-Q. Cheng, Y.S. Sun, Existence of invariant tori in three-dimensional measure-preserving mappings, Celest. Mech. Dyn. Astr. 47 (1990), p. 275-292.

[DPU] L.J. Diaz, E.R. Pujals, R, Ures, Partial hyperbolicity and robust transitivity, Preprint (1997).

[DL_{1}] V.J. Donnay, C. Liverani, Potentials on the two-torus for which the Hamiltonian

is ergodic, Commun. Math. Phys. 135 (1991), p. 267-302. [D_{1}] Adrien Douady, Disques de Siegel et anneaux de Herman, Séminaire Bourbaki

n^{o} 677, Astérisque 152-153, Soc. Math. Fr. (1987), p. 151-172. [D_{2}] Raphaël Douady, Stabilité ou instabilité des points fixes elliptiques, Ann. Sci. Ec.

Norm. Sup., t.21 (1988), p. 1-46.

Some Open Problems in Dynamical Systems 807

[DL_{2}] Raphaël Douady, P. Le Calvez, Exemple de point fixe elliptique non topologique-

ment stable en dimension 4, C.R. Acad. Sci. Paris, t. 296 (1983), p. 895-898.

[E] P. and T. Ehrenfest, The conceptual foundations of the statistical approach in mechanics, Dover.

[G_{1}] V.L. Ginzburg, An embedding S^{2n−1} → ℝ^{2n}, 2n−1 ≥ 7, whose Hamiltonian flow

has no periodic trajectories, I.M.R.N. 2 (1995), p. 83-98. [G_{2}] V.L. Ginzburg, A smooth counterexample to the Hamiltonian Seifert conjecture

in ℝ^{6}, Preprint, Santa-Cruz (1997).

[G] C. Gutirrez , A counterexample to a C^{2}-closing lemma, Erg. Th. and Dyn. Systems 7 (1987), p. 509-530.

[HA] M. Handel , A pathological area preserving C^{∞}-diffeomorphisms of the plane, Proc. Amer. Math. Soc. 86 (1982), p. 163-169.

[H_{1}] M.R. Herman, Sur les courbes invariantes par les difféomorphismes de l’anneau,

Vol.1, Astérisque 103-104, Soc. Math. Fr. (1983). [H_{2}] M.R. Herman, Sur les courbes invariantes par les difféomorphismes de l’anneau,

Vol.2, Astérisque 144, Soc. Math. Fr. (1986). [H_{3}] M.R. Herman, Existence et non existence de tores invariants par des

difféomorphismes symplectiques, Sém. E.D.P. 1987-1988, Exposé XIV, Centre de Mathématiques de l’Ecole Polytechnique, p. 1-24.

[H_{4}] M.R. Herman, Inégalités a priori des tores lagrangiens par des difféomorphismes

symplectiques, Publ. Math. I.H.E.S. n^{o} 70 (1989), p. 47-100. [H_{5}] M.R. Herman, Une méthode pour minorer les exposants de Lyapunov et quelques

exemples montrant le caractère local d’un théorème d’Arnold et de Moser sur le tore de dimension 2, Commentarii Math. Helv. 58 (1983), p. 457-502.

[H_{6}] M.R. Herman, Construction d’un difféomorphisme minimal d’entropie

topologique non nulle, Erg. Th. and Dyn. Systems 1 (1981), p. 61-76. [H_{7}] M.R. Herman, Exemples de flots hamiltoniens dont aucune perturbation en

topologie C^{∞} n’a d’orbites périodiques sur un ouvert de surfaces d’énergie, C.R. Acad. Sci. Paris, t. 312 (1991), p. 989-994.

[H’_{7}] M.R. Herman, Différentiabilité optimale et contre-exemples à la fermeture en topologie C^{∞} des orbites récurrentes de flots hamiltoniens, C.R. Acad. Sci. Paris, t. 313 (1992) p. 153-182.

[H_{8}] M.R. Herman, Dynamics connected with indefinite normal torsion, twist mappings and their applications, Ed. R. Mc Gehee and K. Meyer, IMA Vol.44, Springer (1992), p. 153-182, see § 4.20-4.22.

[H_{9}] M.R. Herman, Construction of some curious diffeomorphisms of the Riemann

sphere, J. London Math. Soc. 34 (1986), p. 375-384. [H_{10}] M.R. Herman, Recent results and some open questions on Siegel’s linearization

theorem of germs of complex analytic diffeomorphism of ℂ^{n} near a fixed point, Proc. VIII Int. Conf. Math. Phys. (Marseille 1989), World Scientific (1987), p. 138-198.

[HZ] H. Hoffer, E. Zehnder, Symplectic invariants and Hamiltonian dynamics, Birkhaus̈er (1994).

[K] A.B. Katok, Lyapunov exponents, entropy and periodic orbits for diffeomorphisms, Publ. Math. I.H.E.S. 51 (1980), p. 137-173.

[K_{1}] A.B. Katok, Bernoulli diffeomorphisms on surfaces, Ann. Math. 110 (1979), p.

529-547. [K_{2}] G. Kuperberg, A volume-preserving counterexample to the Seifert conjecture,

Commentarii Math. Helv. 71 (1996), p. 70-97.

808 Michael Herman

[LC_{1}] P. Le Calvez, Les ensembles d’Aubry-Mather d’un difféomorphisme de l’anneau déviant la verticale sont en général hyperboliques, C.R. Acad. Sci. Paris, t. 306 (1998), p. 51-54.

[LC_{2}] P. Le Calvez, Propriétés dynamiques des régions d’instabilité, Ann. Sci. Ec. Norm.

Sup. 20 (1987), p. 443-464.

[M] R. Mañé, The Lyapunov exponents of generic area preserving diffeomorphisms, Int. Conf. Dyn. Syst. (Montevideo), Pitman Res. Notes Math. Ser. 362 (1996), p. 110-119.

[MA_{1}] J. Mather, Variational construction of orbits for twist diffeomorphisms, J. Amer.

Math. Soc. 4 (1991), p. 203-267. [MA_{2}] J. Mather, Action minimizing invariant measures for positive definite Lagrangian

systems, Math. Z. 207 (1991), p. 169-207. [MA_{3}] J. Mather, Variational construction of connecting orbits, Ann. Inst. Fourier 43

(1993), p. 1349-1386. [MA_{4}] G. Forni, J.N. Mather, Action minimizing orbits in Hamiltonian systems,

Springer Lect. Notes in Math. n^{o} 1589, p. 91-186.

[N] S.E. Newhouse, Quasi-elliptic periodic points in conservative dynamical systems, Amer. J. Math. 99 (1977), p. 1061-1087.

[N_{1}] I. Newton,

• Principa, General Scholium, End of Book III, Translation Mottes, Vol. II, Univ. California Press, p. 544.

• Optics, Dover, Query 31, p.402, see also Query 27, p. 369.

[P] R. Perez-Marco, Siegel disks with quasi-analytic boundary, Preprint Univ. Paris-Sud (1997).

[PY] R. Perez-Marco, J.-C. Yoccoz, Germes de feuilletages holomorphes, Astérisque 222, Soc. Math. Fr. (1994), p. 345-371.

[P_{1}] H. Poincaré Les méthodes nouvelles de la mécanique céleste, t.I, Gauthier-Villars

(1892) §32, p. 82 that refers to §13.

[R] H. Rüssmann, On the convergence of power series transformations of analytic mappings near a fixed point into a normal form, Preprint, I.H.E.S. (1977).

[S] D. Salamon, The Kolmogorov-Arnold-Moser theorem, Preprint ETH Zürich (1986).

[T] T. Tischler, On fibering certain foliated manifolds, Topology 9 (1970), p. 153-154.

[X] Zh. Xia, Existence of invariant tori in volume-preserving diffeomorphisms, Erg. Th. and Dyn. Systems 12 (1992), p. 621-631.

[Y] J.-C. Yoccoz, Travaux de Herman sur les tores invariants, Séminaire Bourbaki n^{o} 754, Astérisque 206, Soc. Math. Fr. (1992), p. 311-344.

[Y_{1}] J.-C. Yoccoz, Conjugaison différentiable des difféomorphismes du cercle dont le nombre de rotation vérifie une condition diophantienne, Ann. Sc. E.N.S. t.17, (1984), p. 333-359.

[Y_{2}] J.-C. Yoccoz, Petits diviseurs en dimension 1, Astérisque n^{o} 231, Soc. Math. Fr. (1995).

Michael Herman

Univ. Denis Diderot/Paris 7

Inst.de Math.- Géométrie & Dynamique Tour 45/55 - 5e étage - 2, Place Jussieu 75251 Paris Cedex 05