Let K be a closed subgroup of
U(n) acting on the (2n + 1)-dimensional Heisenberg group Hn by automorphisms.
One calls (K,Hn) a Gelfand pair when the integrable K-invariant functions on Hn
form a commutative algebra under convolution. We prove that this is the case if and
only if the coadjoint orbits for G := K ⋉ Hn which meet the annihilator k⊥ of the
Lie algebra f of K do so in single K-orbits. Equivalently, the representation of K on
the polynomial algebra over ℂn is multiplicity free if and only if the moment map
from ℂn to k∗ is one-to-one on K-orbits.
It is also natural to conjecture that the spectrum of the quasi-regular
representation of G on L2(G∕K) corresponds precisely to the integral coadjoint orbits
that meet k⊥. We prove that the representations occurring in the quasi-regular
representation are all given by integral coadjoint orbits that meet k⊥. Such
orbits can, however, also give rise to representations that do not appear in
L2(G∕K).
