We study the expressivity and learning process for polynomial neural networks
(PNNs) with monomial activation functions. The weights of the network parametrize
the neuromanifold. We study certain neuromanifolds using tools from algebraic
geometry: we give explicit descriptions as semialgebraic sets and characterize their
Zariski closures, called neurovarieties. We study their dimension and associate an
algebraic degree, the learning degree, to the neurovariety. The dimension serves as a
geometric measure for the expressivity of the network, the learning degree is a
measure for the complexity of training the network and provides upper bounds on the
number of learnable functions. These theoretical results are accompanied with
experiments.
