The most successful and popular machine learning models of atomic-scale properties derive their transferability from a locality ansatz. The properties of a large molecule or a bulk material are written as a sum over contributions that depend on the configurations within finite atom-centered environments. The obvious downside of this approach is that it cannot capture nonlocal, nonadditive effects such as those arising due to long-range electrostatics or quantum interference. We propose a solution to this problem by introducing nonlocal representations of the system, which are remapped as feature vectors that are defined locally and are equivariant in O(3). We consider, in particular, one form that has the same asymptotic behavior as the electrostatic potential. We demonstrate that this framework can capture nonlocal, long-range physics by building a model for the electrostatic energy of randomly distributed point-charges, for the unrelaxed binding curves of charged organic molecular dimers, and for the electronic dielectric response of liquid water. By combining a representation of the system that is sensitive to long-range correlations with the transferability of an atom-centered additive model, this method outperforms current state-of-the-art machine-learning schemes and provides a conceptual framework to incorporate nonlocal physics into atomistic machine learning.