This paper surveys recent advances related to adaptation, learning, and optimization over networks. Various distributed strategies are discussed that enable a collection of networked agents to interact locally in response to streaming data and to continually learn and adapt to track drifts in the data and models. Under reasonable technical conditions on the data, the adaptive networks are shown to be mean square stable in the slow adaptation regime, and their mean square error performance and convergence rate are characterized in terms of the network topology and data statistical moments. Classical results for single-agent adaptation and learning are recovered as special cases. The performance results presented in this work are useful in comparing network topologies against each other, and in comparing adaptive networks against centralized or batch implementations. The presentation is complemented with various examples linking together results from various domains.