This paper proposes a novel approach to improve the performance of distributed nonlinear control systems while preserving stability by leveraging Deep Neural Networks (DNNs). We build upon the Neural System Level Synthesis (Neur-SLS) framework and introduce a method to parameterize stabilizing control policies that are distributed across a network topology. A distinctive feature is that we iteratively minimize an arbitrary control cost function through an unconstrained optimization algorithm, all while preserving the stability of the overall network architecture by design. This is achieved through two key steps. First, we establish a method to parameterize interconnected Recurrent Equilibrium Networks (RENs) that guarantees a bounded L2 gain at the network level. This ensures stability. Second, we demonstrate how the information flow within the network is preserved, enabling a fully distributed implementation where each subsystem only communicates with its neighbors. To showcase the effectiveness of our approach, we present a simulation of a distributed formation control problem for a fleet of vehicles. The simulation demonstrates how the proposed neural controller enables the vehicles to maintain a desired formation while navigating obstacles and avoiding collisions, all while guaranteeing network stability.