Classical models of aerial swarms often describe global coordinated motion as the combination of local interactions that happen at the individual level. Mathematically, these interactions are represented with potential fields. Despite their explanatory success, these models fail to guarantee rapid and safe collective motion when applied to aerial robotic swarms flying in cluttered environments of the real world, such as forests and urban areas. Moreover, these models necessitate a tight coupling with the deployment scenarios to induce consistent swarm behaviours. Here, we propose a predictive model that incorporates the local principles of potential field models in an objective function and optimizes those principles under the knowledge of the agents’ dynamics and environment. We show that our approach improves the speed, order and safety of the swarm, it is independent of the environment layout and is scalable in the swarm speed and inter-agent distance. Our model is validated with a swarm of five quadrotors that can successfully navigate in a real-world indoor environment populated with obstacles.