We present the M3 framework, a formal and generic computational framework for modeling and controlling distributed stochastic systems of purely reactive robots in an automated and real-time fashion. Based on the trajectories of the robots, the framework builds up an internal microscopic representation of the system, which then serves as a blueprint of models at higher abstraction levels. These models are then calibrated using a Maximum Likelihood Estimation (MLE) approach. We illustrate the structure and performance of the framework by performing the online optimization of a simple bang-bang controller for the stochastic self-assembly of water- floating passive robots. The experimental results demonstrate that the generated models can successfully optimize the assem- bly of desired structures.