The significant progress that has been made in recent years both in hardware implementations and in numerical computing has rendered real-time optimization-based control a viable option when it comes to advanced industrial applications. At the same time, the field of big data has emerged, seeking solutions to problems that classical optimization algorithms are incapable of providing. Though for different reasons, both application areas triggered interest in revisiting the family of optimization algorithms commonly known as decomposition schemes or operator splitting methods. This lately revived interest in these methods can be mainly attributed to two characteristics: Com- putationally low per-iteration cost along with small memory footprint when it comes to embedded applications, and their capacity to deal with problems of vast scales via decomposition when it comes to machine learning-related applications. In this thesis, we design decomposition methods that tackle both small-scale centralized control problems and larger-scale multi-agent distributed control problems. In addition to the classical objective of devising faster methods, we also delve into less usual aspects of operator splitting schemes, which are nonetheless critical for control. In the centralized case, we propose an algorithm that uses decomposition in order to exactly solve a classical optimal control problem that could otherwise be solved only approximately. In the multi-agent framework, we propose two algorithms, one that achieves faster convergence and a second that reduces communication requirements.