The incremental deployment of decentralized renewable energy sources in the distribution grid is triggering a paradigm change for the power sector. This shift from a centralized structure with big power plants to a decentralized scenario of distributed energy resources, such as solar and wind, calls for a more active management of the distribution grid. Conventional distribution grids were passive systems, in which the power was flowing unidirectionally from upstream to downstream. Nowadays, and increasingly in the future, the penetration of distributed generation (DG), with its stochastic nature and lack of controllability, represents a major challenge for the stability of the network, especially at the distribution level. In particular, the power flow reversals produced by DG cause voltage excursions, which must be compensated. This poses an obstacle to the energy transition towards a more sustainable energy mix, which can however be mitigated by using a more active approach towards the control of the distribution networks. Demand side management (DSM) offers a possible solution to the problem, allowing to actively control the balance between generation, consumption and storage, close to the point of generation. An active energy management implies not only the capability to react promptly in case of disturbances, but also to ability to anticipate future events and take control actions accordingly. This is usually achieved through model predictive control (MPC), which requires a prediction of the future disturbances acting on the system.

This thesis treat challenges of distributed DSM, with a particular focus on the case of a high penetration of PV power plants. The first subject of the thesis is the evaluation of the performance of models for forecasting and control with low computational requirements, of distributed electrical batteries. The proposed methods are compared by means of closed loop deterministic and stochastic MPC performance. The second subject of the thesis is the development of model based forecasting for PV power plants, and methods to estimate these models without the use of dedicated sensors. The third subject of the thesis concerns strategies for increasing forecasting accuracy when dealing with multiple signals linked by hierarchical relations. Hierarchical forecasting methods are introduced and a distributed algorithm for reconciling base forecasters is presented. At the same time, a new methodology for generating aggregate consistent probabilistic forecasts is proposed. This method can be applied to distributed stochastic DSM, in the presence of high penetration of rooftop installed PV systems. In this case, the forecasts' errors become mutually dependent, raising difficulties in the control problem due to the nontrivial summation of dependent random variables. The benefits of considering dependent forecasting errors over considering them as independent and uncorrelated, are investigated. The last part of the thesis concerns models for distributed energy markets, relying on hierarchical aggregators. To be effective, DSM requires a considerable amount of flexible load and storage to be controllable. This generates the need to be able to pool and coordinate several units, in order to reach a critical mass. In a real case scenario, flexible units will have different owners, who will have different and possibly conflicting interests. In order to recruit as much flexibility as possible, it is therefore important