Accurate forecasting of photovoltaic (PV) and wind production is crucial for the integration of more renewable energy sources into the power grid. To address the limited resolution and costs of methods based on numerical weather predictions (NWP), we take PV production data as main input for forecasting. Since PV power is affected by weather and cloud dynamics, we model spatio-temporal correlations between production data by representing PV systems as nodes of a dynamic graph and embedding production data, geographical information and clear-sky irradiance as signals on that graph. We introduce a new temporal-spatial multi -windows graph attention network (TSM-GAT) for predicting future PV power production. TSM-GAT can adapt to the dynamics of the problem, by learning different graphs over time. It consists of temporal attention with an overlapping-window mechanism that finds the temporal correlations and spatial attention with a multi-window mechanism, which captures different dynamical spatio-temporal correlations for different parts of the forecasting horizon. Thus, it is possible to interpret which PV stations have the most influence when making a prediction for short-, medium-and long-term intra-day forecasts. TSM-GAT outperforms multi-site state-of-the-art models for four to six hours ahead predictions, with average NRMSE 12.4% and 10.5% on a real and synthetic dataset, respectively. Furthermore, it outperforms state-of-the-art models that use NWP as inputs for up to five hours ahead predictions. TSM-GAT yields predicted signals with a closer shape to ground truth than state-of-the-art models, which indicates that it is better at capturing cloud motion and may lead to better generalization capabilities.