The aim of modern energy systems is to be independent of fossil fuels and to minimize their impact on the environment. Photovoltaic (PV) panels are a common solution for renewable electric energy generation, but the volatility caused by the fluctuation of solar irradiation challenges the the electrical power grid. The aim of this paper is to explore the influence of different orientations of PV panels on the buildings self-sufficiency and self-consumption. This paper is based on a mixed integer-linear programming approach aiming at optimizing the installed size of a range of energy conversion technologies. Compared to conventional building energy system optimization research, the proposed approach offers a more detailed modelling of the energy generated by PV panels that considers the azimuth and the tilt of roof surfaces and PV panels, also including the shading effects between different panels. The proposed methodology is applied to a case study building in the area of Rolle, Switzerland. The results of this paper suggest that the tilt and azimuth of PV panels and of the roof they are installed on should not be neglected: not considering these aspects generates an error in the estimation of the energy generated by the PV panels, and not including the choice of the roof (and, in case of flat roofs, the choice of PV panel tilt and orientation) where the panels should be installed on as design variables generates a sub-optimization of the system. This is particularly true since the most optimal choice is not necessarily that of filling first flat surfaces and South-oriented ones, a consequence of the better match between generation and demand offered by West-oriented panels, with a higher tilt angle.