The numerical solution of partial differential equations (PDEs) depending on para- metrized or random input data is computationally intensive. Reduced order modeling techniques, such as the reduced basis methods, have been developed to alleviate this computational burden, and are nowadays exploited to accelerate real-time analysis, as well as the solution of PDE-constrained optimization and inverse problems. These methods are built upon low-dimensional spaces obtained by selecting a set of snap- shots from a parametrically induced manifold. However, for these techniques to be effective, both parameter-dependent and random input data must be expressed in a convenient form. To address the former case, the empirical interpolation method has been developed. In the latter case, a spectral approximation of stochastic fields is often generated by means of a Karhunen-Loève expansion. In all these cases, a low dimensional space to represent the function being approximated (PDE solution, parametrized data, stochastic field) can be obtained through proper orthogonal de- composition. Here, we review possible ways to exploit this methodology in these three contexts, we recall its optimality properties, and highlight the common mathematical structure beneath.