We introduce a density-functional perturbational scheme based on ultrasoft pseudopotentials for calculating dielectric tensors of periodic systems. We obtain a variational functional for the second-order derivative of the energy with respect to an electric field. Our scheme makes use of the correspondence between all-electron and pseudo-wave-functions introduced in the projector augmented wave method. While we here specifically focus on ultrasoft pseudopotentials, our scheme also covers the case of norm-conserving ones. In the latter case, our formulation coincides with earlier ones but highlights the implied approximations. By construction, our scheme also applies to the all-electron projector augmented wave method. We first assess the validity of our scheme by calculating polarizability tensors and Raman intensities of small molecules (H2O, CH4, NH3). We find good agreement with both experimental data and previous all-electron results. We then illustrate the potential of our scheme for treating systems of relatively large size through an application to a disordered model structure of vitreous silica.