We develop a fully vectorial formalism for the investigation of electromagnetic scattering in polarizable backgrounds, i.e., where the scatterers are not in vacuum but situated in a medium with a dielectric permittivity different from unity. Our approach is based on the Green's tensor technique and the corresponding Green's tensors for two-dimensional (2D) and three-dimensional (3D) systems are developed. The analysis of 2D systems is not restricted to the case where transverse electric (TE) and transverse magnetic (TM) modes are decoupled, but treated in a general manner. Practical examples illustrate the application of the method: scattering by a microcavity for two dimensions and color formation in opal for three dimensions.