Invariance under time translation (or stationarity) is probably one of the most important assumptions made when investigating electromagnetic phenomena. Breaking this assumption is expected to open up novel possibilities and result in exceeding conventional limitations. However, to explore the field of time-varying electromagnetic structures, we primarily need to contemplate the fundamental principles and concepts from a nonstationarity perspective. Here, we revisit one of those key concepts: the polarizability of a small particle, assuming that its properties vary in time. We describe the creation of induced dipole moment by external fields in a nonstationary, causal way, and introduce a complex-valued function, called temporal complex polarizability, for elucidating a nonstationary Hertzian dipole under time-harmonic illumination. This approach can be extended to any subwavelength particle exhibiting electric response. In addition, we also study the classical model of the polarizability of an oscillating electron using the equation of motion whose damping coefficient and natural frequency are changing in time. Next, we theoretically derive the effective permittivity corresponding to time-varying media (comprising free or bound electrons, or dipolar meta-atoms) and explicitly show the differences with the conventional macroscopic Drude–Lorentz model. This paper will hopefully pave the road towards better understanding of nonstationary scattering from small particles and homogenization of time-varying materials, metamaterials, and metasurfaces.