In this paper, we study the interactions of electromagnetic waves with a non-dispersive dynamic medium that is temporally dependent. Electromagnetic fields under material time-modulation conserve their momentum but not their energy. We assume a time-variation of the permittivity, permeability and conductivity and derive the appropriate time-domain solutions based on the causality state at a past observation time. We formulate a time-transitioning state matrix and connect the unusual energy transitions of electromagnetic fields in time-varying media with the exceptional point theory. This state-matrix approach allows us to analyze further the electromagnetic waves in terms of parity and time-reversal symmetries and signify parity-time symmetric wave-states without the presence of a spatially symmetric distribution of gain and loss, or any inhomogeneities and material periodicity. This paper provides a useful arsenal to study electromagnetic wave phenomena under time-varying media and points out novel physical insights connecting the resulting energy transitions and electromagnetic modes with exceptional point physics and operator symmetries.