We derive the electric and magnetic Green's functions in homogeneous dielectric uniaxial media by casting the problem as ordinary differential equations. These Green's functions feature an apparent singularity on the medium's distinguished axis. Using rigorous mathematical models of the fields, thanks to Schwartz distributions, we show that Green's functions are only singular at the origin. In doing so, we give a practical criterion to determine the singular behavior of derivatives of singular functions. Next, we provide the value of Green's functions along the distinguished axis. Finally, we outline possible generalizations of the approach to homogeneous biaxial media.