We study the capacity of the power-constrained additive Gaussian channel with an entropy constraint at the input. In particular, we characterize this capacity in the low signal-to-noise ratio regime at small entropy. This follows as a corollary of the following general result on a moment matching problem: We show that for any continuous random variable with finite moments, the largest number of initial moments that can be matched by a discrete random variable of sufficiently small but positive entropy is three.