We propose and demonstrate a versatile technique to measure the lifetime of the one-phonon Fock state using two-color pump-probe Raman scattering and spectrally resolved, time-correlated photon counting. Following pulsed laser excitation, the n = 1 phonon Fock state is probabilistically prepared by projective measurement of a single Stokes photon. The detection of an anti-Stokes photon generated by a second, time-delayed laser pulse probes the phonon population with subpicosecond time resolution. We observe strongly nonclassical Stokes-anti-Stokes correlations, whose decay maps the single phonon dynamics. Our scheme can be applied to any Raman-active vibrational mode. It can be modified to measure the lifetime of n >= 1 Fock states or the phonon quantum coherences through the preparation and detection of two-mode entangled vibrational states.