The vector analysis of stimulated Brillouin scattering amplification in birefringent fibers is extended to include signal pulses. The analysis finds that the different slow-light delays experienced by the states of polarization corresponding to maximum and minimum gain may result in severe pulse distortion. Thus, a generally polarized pulse, experiencing only a moderate gain, can become broader than a pulse aligned for maximum gain and delay. The effect is demonstrated in both numerical simulations and experiments.