We provide a time-domain analysis of the robustness and stability performance of Gauss- Newton recursive methods that are often used in identification and control. Several free parameters are included in the filter description while combining the covariance update with the weight-vector update; the exponentially weighted recursive least squares algorithm being an important special case. One of the contributions of this work is to show that by properly selecting the free parameters, the resulting filter can be shown to impose certain bounds on the error quantities, thus resulting in desireable robustness and stability properties. We also show that an intrinsic feedback structure, mapping the noise sequence and the initial weight guess to the a priori estimation errors and the final weight estimate, can be associated with such schemes. The feedback configuration is motivated via energy arguments and is shown to consist of two major blocks: a time-variant lossless (i.e., energy preserving) feedforward path and a time-variant feedback path.