This paper provides a time-domain feedback analysis of the robustness performance of Gauss-Newton recursive methods that are often used in identification and control. These are recursive estimators that also involve updates of sample covariance matrices. Several free parameters are included in the filter descriptions while combining the covariance updates with the weight-vector updates. One of the contributions of this work is to show that by properly selecting the free parameters, the resulting filters can be made to impose certain bounds on the error quantities, thus resulting in desirable robustness properties (along the lines of H∞ filter designs). It is also shown that an intrinsic feedback structure, mapping the noise sequence and the initial weight error to the a priori estimation errors and the final weight error, can be associated with such schemes. The feedback configuration is motivated via energy arguments and is shown to consist of two major blocks: a time-variant lossless (i.e., energy preserving) feedforward path and a time-variant feedback path. Emphasis is further given to filtered-error variants that give rise to dynamic time-variant feedback loops rather than memoryless loops. Such variants arise in IIR modeling.