The respiratory rate is an important vital sign that needs to be monitored continuously in clinical and non-clinical health monitoring applications. It is commonly estimated from electrocardiogram (ECG)-derived respiratory waveforms such as the respiratory sinus arrhythmia~(RSA) and the ECG R peak amplitude~(RPA). Current methods combine respiratory information from these two waveforms but produce large delays in estimating the respiratory rate. In this work, the power of the outputs of a bank of order-3 FIR notch filters were used in an adaptive scheme to estimate in a real-time manner and with minimal delay the respiratory rate from the RSA and the RPA waveforms simultaneously. The algorithm was tested on the public Physionet Fantasia data set and compared to the state-of-the-art in terms of estimation accuracy and delay. It was shown that the proposed method provides more accurate estimates with smaller delays than those of the state-of-the-art.