The paper develops a leakage-based adaptive algorithm, referred to as circular-leaky, which in addition to solving the drift problem of the classical least mean squares (LMS) adaptive algorithm, it also avoids the bias problem that is created by the standard leaky LMS solution. These two desirable properties of unbiased and bounded estimates are guaranteed by circular-leaky at essentially the same computational cost as LMS. The derivation in the paper relies on results from averaging theory and from Lyapunov stability theory, and the analysis shows that the above properties hold not only in infinite-precision but also in finite-precision arithmetic. The paper further extends the results to a so-called switching-/spl sigma/ algorithm, which is a leakage-based solution used in adaptive control.