In this paper, we consider mixture approaches that adaptively combine outputs of several parallel running adaptive algorithms. These parallel units can be considered as diversity branches that can be exploited to improve the overall performance. We study various mixture structures where the final output is constructed as the weighted linear combination of the outputs of several constituent filters. Although the mixture structure is linear, the combination weights can be updated in a highly nonlinear manner to minimize the final estimation error such as in Singer and Feder 1999; Arenas-Garcia, Figueiras-Vidal, and Sayed 2006; Lopes, Satorius, and Sayed 2006; Bershad, Bermudez, and Tourneret 2008; and Silva and Nascimento 2008. We distinguish mixture approaches that are convex combinations (where the linear mixture weights are constrained to be nonnegative and sum up to one) [Singer and Feder 1999; Arenas-Garcia, Figueiras-Vidal, and Sayed 2006], affine combinations (where the linear mixture weights are constrained to sum up to one) [Bershad, Bermudez, and Tourneret 2008] and, finally, unconstrained linear combinations of constituent filters [Kozat and Singer 2000]. We investigate mixture structures with respect to their final mean-square error (MSE) and tracking performance in the steady state for stationary and certain nonstationary data, respectively. We demonstrate that these mixture approaches can greatly improve over the performance of the constituent filters. Our analysis is also generic such that it can be applied to inhomogeneous mixtures of constituent adaptive branches with possibly different structures, adaptation methods or having different filter lengths.