The softening behaviour during non-isothermal annealing of a cold-rolled Al–Mn–Fe–Si model alloy was studied as a function of the state of microchemistry, in terms of the solute level of Mn, size and spatial distribution of the Mn-bearing dispersoids, as well as their temporal evolution. Microchemistry significantly affects the recrystallization microstructure, crystallographic texture as well as the mechanical property of the investigated alloy after non-isothermal annealing. The nucleation and growth of grains with different orientations are strongly dependent on both annealing temperature and microchemistry, in that pre-existing dispersoids have a less profound effect on retarding recrystallization than dispersoids forming concurrently during back-annealing. Strong concurrent precipitation suppresses nucleation and retards recrystallization, which finally leads to a coarse and pan-cake shaped grain structure, accompanied by strong P {011}〈566〉 and/or M {113}〈110〉 texture components and a relatively weaker ND-rotated cube {001}〈310〉 component. A refined grain structure with medium strength P and cube {001}〈100〉 components is obtained when the pre-existing dispersoids are coarser and fewer, and concurrent precipitation is limited. P-oriented grains are less affected by second phase particles and experience a growth advantage at low annealing temperatures (<350 °C), while M-oriented grains appear at higher temperatures. The intensity of the P texture does not necessarily increase with increasing supersaturation of Mn as observed during isothermal annealing, whereas the level of supersaturated Mn promotes the strength of the M texture. The mechanisms behind are discussed.