We obtain phonon lifetimes in aluminium by inelastic neutron scattering experiments, by ab initio molecular dynamics, and by perturbation theory. At elevated temperatures significant discrepancies are found between experiment and perturbation theory, which disappear when using molecular dynamics due to the inclusion of full anharmonicity and the correct treatment of the multiphonon background. We show that multiple-site interactions are small and that local pairwise anharmonicity dominates phonon-phonon interactions, which permits an efficient computation of phonon lifetimes.