Various direct-ambient signal decompositions, applicable to two-channel stereophonic signals, have been proposed previously. Strategies to enhance these decompositions for multi-channel surround audio signals are not obvious. A method is proposed, which estimates direct and ambient signal components in a downmix of a given surround signal. Those estimates are then used to compute filters to obtain direct and ambient surround signals. Approaches on how to use the decomposed signals for upmixing of surround audio signals in two and three dimensions are also discussed.