Most orthogonal signal decompositions, including block transforms, wavelet transforms, wavelet packets, and perfect reconstruction filterbanks in general, can be represented by a paraunitary system matrix. Here, we consider the general problem of finding the optimal P × P paraunitary transform that minimizes the approximation error when a signal is reconstructed from a reduced number of components Q < P. This constitutes a direct extension of the Karhunen-Loève transform which provides the optimal solution for block transforms (unitary system matrix). We discuss some of the general properties of this type of solution. We review different approaches for finding optimal and sub-optimal decompositions for stationary processes. In particular, we show that the solution can be determined analytically in the unconstrained case. If one includes order or length constraints, then the optimization problem turns out to be much more difficult.