A new set of algorithms for transform adaptation in adaptive transform coding is presented. These algorithms are inspired by standard techniques in adaptive finite impulse response (FIR) Wiener filtering and demonstrate that similar algorithms with simple updates exist for tracking principal components (eigenvectors of a correlation matrix). For coding an {N}-dimensional source, the transform adaptation problem is posed as an unconstrained minimization over {K = N(N-1)/2} parameters, and this for two possible performance measures. Performing this minimization through a gradient descent gives an algorithm analogous to LMS@. Step size bounds for stability similar in form to those for LMS are proven. Linear and fixed-step random search methods are also considered. The stochastic gradient descent algorithm is simulated for both time-invariant and slowly-varying sources. A ``backward-adaptive'' mode, where the adaptation is based on quantized data so that the decoder and encoder can maintain the same state without side information, is also considered.