The Karhunen-Loeve transform (KLT) is a key element of many signal processing tasks, including approximation, compression, and classification. Many recent applications involve {\em distributed} signal processing where it is not generally possible to apply the KLT to the signal; rather, the KLT must be approximated in a distributed fashion. This paper investigates such distributed approximations to the KLT. First, we present explicit solutions to special cases, including a partial KLT (where only a subset of the sources is observed), a conditional KLT (where some sources act as side information), and the combination of these two special cases. These results are used to derive an algorithm that finds the best distributed approximation to the KLT. Applications of our results to sensor networks and to distributed databases are discussed