In the distributed linear source coding problem, a set of distributed sensors observe subsets of a data vector with noise, and provide the fusion center linearly encoded data. The goal is to determine the encoding matrix of each sensor such that the fusion center can reconstruct the entire data vector with minimum mean square error. The recently proposed local Karhunen-Loeve transform approach performs this task by optimally determining the encoding matrix of each sensor assuming the other matrices are fixed. This approach is implemented iteratively until convergence is reached. Herein, we propose a greedy algorithm. In each step, one of the encoding matrices is updated by appending an additional row. The algorithm selects in a greedy fashion a single sensor that provides the largest improvement in minimizing the mean square error. This algorithm terminates after a finite number of steps, that is, when all the encoding matrices reach their predefined encoded data size. We show that the algorithm can be implemented recursively, and compared to the iterative approach, the algorithm reduces the computational load from cubic dependency to quadratic dependency on the data size. This makes it a prime candidate for on-line and real-time implementations of the distributed Karhunen-Loeve transform. Simulation results suggest that the mean square error performance of the suggested algorithm is equivalent to the iterative approach.