In the distributed linear source coding problem a set of distributed sensors observe subsets of a data vector, and provide the fusion center with linearly encoded data. The goal is to determine the encoding matrix of each sensor such that the fusion center reconstructs the entire data vector with minimum mean square error (MSE). The recently proposed local Karhunen Loeve transform (KLT) approach performs this task by optimally determining the encoding matrix of each sensor assuming the other matrices are fixed. This approach is implemented iteratively until convergence is reached. Herein, we propose a greedy-based non-iterative algorithm. In each step, one of the encoding matrices is updated by appending an additional row. The algorithm selects in a greedy fashion one sensor that provides the largest improvement in MSE, and terminates when all the encoding matrices reach their predefined encoded data size. The algorithm can be implemented recursively, and it reduces the complexity from cubic dependency on the data size, using the iterative method, to quadratic dependency. This makes it a prime candidate for on-line and real-time implementations of the distributed KLT. Simulation results show that for many covariance matrix types, the MSE performance of the suggested algorithm is equivalent to the iterative approach.