Consider a scenario where a distributed signal is sparse and is acquired by various sensors that see different versions. Thus, we have a set of sparse signals with both some common parts, and some variations. The question is how to acquire such signals and how to reconstruct them perfectly (noiseless case) or approximately (noisy case). We propose an extension of the annihilating filter method [3] to this distributed scenario. We model the inter- relation between the sparse signals by introducing three joint sparse models. For each model, we propose sensing and reconstruction algorithms that reduce the number of measurements below the limit for the single sensor scenario and results in power and bandwidth reduction in the system. In the noiseless scenario, we are close to the minimum number of measurements possible for the perfect reconstruction while by taking more measurements, we introduce redundancy in the system to effectively mitigate the noise. Simulation results justify the applicability of the approach.