Foundations of signal processing are heavily based on Shannon's sampling theorem for acquisition, representation and reconstruction. This theorem states that signals should not contain frequency components higher than the Nyquist rate, which is half of the sampling rate. Then, the signal can be perfectly reconstructed from its samples. Increasing evidence shows that the requirements imposed by Shannon's sampling theorem are too conservative for many naturally-occurring signals, which can be accurately characterized by sparse representations that require lower sampling rates closer to the signal's intrinsic information rates. Finite rate of innovation (FRI) is a new theory that allows to extract underlying sparse signal representations while operating at a reduced sampling rate. The goal of this PhD work is to advance reconstruction techniques for sparse signal representations from both theoretical and practical points of view. Specifically, the FRI framework is extended to deal with applications that involve temporal and spatial localization of events, including inverse source problems from radiating fields. We propose a novel reconstruction method using a model-fitting approach that is based on minimizing the fitting error subject to an underlying annihilation system given by the Prony's method. First, we showed that this is related to the problem known as structured low-rank matrix approximation as in structured total least squares problem. Then, we proposed to solve our problem under three different constraints using the iterative quadratic maximum likelihood algorithm. Our analysis and simulation results indicate that the proposed algorithms improve the robustness of the results with respect to common FRI reconstruction schemes. We have further developed the model-fitting approach to analyze spontaneous brain activity as measured by functional magnetic resonance imaging (fMRI). For this, we considered the noisy fMRI time course for every voxel as a convolution between an underlying activity inducing signal (i.e., a stream of Diracs) and the hemodynamic response function (HRF). We then validated this method using experimental fMRI data acquired during an event-related study. The results showed for the first time evidence for the practical usage of FRI for fMRI data analysis. We also addressed the problem of retrieving a sparse source distribution from the boundary measurements of a radiating field. First, based on Green's theorem, we proposed a sensing principle that allows to relate the boundary measurements to the source distribution. We focused on characterizing these sensing functions with particular attention for those that can be derived from holomorphic functions as they allow to control spatial decay of the sensing functions. With this selection, we developed an FRI-inspired non-iterative reconstruction algorithm. Finally, we developed an extension to the sensing principle (termed eigensensing) where we choose the spatial eigenfunctions of the Laplace operator as the sensing functions. With this extension, we showed that eigensensing principle allows to extract partial Fourier measurements of the source functions from boundary measurements. We considered photoacoustic tomography as a potential application of these theoretical developments.