We establish in the world of stochastic processes a theoretical relation between sparsity and wavelets. The underlying principle is to treat stochastic processes as generalized functions, which facilitates the study of their properties in a transform domain. We focus on symmetric-$\alpha$-stable (S$\alpha$S) processes, with $\alpha\in(0,2]$. They are central to a recently proposed framework for sparse stochastic processes. The case $0<\alpha<2$ corresponds to heavy-tail processes and, thus, to sparse signals with the level of sparsity being more pronounced for smaller values of $\alpha$. The limit case $\alpha=2$ yields the classical Gaussian processes, which are not considered to be sparse. In the first part of the thesis, we identify a particular class of wavelets and show that they provide an independent-component analysis for S$\alpha$S AR(1) signals, which form an important subclass of sparse processes. This contribution can be considered as the first theoretical result ever found about the optimality of wavelets in a stochastic framework. It is remarkable that, by decreasing the parameter $\alpha$, we continuously run the gamut from Fourier (for $\alpha=2$) to wavelets (for $\alpha\le1$), thus providing a unifying view that sees Fourier and wavelet transforms as two extremes of a single spectrum. In the second part of the thesis, we study the wavelet decomposition of self-similar processes---which form a second important family of sparse S$\alpha$S processes. In particular, in the context of the wavelet-based denoising of a stochastic process, we are able to theoretically predict the exact performance of an arbitrary orthonormal wavelet basis. As it turns out, our prediction takes a tractable simple form, which allows us to design the optimal Meyer wavelet as an example of application. Our theoretical results suggest that the performance depends crucially on the localization of wavelets. Consequently, in an image-processing context, we introduce a moment-based measure of localization and propose an optimization framework to design better wavelets. We end the thesis with the experimental confirmation that the proposed wavelets outperform the previously existing ones.