The remarkable ability of deep learning (DL) models to approximate high-dimensional functions from samples has sparked a revolution across numerous scientific and industrial domains that cannot be overemphasized. In sensitive applications, the good performance of DL is unfortunately sometimes overshadowed by unexpected behaviors, including hallucinations in medical image reconstruction. Serious concerns have thus been raised regarding the extent to which one can trust the output of DL models. Restoring trust is challenging since the same depth that fuels the performance causes DL models to be black boxes. The parameters of the model are indeed only remotely connected to the function they parameterize, and enforcing constraints on the model to obtain guarantees on its output usually wipes out the performance boost of DL. In this thesis, we pursue the goal of improving the trustworthiness of several DL methods while maintaining performance. Our approach tackles the problem via the design of expressive, stable and interpretable spline-based parameterizations across various contexts. The contributions of this thesis are divided into three parts. In the first part, we concentrate on parameterizations for low-dimensional regression tasks. There, depth is not beneficial and one can have it all---stability, expressivity and interpretability---with linear combinations of well-chosen atoms. This is first shown with the design of shortest-support multi-spline bases, and then with the study of the stability of a local parameterization of continuous and piecewise-linear functions (CPWL). In the second part, we focus on deep parameterizations to cope with higher-dimensional problems. We first study the composition operation within CPWL neural networks (NN) and give some new insights into the role of the activation function in the expressivity of the NN. We then propose to use Lipschitz-constrained learnable linear spline activations to build expressive and provably stable deep NN. We characterize some universal properties of our framework, develop an efficient procedure to train the activations under the constraint, and, lastly, show experimental improvements over competing frameworks with similar constraints on various tasks, including plug-and-play image reconstruction with provably nonexpansive denoisers. In the third and final part, we refine the parameterization by focusing on image reconstruction tasks. We propose a framework to learn convex regularizers, which rely on our learnable Lipschitz-constrained spline activations. The parameterization yields lightweight and transparent---in contrast to black boxes---models with theoretical guarantees on the reconstruction. Our method exhibits state-of-the-art performance for CT and MRI reconstruction among convex regularization methods. Lastly, we extend the framework to learn weakly-convex regularizers to boost performance while maintaining most guarantees.