In this article, a finite element error analysis is performed on a class of linear and nonlinear elliptic problems with small uncertain input. Using a perturbation approach, the exact (random) solution is expanded up to a certain order with respect to a parameter that controls the amount of randomness in the input and discretized by finite elements. We start by studying a diffusion (linear) model problem with a random coefficient characterized via a finite number of random variables. The main focus of the article is the derivation of a priori and a posteriori error estimates of the error between the exact and approximate solution in various norms, including goal-oriented error estimation. The analysis is then extended to a class of nonlinear problems. We finally illustrate the theoretical results through numerical examples, along with a comparison with the Stochastic Collocation method in terms of computational costs.