In this paper, a finite element error analysis is performed on a class of linear and nonlinear elliptic problems with small uncertain input. Using a perturbation approach, the exact (random) solution is expanded up to a certain order with respect to a parameter that controls the amount of randomness in the input and discretized by finite elements. We start by studying a diffusion (linear) model problem with a random coefficient characterized via a finite number of random variables. A priori and a posteriori estimates of the error between the exact and approximate solution are given in various norms, including goal-oriented error estimation. The analysis is then extended to a class of nonlinear problems. We finally illustrate the theoretical results through numerical examples, along with a comparison with the Stochastic Collocation method in terms of computational costs.