We present a technique for the rapid and reliable prediction of linear-functional outputs of coercive and non-coercive linear elliptic partial differential equations with affine parameter dependence. The essential components are: (i) rapidly convergent global reduced basis approximations – (Galerkin) projection onto a space WN spanned by solutions of the governing partial differential equation at N judiciously selected points in parameter space; (ii) a posteriori error estimation – relaxations of the error-residual equation that provide inexpensive yet sharp bounds for the error in the outputs of interest; and (iii) offline/online computational procedures – methods which decouple the generation and projection stages of the approximation process. The operation count for the online stage – in which, given a new parameter value, we calculate the output of interest and associated error bound – depends only on N (typically very small) and the parametric complexity of the problem. In this paper we propose a new “natural norm” formulation for our reduced basis error estimation framework that: (a) greatly simplifies and improves our inf–sup lower bound construction (offline) and evaluation (online) – a critical ingredient of our a posteriori error estimators; and (b) much better controls – significantly sharpens – our output error bounds, in particular (through deflation) for parameter values corresponding to nearly singular solution behavior. We apply the method to two illustrative problems: a coercive Laplacian heat conduction problem – which becomes singular as the heat transfer coefficient tends to zero; and a non-coercive Helmholtz acoustics problem – which becomes singular as we approach resonance. In both cases, we observe very economical and sharp construction of the requisite natural-norm inf–sup lower bound; rapid convergence of the reduced basis approximation; reasonable effectivities (even for near-singular behavior) for our deflated output error estimators; and significant – several order of magnitude – (online) computational savings relative to standard finite element procedures.