An adaptive reduced basis finite element heterogeneous multiscale method (RB-FE-HMM) is proposed for elliptic problems with multiple scales. The multiscale method is based on the RB-FE-HMM introduced in [A. Abdulle, Y. Bai, Reduced basis finite element heterogeneous multiscale method for high-order discretizations of elliptic homogenization problems, J. Comput. Phys. 231 (21) (2012) 7014-7036]. It couples a macroscopic solver with effective data recovered from the solution of micro problems solved on sampling domains. Unlike classical numerical homogenization methods, the micro problems are computed in a finite dimensional space spanned by a small number of accurately computed representative micro solutions (the reduced basis) obtained by a greedy algorithm in an offline stage. In this paper we present a residual-based a posteriori error analysis in the energy norm as well as an a posteriori error analysis in quantities of interest. For both type of adaptive strategies, rigorous a posteriori error estimates are derived and corresponding error estimators are proposed. In contrast to the adaptive finite element heterogeneous multiscale method (FE-HMM), there is no need to adapt the micro mesh simultaneously to the macroscopic mesh refinement. Up to an offline preliminary stage, the RB-FE-HMM has the same computational complexity as a standard adaptive FEM for the effective problem. Two and three dimensional numerical experiments confirm the efficiency of the RB-FE-HMM and illustrate the improvements compared to the adaptive FE-HMM. (C) 2013 Elsevier B.V. All rights reserved.