In this paper, we propose reduced basis multiscale finite element methods (RB-MsFEM) for elliptic problems with highly oscillating coefficients. The method is based on multiscale finite element methods with local test functions that encode the oscillatory behavior ([4, 14]). For uniform rectangular meshes, the local oscillating test functions are represented by a reduced basis method, parameterizing the center of the elements. For triangular elements, we introduce a slightly different approach. By exploring over-sampling of the oscillating test functions, initially introduced to recover a better approximations of the global harmonic coordinate map, we first build the reduced basis on uniform rectangular elements containing the original triangular elements and then restrict the oscillating test function to the triangular elements. These techniques are also generalized to the case where the coefficients dependent on additional independent parameters. The analysis of the proposed methods is supported by various numerical results, obtained on regular and unstructured grids.