A reduced basis Darcy-Stokes finite element heterogeneous multiscale method (RB-DS-FE-HMM) is proposed for the Stokes problem in porous media. The multiscale method is based on the Darcy-Stokes finite element heterogeneous multiscale method (DS-FE-HMM) introduced in [A. Abdulle, O. Budáč, Multiscale Model. Simul. 13 (2015)] that couples a Darcy equation solved on a macroscopic mesh, with missing permeability data extracted from the solutions of Stokes micro problems at each macroscopic quadrature point. To overcome the increasingly growing cost of repeatedly solving the Stokes micro problems as the macroscopic mesh is refined, we parametrize the microscopic solid geometry and approximate the infinite-dimensional manifold of parameter dependent solutions of Stokes problems by a low-dimensional space. This low-dimensional (reduced basis) space is obtained in an offline stage by a greedy algorithm and used in an online stage to compute the effective Darcy permeability at a cost independent of the microscopic mesh. The discretization of the parametrized Stokes problems relies on a Petrov-Galerkin formulation that allows for a stable and fast online evaluation of the required permeabilities. A priori and a posteriori estimates of the RB-DS-FE-HMM are derived and a residual-based adaptive algorithm is proposed. Two- and three-dimensional numerical experiments confirm the accuracy of the RB-DS-FE-HMM and illustrate the speedup compared to the DS-FE-HMM.