A family of models is proposed to describe the motion of holes in a fluctuating quantum dimer background on the square lattice. Following Castelnovo et al. [Ann. Phys. (N.Y.) 318, 316 (2005)], a generalized Rokhsar-Kivelson Hamiltonian at finite doping which can be mapped on a doped interacting classical dimer model is constructed. A simple physical extension of this model is also considered. Using numerical computations and simple considerations based on the above exact mapping, we determine the phase diagram of the model showing a number of quantum phases typical of a doped Mott insulator. The two-hole correlation function generically exhibits short-range or long-range algebraic correlations in the solid (columnar) and liquid (critical) phases of the model, respectively. Evidence for an extended region of a doped valence bond solid phase exhibiting holon pairing but no phase separation is given. In contrast, we show that hole deconfinement occurs in the staggered dimer phase.