This paper addresses the control problem of formation keeping of a fleet of underactuated vehicles under communication constraints, where each agent is allowed to communicate with only a subset of the agents of the fleet. We adopt a virtual vehicle approach where every underactuated agent tracks a virtual vehicle, described by a single integrator model, driven by a consensus-based distributed controller. This approach results in a distributed dynamic controller for formation keeping of underactuated vehicles with exponential convergence guarantee of the formation error to zero. Simulation results are presented for both wheeled-like vehicles (2-D case) and UAV-like vehicles (3-D case).