As the share of electricity generation from non-dispatchable sources like wind and photovoltaics grows, so does the need for demand response to stabilize the grid. Since the electricity consumption of heat pumps in buildings is both substantial and flexible, they offer a large potential in this regard. Looking beyond the well-established time-of-use schemes, we investigate the more challenging task of frequency control, in which a baseline consumption and flexibility band are pre-calculated for the following day, then executed based on instantaneous commands from the grid operator. Since most real-world building automation systems do not allow a direct manipulation of the heat pump power, we identify compatible, commissioning-oriented models for predictive control, treating thermostat setpoints as an input and heat pump power as an output. In a two-month simulation study, a piecewise-affine model structure shows significantly better prediction accuracy than a simpler linear model. However, the control performance with both models is similar, with a ratio of total flexibility offered to total energy consumed of 57.1 % and 59.0 %, and a normalized tracking error of 13.7 % and 12.2 %, respectively. We further provide estimates of the efficacy of local battery storage and aggregation over multiple buildings for improving the tracking accuracy, and find an exponential decay of the error as a function of battery size and aggregation volume.