Groundwater is a key water resource in semiarid and seasonally dry regions around the world, which is replenished by intermittent precipitation events and mediated by vegetation, soil, and regolith properties. Here, a climate reconstruction of 4500 years for the Jerusalem region was used to determine the relation between climate, vegetation, and groundwater recharge. Despite changes in air temperature and vegetation characteristics, simulated recharge remained linearly related to precipitation over the entire analyzed period, with drier decades having lower rates of recharge for a given annual precipitation due to soil memory effects. We show that in recent decades, the lack of changes in the precipitation–groundwater recharge relation results from the compensating responses of vegetation to increasing CO2, i.e., increased leaf area and reduced stomatal conductance. This multicentury relation is expected to be modified by climate change, with changes up to −20% in recharge for unchanged precipitation, potentially jeopardizing water resource availability.