Macroscopic fundamental diagrams (MFDs) have been widely adopted to model the traffic flow of large-scale urban networks. Coupling perimeter control and regional route guidance (PCRG) is a promising strategy to decrease congestion heterogeneity and reduce delays in large-scale MFD-based urban networks. For MFD-based PCRG, one needs to distinguish between the dynamics of (a) the plant that represents reality and is used as the simulation tool and (b) the model that contains easier-to-measure states than the plant and is used for devising controllers, that is, the model-plant mismatch should be considered. Traditional model-based methods (e.g., model predictive control (MPC)) require an accurate representation of the plant dynamics as the prediction model. However, because of the inherent network uncertainties, such as uncertain dynamics of heterogeneity and demand disturbance, MFD parameters could be time-varying and uncertain. Conversely, existing data-driven methods (e.g., reinforcement learning) do not consider the model-plant mismatch and the limited access to plant-generated data, for example, subregional OD-specific accumulations. Therefore, we develop an iterative adaptive dynamic programming (IADP)-based method to address the limited data source induced by the model-plant mismatch. An actor-critic neural network structure is developed to circumvent the requirement of complete information on plant dynamics. Performance comparisons with other PCRG schemes under various scenarios are carried out. The numerical results indicate that the IADP controller trained with a limited data source can achieve comparable performance with the "benchmark" MPC approach using perfect measurements from the plant. The results also validate the IADP's robustness against various uncertainties (e.g., demand noise, MFD error, and trip distance heterogeneity) when minimizing the total time spent in the urban network. These results demonstrate the great potential of the proposed scheme in improving the efficiency of multiregion MFD systems.