Traffic simulation models are frequently used to support decisions when planning an evacuation. These models typically focus on traffic dynamics and the effect of traffic control measures in order to locate possible bottlenecks and predict evacuation times. A clear view on the crucial factors that determine the evacuation time and emergent traffic states is however lacking. In this paper, the authors identify and quantify the impact of variations in both travel demand and network supply for the case of evacuation by performing a structured and comprehensive sensitivity analysis. The sensitivity analysis is done by applying the macroscopic evacuation traffic simulation model EVAQ in which the authors systematically vary aspects such as trip generation, departure rates, route flow rates, road capacities, and maximum speeds. The authors do so using a case study describing the evacuation of the Rotterdam metropolitan area, a city in the Netherlands. The authors conclude that departure rates and route flow rates have a substantial non-linear impact on the network conditions and arrival pattern, in particular when the network load is relatively high, while the trip generation and road capacities have a smaller quasi-linear impact. Maximum speeds, independent of the effect hereof on road capacities, have no significant impact on the evacuation. The results, discussion and conclusions presented here can be used to identify the most important factors in i) verifying, calibrating, and validating an evacuation model, ii) designing a network for evacuation studies, and iii) evaluating and testing robustness of evacuation plans.