We investigate the dynamic behavior of concrete in relation to its composition within a computational framework (FEM). Concrete is modeled using a meso-mechanical approach in which aggregates and mortar are represented explicitly. Both continuum phases are considered to behave elastically, while nucleation, coalescence and propagation of cracks are modeled using the cohesive-element approach. In order to understand the loading-rate sensitivity of concrete, we simulate direct tensile-tests for strain rates ranging 1–1000 s−1. We investigate the influence of aggregate properties (internal ordering, size distribution and toughness) on peak strength and dissipated fracture energy. We show that a rate independent constitutive law captures the general increase of peak strength with strain rate. However, a phenomenological rate-dependent cohesive law is needed to obtain a better agreement with experiments. Furthermore, at low rates, peak strength is sensitive to the inclusions' toughness, while the matrix dominates the mechanical behavior at high rates.