Objective: Time-domain diffuse optical imaging (DOI) requires accurate forward models for photon propagation in scattering media. However, existing simulators lack comprehensive experimental validation, especially for non-contact configurations with oblique illumination. This study rigorously evaluates three widely used open-source simulators, including MMC (Mesh-based Monte Carlo), NIRFASTer, and Toast++ (the latter two are finite-element method (FEM)-based), using time-resolved experimental data. Approach: All simulations employed a unified mesh and point-source illumination. Virtual source approximation was applied to FEM solvers to reconcile the directional oblique beam with the isotropic diffusion equation. A time-resolved DOI system with a 32 × 32 single-photon avalanche diode (SPAD) array acquired transmission-mode data from 16 standardized phantoms certified under the BIP protocol with varying absorption coefficient μ_a and reduced scattering coefficient μ_s'. The simulation results were quantified across five metrics: spatial-domain (SD) accuracy, time-domain (TD) accuracy, oblique beam accuracy, computational speed, and mesh-density independence. To further quantify the simulators’ practical applicability, optical property recovery was conducted using a grid-search optimization. Results: Among the three simulators, MMC achieves superior accuracy in SD and TD metrics (SD MSE: 0.072 ± 0.053; TD MSE: 0.179 ± 0.080), and shows robustness across all optical properties. NIRFASTer and Toast++ demonstrate comparable overall performance. In general, MMC is optimal for accuracy-critical TD-DOI applications, while NIRFASTer and Toast++ suit scenarios prioritizing speed (e.g., image reconstruction) with sufficiently large μ_s'. Besides, virtual source approximation is essential for non-contact FEM modeling, which reduced average errors by > 34% in large-angle scenarios. Significance: This work suggests a simplified framework for balancing simulation fidelity and computational speed through optimized solver selection and configuration, facilitating fast prototyping of novel TD-DOI systems. Our work represents the first study to systematically validate TD simulators against SPAD array-based data under clinically relevant non-contact conditions, bridging a critical gap in biomedical optical simulation standards.