Autonomous driving is a revolutionary technology that has seen considerable advancements through the adoption of deep learning solutions. One of the major challenges in this field is the interaction with other road users. This interaction necessitates a "trajectory forecasting" component, \textit{i.e.,} to forecast the future positions of these users in the vicinity of the autonomous vehicle.

There are two paradigms for trajectory forecasting models. The first one is knowledge-based models built upon the available domain-specific knowledge. These models, while generalizable to various environments, are limited in accuracy due to their lower capacity. The second paradigm is data-driven approaches, mostly in the form of deep learning-based models. These models have higher accuracy but often struggle to generalize to new, unseen environments. The first contribution of this thesis is to combine the two paradigms harnessing the strengths of both worlds. Through our research, we demonstrate that this hybrid approach yields superior generalization and accuracy compared to either knowledge-based or data-driven counterparts. Our experiments reveal that while the data-driven models' performance drops significantly in new environments, the existing evaluation pipelines are unable to demonstrate this shortcoming.

The thesis then continues by investigating evaluation methodologies for trajectory forecasting and contributes to enhancing the evaluation pipeline in three significant ways. First, we introduce a framework that integrates multiple datasets, enabling cross-dataset generalization evaluation. Our evaluation shows that models cannot generalize to other datasets. This framework also opens the door to various research inquiries, including examining how data scaling affects model performance and analyzing different datasets. We then explore novel fine-grained evaluations to analyze models in more detail. As the second evaluation enhancement, we propose a methodology that focuses on evaluating the social understanding of forecasting models by employing an adversarial attack approach. Our findings reveal that existing models have a limited social understanding. Our third evaluation approach is a methodology that assesses models' scene understanding capabilities based on atomic scene generation functions. It reveals that the state-of-the-art forecasting models are still inefficient in scene reasoning, leaving room for further improvements.

The final part of the thesis tackles a critical challenge in trajectory forecasting: dealing with imperfect perception systems. Errors in perception systems introduce noise into the inputs of trajectory forecasting models, leading to uncertainty in their error bounds. We provide certification methods for trajectory forecasting models which provide certified error bounds for the models given noisy input data.