In this chapter, we introduce a method for trajectory pattern analysis through the probabilistic inference model with both regional and velocity observations. By embedding Gaussian models into the discrete topic model framework, our method uses continuous velocity as well as regional observations unlike the existing approaches. In addition, the proposed framework combined with Hidden Markov Model can cover the temporal transition of the scene state, which is useful in checking violation of the rule that some conflict topics (e.g., two cross traffic patterns) should not occur at the same time. To achieve online learning even with the complexity of the proposed model, we suggest a novel learning scheme instead of collapsed Gibbs sampling. The proposed two-stage greedy learning scheme is not only efficient at reducing the search space but also accurate in a way that the accuracy of online learning becomes not worse than that of the batch learning. To validate the performance of our method, experiments were conducted on various datasets. Experimental results show that our model explains satisfactorily the trajectory patterns with respect to scene understanding, anomaly detection, and prediction.