This paper presents a framework for analyzing probabilistic safety and reachability problems for discrete time stochastic hybrid systems in scenarios where system dynamics are affected by rational competing agents. In particular, we consider a zero-sum game formulation of the probabilistic reach-avoid problem, in which the control objective is to maximize the probability of reaching a desired subset of the hybrid state space, while avoiding an unsafe set, subject to the worst-case behavior of a rational adversary. Theoretical results are provided on a dynamic programming algorithm for computing the maximal reach-avoid probability under the worst-case adversary strategy, as well as the existence of a max-min control policy which achieves this probability. The modeling framework and computational algorithm are demonstrated using an example derived from a robust motion planning application. © 2011 IEEE.