In this paper we consider the problem of simultaneously localizing all members of a team of robots. Each robot is equipped with proprioceptive sensors and exteroceptive sensors. The latter provide relative observations between the robots. Proprioceptive and exteroceptive data are fused with an Extended Kalman Filter. We derive the equations for this estimator for the most general relative observation between two robots. Then we consider three special cases of relative observations and we present the structure of the filter for each case. Finally, we study the performance of the approach through many accurate simulations.