In this paper, we show that tracking multiple people whose paths may intersect can be formulated as a multi-commodity network flow problem. Our proposed framework is designed to exploit image appearance cues to prevent identity switches. Our method is effective even when such cues are only available at distant time intervals. This is unlike many current approaches that depend on appearance being exploitable from frame to frame. Furthermore, our algorithm lends itself to a real-time implementation. We validate our approach on three publicly available datasets; APIDIS basketball dataset, ISSIA soccer dataset and the PETS’09 pedestrian dataset, all contain long and complex sequences. In addition, we evaluate the approach on a new basketball dataset, consists of full world championship basketball matches. In all cases, our approach preserves identity better then the state-of-the-art tracking algorithms.