A generic approach is presented to detect and track people with a network of fixed and omnidirectional cameras given severely degraded foreground silhouettes. The problem is formulated as a sparsity constrained inverse problem. A dictionary made of atoms representing the presence of a person at a given location is used within the problem formulation. A re- weighted scheme is considered to better approximate the sparsity prior. Although the framework is generic to any scene, the focus of this paper is to evaluate the strength of the proposed approach on a basketball game. The main challenges come from the players' behavior, their similar appearance, and the mutual occlusions present in the views. In addition, the extracted foreground silhouettes are severely degraded due to the polished floor reflecting the players, and the strong shadow present in the scene. We present qualitative and quantitative results with the APIDIS dataset as part of the ICDSC sport challenge.