Anomalies and changes in sensor networks which are deployed for activity recognition may abate the classification performance. Detection of anomalies followed by compensatory reaction would ameliorate the performance. This paper introduces a novel approach to detect the faulty or degraded sensors in a multi-sensory environment and a way to compensate it. The approach considers the distance between each classifier output and the fusion output to decide whether a sensor (classifier) is degraded or not. Evaluation is done on two activity datasets with different configuration of sensors and different types of noise. The results show that using the method improves the classification accuracy.