For classification problems, it is important that the classifier is trained with data which is likely to appear in the future. Discriminative models, because of their nature to focus on the boundary between classes rather than data itself, usually do not have the capability to deal with noisy training data. We propose the use of generative models as filters to make discriminative models more robust against noise. Firstly the distribution of the training data is estimated, then examples which do not satisfy some criterion, like having low likelihood, will be considered as outliers and discarded before training discriminative models. The idea was tested on a noisy data set from the UCI Machine Learning Repository.