In recent years, several studies have been published about the smart definition of training set using active learning algorithms. However, none of these works consider the contradiction between the active learning methods, which rank the pixels according to their uncertainty, and the confidence of the user in labeling, which is related both to the homogeneity of the pixel context and to the knowledge of the user of the scene. In this paper, we propose a two-steps procedure based on a filtering scheme to learn the confidence of the user in labeling. This way, candidate training pixels are ranked according both to their uncertainty and to the chances of being labeled correctly by the user. In this way, we avoid the queries where the user would not be able to provide a class for the pixel. We consider the capacity of a model in learning the user's confidence and report experiments on a QuickBird image: the filtering scheme proposed maximizes the number of useful queries with respect to traditional active learning.