Statistical pattern recognition occupies a central place in the general context of machine learning techniques, as it provides the theoretical insights and the practical means for solving a variety of problems ranging from character recognition to face recognition, from malignant nodules classification to speaker identification, to name just a few domains of application. In this very general framework, the work presented in this thesis focuses on two theoretical aspects – feature extraction and pattern classification – and a more practical one – human face class modeling. We first propose a new framework for using the autocorrelation-based features for pattern classification. In this context, we extend their use to higher orders by exploiting a property of the inner product of two autocorrelation feature vectors. As a large number of classification algorithms may be cast into a form in which data is involved only by means of inner products between vectors, this property provides a very convenient way of avoiding explicitly computing the feature vectors – an operation which is normally very costly. Using this approach, we show that, practically, there is no upper limit for the order of autocorrelation functions used – rather this limit depends on the application. By defining different topologies over which the autocorrelation feature vectors are applied, we obtain extended feature vectors, for which the inner product property still holds. We illustrate the use of autocorrelation feature vectors by first showing how one can perform Principal Component Analysis in the autocorrelation space and we apply it to the classification of unidimensional signals. Another application presented uses Support Vector Machines (SVM) for detecting human faces described in terms of extended autocorrelation feature vectors. Next, we develop a family of algorithms that generate sparse representations of the discriminant functions. This family is based on a recently proposed algorithm, the Kernel Matching Pursuit [176] (KMP), which is considerably extended and enhanced. Using as a starting point the theory of n-term greedy function approximation, we justify the introduction of different versions of KMP. Firstly, we introduce the Weak KMP which uses a learning rate-like parameter for controling the importance of the decisions taken at each iteration. We propose then another version – Stochastic KMP – designed to make the application of KMP to large datasets feasible. Finally, two approaches for building adaptive versions of KMP are presented and compared. All these algorithms are tested on various standard datasets and their performance is compared to that of SVM. On the application side we discuss various aspects of human face detection. In this context we introduce a new method for estimating the performance of the face detection and localization algorithms which is designed to make the results comparable in an objective way. At the core of the technique is the precise definition of what a good detection or localization means. Using our definition we set the performance criteria and the evaluation method. As an example, we apply this technique to a standard detector and we discuss its performance. Finally, we present an approach to human face class modeling using SVM trained in the eigenfaces space. We discuss different issues that arise in building the eigenfaces space and training the SVM and we propose a new method for building a composite classifier based on SVMs which has various advantages over using a single SVM. We compare then the results of applying two face detectors (a classical one based on SVM trained in the pixel space and one using an SVM trained in the eigenfaces space) on a large dataset using the technique described before and we show that the new proposed method is superior.