This thesis presents an algorithm for face tracking in video sequences. We investigate the application of affine invariant, local features for face tracking under random poses and expressions. In order to capture as much as possible of the facial variability, a combination of region detectors is used to extract the various facial points of interest. Pairwise matching of SIFT descriptors for those regions is used to identify possible similarity transformations between consecutive frames. If the matching process does not provide satisfying candidates, various translation parameters are used to determine the set of possible candidates. The similariy transformations are finally ranked according to their compatibility with the color and orientation descriptors of the previous template. The candidate with the best score is chosen as the new template. We have applied the above method in a small data set of video sequences and found it to work well under various settings and conditions.