An algorithm for feature point tracking is proposed. The Interacting Multiple Model (IMM) filter is used to estimate the state of a feature point. The problem of data association, i.e. establishing which feature point to use in the state estimator, is solved by an assignment algorithm. A track management method is also developed. In particular a track continuation method is presented. The evaluation of the tracking system on real sequences shows that the IMM filter combined with the assignment algorithm outperforms the Kalman filter, used with the Nearest Neighbor (NN) filter, in terms of data association performance and robustness to sudden feature point maneuvers.