System identification using multiple-model strategies may involve thousands of models with several parameters. However, only a few models are close to the correct model. A key task involves finding which parameters are important for explaining candidate models. The application of feature selection to system identification is studied in this paper. A new feature selection algorithm is proposed. It is based on the wrapper approach and combines two algorithms. The search is performed using stochastic sampling and the classification uses a support vector machine strategy. This approach is found to be better than genetic algorithm-based strategies for feature selection on several benchmark data sets. Applied to system identification, the algorithm supports subsequent decision making.