In this paper, a new high-order multiscale finite element method is developed for elliptic problems with highly oscillating coefficients. The method is inspired by the multiscale finite element method developed in [3], but a more explicit multiscale finite element space is constructed. The approximation space is nonconforming when oversampling technique is used. We use a Petrov- Galerkin formulation suggested in [14] to simplify the implementation and to improve the accuracy. The method is natural for high-order finite element methods used with advantage to solve the coarse grained problem. We prove optimal error estimates in the case of periodically oscillating coefficients and support the findings by various numerical experiments.