In the present paper, a multi-step reconstruction procedure is proposed for high order finite volume schemes on unstructured grids using compact stencil. The procedure is a recursive algorithm that can eventually provide sufficient relations for high order reconstruction in a multi-step procedure. Two key elements of this procedure are the partial inversion technique and the continuation technique. The partial inversion can be used not only to obtain lower order reconstruction based on existing reconstruction relations, but also to regularize the existing reconstruction relations to provide new relations for higher order reconstructions. The continuation technique is to extend the regularized relations on the face-neighboring cells to current cell as additional reconstruction relations. This multi-step procedure is operationally compact since in each step only the relations defined on a compact stencil are used. In the present paper, the third and fourth order finite volume schemes based on two-step quadratic and three-step cubic reconstructions are studied. (C) 2019 Elsevier Inc. All rights reserved.