We present a semi-automatic approach to solve occlu- sion in AR systems. Once the occluding objects have been segmented by hand in selected views called key-frames, the occluding boundary is computed automatically in the in- termediate views. To do that, the 3D reconstruction of the occluding boundary is achieved from the outlined silhou- ettes. This allows us to recover a good prediction of the 2D occluding boundary which is refined using region-based tracking and active contour models. As a result, we get an accurate estimation of the occluding objects. Various results are presented demonstrating occlusion resolution on real video sequences. Results and videos are available at the URL: http://cvlab.epfl.ch/~lepetit