In our previous work, tracking the iso-level sets through total variation scale-space proved to be a very efficient tool for unsupervised segmentation. Stepping on these results, we propose a new segmentation approach in a unified total variation framework. The main idea is to use the total variation energy at each scale to drive the region merging process. We show that this total variation formulation, which was originally proposed for restoration and enhancement, is also well suited for segmentation. In addition, this energy functional can be derived from a Bayesian principle using a Markov random field prior. We demonstrate the effectiveness of our method on gray scale, noisy, color and texture images.