We propose a fast splitting approach to the classical variational formulation of the image partitioning problem, which is frequently referred to as the Potts or piecewise constant Mumford-Shah model. For vector-valued images, our approach is significantly faster than the methods based on graph cuts and convex relaxations of the Potts model which are presently the state-of-the-art. The computational costs of our algorithm only grow linearly with the dimension of the data space which contrasts the exponential growth of the state-of-the-art methods. This allows us to process images with high-dimensional codomains such as multispectral images. Our approach produces results of a quality comparable with that of graph cuts and the convex relaxation strategies, and we do not need an a priori discretization of the label space. Furthermore, the number of partitions has almost no influence on the computational costs, which makes our algorithm also suitable for the reconstruction of piecewise constant (color or vectorial) images.