In this paper, we propose an original framework for an intuitive tuning of parameters in image and video segmentation algorithms. The proposed framework is very flexible and generic and does not depend on a specific segmentation algorithm, a particular evaluation metric, or a specific optimization approach, which are the three main components of its block diagram. This framework requires a manual segmentation input provided by a human operator as he/she would have performed intuitively. This input allows the framework to search for the optimal set of parameters which will provide results similar to those obtained by manual segmentation. On one hand, this allows researchers and designers to quickly and automatically find the best parameters in the segmentation algorithms they have developed. It helps them to better understand the degree of importance of each parameter's value on the final segmentation result. It also identifies the potential of the segmentation algorithm under study in terms of best possible performance level. On the other hand, users and operators of systems with segmentation components, can efficiently identify the optimal sets of parameters for different classes of images or video sequences. In a large extent, this optimization can be performed without a deep understanding of the underlying algorithm, which would facilitate the exploitations and optimizations in real applications by non-experts in segmentation. A specific implementation of the proposed framework was obtained by adopting a video segmentation algorithm invariant to shadows as segmentation component, a full reference segmentation quality metric based on a perceptually motivated spatial context, as the evaluation component, and a down-hill simplex method, as optimization component. Simulation results on various test sequences, covering a representative set of indoor and ourdoor video, show that optimal set of parameters can be obtained efficiently and largely improve the results obtained when compared to a simple implementation of the same segmentation algorithm with ad-hoc parameter setting strategy.