This paper is a joint effort between five institutions that introduces several novel similarity measures and combines them to carry out a multimodal segmentation evaluation. The new similarity measures proposed are based on the location and the intensity values of the misclassified voxels as well as on the connectivity and the boundaries of the segmented data. We show experimentally that the combination of these measures improve the quality of the evaluation. The study that we show here has been carried out using four different segmentation methods from four different labs applied to a MRI simulated dataset of the brain. We claim that our new measures improve the robustness of the evaluation and provides better understanding about the difference between segmentation methods.