In recent years, multi-atlas fusion methods have gained significant attention in medical image segmentation. In this paper, we propose a general Markov Random Field (MRF) based framework that can perform edge-preserving smoothing of the labels at the time of fusing the labels itself. More specifically, we formulate the label fusion problem with MRF-based neighborhood priors, as an energy minimization problem containing a unary data term and a pairwise smoothness term. We present how the existing fusion methods like majority voting, global weighted voting and local weighted voting methods can be reframed to profit from the proposed framework, for generating more accurate segmentations as well as more contiguous segmentations by getting rid of holes and islands. The proposed framework is evaluated for segmenting lymph nodes in 3D head and neck CT images. A comparison of various fusion algorithms is also presented.