In this paper, we improve upon earlier approaches to segmenting mitochondria in Electron Microscopy images by explicitly modeling the double membrane that encloses mitochondria, as well as using features that capture context over an extended neighborhood. We demonstrate that this results in both improved classification accuracy and reduced computational requirements for training.