Atherosclerosis is a complex disease altering vasculature morphology, and subsequently flow, with progressive plaque formation, mural disruption, and lumen occlusion. Determination of clinically-relevant plaque components-particularly calcium, lipid, and fibrous tissue-has driven automated image-based tissue characterization. Atherosclerotic tissue of mixed composition type arises when these principal components interdigitate and combine during the course of progressive atherosclerosis. Nevertheless, such mixed plaque is treated non uniformly, and often neglected, as a distinct class in image analysis. We therefore quantitatively investigate frameworks to characterize mixed and other plaque tissue types, and examine their implications. Convolutional neural networks operated on labeled intravascular optical coherence tomography images using various characterization frameworks. The treatment of mixed plaque by image-based classifiers influenced the accuracy and homogeneity of the segmented classes. Excluding mixed plaque as a class on to itself necessarily assigns heterogeneous lesion subcomponents to one of the three homogeneous subtypes; when included, 61.7% of mixed tissue is labeled as calcium, reducing specificity in homogeneous calcium detection by 34.8%. Segmenting mixed plaque as distinct from homogeneous, non-mixed tissue improves lesion classification. This can be achieved either on the basis of homogeneous tissue classifier prediction uncertainty (77.8% overall accuracy) or by training classifiers to identify mixed plaque as a discrete tissue class (82.9% overall accuracy). Alternatively, mixed plaque can be grouped with one of the homogeneous classes, yielding a single histologically diverse class that helps preserve the homogeneity of the others. Ultimately, the best approach depends upon the alignment of histological and functional distinctions. While no vascular lesion characterization framework or method is universally optimal or appropriate, context should remain central in selecting tissue characterization techniques.