We introduce a modeling approach for pedestrian speed-density relationship. It is motivated by a high scatter in real data that precludes the use of traditional equilibrium relationships. To characterize the observed pattern we relax the homogeneity assumption of equilibrium relations and propose a multi-class model. In addition to the general modeling framework, we also present some concrete model specifications. Real data is utilized to test the performance of the approach. The approach is able to reveal fundamental properties causing the heterogeneity in population and describe their impact on pedestrian movement. We also show the advantages of the proposed approach compared to approaches from the literature. The proposed model is flexible, and it provides better fit and richer information than traditional models.