A new region-based depth ordering algorithm is proposed based on the segmented motion layers with affine motion models. Starting from an initial set of layers that are independently extracted for each frame of an input sequence, relative depth order of every layer is determined following a bottom-to-top approach from local pair-wise relations to a global ordering. Layer sets of consecutive time instants are warped in two opposite directions in time to capture pair-wise occlusion relations of neighboring layers in the form of pixel voting statistics. Global depth order of layers is estimated by mapping the pair-wise relations to a directed acyclic graph and solving the longest path problem via a breadth-first search strategy. Temporal continuity is enforced both at the region segmentation and depth ordering stages to achieve temporally coherent layer support maps and depth order relations. Experimental results show that the proposed algorithm yields quite promising results even on dynamic scenes with multiple motions.