The crystal structure and the magnetic ordering pattern of the electrically insulating perovskite CeVO3 was investigated by high-resolution powder X-ray diffraction and single-crystal neutron diffraction. A structural phase transition from an orthorhombic to a monoclinic structure (with space groups Pbnm and P21/b, respectively) was observed upon cooling below T s = 136 K. This transition is associated with a strong distortion of the VO6-octahedra and can be attributed to orbital ordering. A magnetic ordering transition driven by exchange interactions between vanadium moments is observed at T N = 124 K, and antiferromagnetic interactions between magnetic moments on vanadium and cerium ions induce a progressive magnetic polarization of the cerium sublattice at lower temperatures. The full magnetic structure is described by a superposition of the modes (C x , F y , −) and (F x , C y , −). The unit cell volume and the tilt angles of the VO6-octahedra in the CeVO3-crystal structure are anomalous compared to those of other members of the series RVO3 (R = lanthanide atom), and the ordered magnetic moments on both vanadium and cerium sublattices at low temperatures are considerably smaller than the free-ion values of V3+ and Ce3+. Possible origins of this behavior are discussed.