We report powder and single-crystal neutron diffractionmeasurements, combinedwith x-ray powder diffraction data, to unravel the complex magnetic phase diagram and exchange coupling in Co3TeO6. The magnetic structures of the various phases differ markedly from those proposed by Ivanov et al. [Mater. Res. Bull. 47, 63 (2012)] on the basis of only powder diffraction data. The dominant exchange interactions are identified by considering the geometrical arrangement of severely distorted CoO6 octahedra and CoO4 tetrahedra, which naturally divide into two different types of layers, one of which consists of zigzag chains. These zigzag chains are the first to develop magnetic order at T-M1 = 26 K, which is incommensurate in nature. The other separate layer of Co spins develops antiferromagnetic order of Gamma 4 symmetry at zero wave vector at T-M2 = 19.5 K. Our results are consistent with the previous findings of a spontaneous polarization below T-M3 = 18 K. Our neutron powder diffraction data indicate that the increase in the single-crystal (600) Bragg peak is due to a relief of extinction rather than to magnetic effects associated with the observed anomalous variation in the incommensurate wave vector at T-M4 = 16 K. The commensurate order parameter is shown to have a small dependence on the applied electric field, whereas no such effect is found for the incommensurate ordering. Below T-M3, the thermal expansion is negative, and it also exhibits anomalies at T-M2 and T-M4. A symmetry analysis and comprehensive phase diagram are given.