We report on small-angle neutron scattering studies of the intrinsic vortex lattice (VL) structure in detwinned YBa2Cu3O7 at 2 K, and in fields up to 10.8 T. Because of the suppressed pinning to twin-domain boundaries, a new distorted hexagonal VL structure phase is stabilized at intermediate fields. It is separated from a low-field hexagonal phase of different orientation and distortion by a first-order transition at 2.0(2) T that is probably driven by Fermi surface effects. We argue that another first-order transition at 6.7(2) T, into a rhombic structure with a distortion of opposite sign, marks a crossover from a regime where Fermi surface anisotropy is dominant, to one where the VL structure and distortion is controlled by the order-parameter anisotropy.