Magnetization, neutron diffraction, and antiferromagnetic resonance measurements were employed to investigate the magnetic ground state of Ni-5(TeO3)(4)Br-2 single crystal. Despite the layered topology of the Ni2+ sites, typical for frustrated antiferromagnets, the system orders antiferromagnetically below T-N=29 K. Noncollinear arrangements of the Ni sublattices having a very complex temperature dependence were found from the neutron diffraction and magnetization measurements. Single-ion anisotropy, associated with the strongly distorted Ni-centered octahedra, has the same magnitude as the antiferromagnetic exchange interaction. The effect of these anisotropies prevails over the geometrical frustration leading to a long-range magnetic ordering below T-N.