We report high-field magnetization, high-frequency electron spin resonance (ESR), and Se-77 nuclear magnetic resonance (NMR) measurements on the linear spin tetramer system CuSeO3, consisting of strongly interacting Cu(1) dimers and weakly coupled Cu(2) spins. The magnetization exhibits anisotropic half-step magnetization plateaus at mu H-0 = 45 T, depending on a crystallographic orientation. A temperature dependence of the ESR linewidth Delta H-pp in a paramagnetic phase points towards the significance of anisotropic exchange interactions. Below T-N = 9-10 K long-range magnetic order is evidenced by the observation of a critical divergence of both Delta H-pp(T) and the nuclear spin-lattice relaxation rate 1/T-1. In addition, we identify a magnetic anomaly at T* = 6.0(5) K below T-N, which is caused by a spin reorientation. The nuclear spin-spin relaxation rate 1/T-2 unveils the development of site-specific spin correlations. The intriguing magnetism of CuSeO3 is discussed in terms of the energy hierarchy of Cu(1) and Cu(2) spins in concert with additional intertetramer interactions.