The single-atom bit represents the ultimate limit of the classical approach to high-density magnetic storage media. So far, the smallest individually addressable bistable magnetic bits have consisted of 3-12 atoms(1-3). Long magnetic relaxation times have been demonstrated for single lanthanide atoms in molecular magnets(4-12), for lanthanides diluted in bulk crystals(13), and recently for ensembles of holmium (Ho) atoms supported on magnesium oxide (MgO)(14). These experiments suggest a path towards data storage at the atomic limit, but the way in which individual magnetic centres are accessed remains unclear. Here we demonstrate the reading and writing of the magnetism of individual Ho atoms on MgO, and show that they independently retain their magnetic information over many hours. We read the Ho states using tunnel magnetoresistance(15,16) and write the states with current pulses using a scanning tunnelling microscope. The magnetic origin of the long-lived states is confirmed by single-atom electron spin resonance(17) on a nearby iron sensor atom, which also shows that Ho has a large out-of-plane moment of 10.1 +/- 0.1 Bohr magnetons on this surface. To demonstrate independent reading and writing, we built an atomic-scale structure with two Ho bits, to which we write the four possible states and which we read out both magnetoresistively and remotely by electron spin resonance. The high magnetic stability combined with electrical reading and writing shows that single-atom magnetic memory is indeed possible.