We report on the increase in optical quality factor of a suspended single crystal diamond micro-disk resonator by thermal annealing. The resonators are fabricated by Deep Reactive Ion Etching (DRIE) to obtain circular pillars in bulk single crystal diamond, followed by multidirectional Focused Ion Beam (FIB) milling to shape the anchor, thin the micro disk and smooth the sidewalls. Thermal annealing in air at 500 degrees C for 4 h eliminates FIB induced crystal damage yielding optical quality factors of up to 5'700, corresponding to an increase of up to 5x compared to the resonators before annealing.