We have stabilized an external cavity diode laser to a whispering gallery mode resonator formed by a protrusion of a single-crystal magnesium fluoride cylinder. The cylinder's compact dimensions (less than or similar to 1 cm(3)) reduce the sensitivity to vibrations and simplify the stabilization of its temperature in a compact setup. In a comparison to an ultrastable laser used for precision metrology we determine a minimum Allan deviation of 20Hz at an optical wavelength of 972 nm, corresponding to a relative Allan deviation of 6 x 10(-14), at an integration time of 100 ms. This level of instability is compatible with the limits imposed by fundamental fluctuations of the material's refractive index at room temperature.