In digital holographic microscopy (DHM), the long coherence length of laser light causes parasitic interferences due to multiple reflections in and by optical components in the optical path of the microscope and thus degrades the image quality. The parasitic effects are greatly reduced by using a short coherence length light. The main drawback of using a short coherence light source in an off-axis digital holographic microscope, is the reduction of the interference fringe contrast occurring in the field of view. Previously, we introduced a volume diffractive optical element (VDOE) placed in the reference arm of a DHM to correct the coherence plane tilt so as to obtain a high interference contrast throughout the field of view. Here, we experimentally quantify the spatial and temporal phase noise in the extracted phase image caused by non-homogeneities and scattering of the VDOE element itself. The results over five VDOEs show that the temporal phase noise is unchanged and a slight increase (up to 20%) is observed in the spatial phase noise. These results show that even with a low coherence source, a full field of view can be obtained with an off-axis DHM thanks to the VDOE without introducing significant additional phase noise.