This paper presents an automated test bench, based on a rigorous measurement procedure, used to fully characterize vibration energy harvesters including determination of the resonance frequency, impedance, optimal load, and output power as a function of both frequency and acceleration. The potential of this method and the performance of the automated test bench allows systematic data acquisition which is essential for a good comparison of all harvesters. A dedicated automation circuit was designed and fabricated. It uses stepper motors to mechanically control trimmers to vary the resistive load and reed relays to switch between the measurement sequences. With this, the setup is able to determine the optimal load of the device-under-test at its resonant frequency for a given acceleration. The test bench was used to fully characterize several types of vibration harvesters fabricated on both silicon and polymeric substrates. A comparison of the characterized devices is discussed using a figure of merit proposed here. A survey and compilation of current practices used to benchmark vibration harvesters is also reported.