We show that constant-Fermi-level ab initio molecular dynamics can be used as a computer-based tool to reveal and control relevant defects in semiconductor materials. In this scheme, the Fermi level can be set at any position within the band gap during the defect generation process, in analogy to experimental growth conditions in the presence of extra electrons or holes. The scheme is illustrated in the case of GaAs, for which we generate melt-quenched amorphous structures through molecular dynamics at various Fermi levels. By a combined analysis which involves both the atomic structure and a Wannier-function decomposition of the electronic structure, we achieve a detailed description of the generated defects as a function of the Fermi level. This leads to the identification of As-As homopolar bonds and Ga dangling bonds for Fermi levels set in the vicinity of the valence band. These defects convert into As dangling bonds and Ga-Ga homopolar bonds, as the Fermi level moves toward the conduction band. This demonstrates a computer-aided procedure to identify semiconductor defects in an unbiased way.