In and Ga impurities substitutional to Al in the oxide layer resulting from diffusion out of the substrate are identified as candidates for electron traps under inversion at In0.53Ga0.47As/Al2O3 interfaces. Through density-functional calculations, these defects are found to be thermodynamically stable in amorphous Al2O3 and to be able to capture two electrons in a dangling bond upon breaking bonds with neighboring O atoms. Through a band alignment based on hybrid functional calculations, it is inferred that the corresponding defect levels lie at similar to 1 eV above the conduction band minimum of In0.53Ga0.47As, in agreement with measured defect densities. These results support the technological importance of avoiding cation diffusion into the oxide layer. Published by AIP Publishing.