The rise of high-throughput first-principles calculations of materials since the turn of the millennium has gifted the fields of physics, quantum chemistry, and materials science with a mountainous and ever-growing pile of data ready to be mined for hidden gems of scientific insight and discovery. For the last decade and a half, researchers have been hard at work designing machine learning (ML) methods to both sift through these data and leverage them to further accelerate the determination of quantum-mechanically accurate properties. For some tasks, like predicting the potential energy between atoms used to integrate Newton's equations of motion, efficient and accurate ML methods are already well-established as indispensable tools for understanding the macroscopic properties of materials at a level of accuracy that was previously prohibitively expensive. A key factor in the rapid success of these approaches has been the wealth of detailed knowledge of the physical laws governing the interactions of electrons and atoms which serve to constrain and guide the development of ML in the field. However, the majority of these approaches gloss over the key players that actually make molecules and materials what they are: the electrons. In this thesis, we aim to investigate the construction and use of ML methods which prioritize the electronic structure of materials, beginning by exploiting the fundamentally quantum nature of electronic matter and the associated tools of electronic structure theory to formulate representations of materials. Many complex material properties, like their interactions with light and transport of electricity, cannot be readily explained by the geometry of their constituent atoms and are therefore difficult to predict from the standard atomic descriptions used for ML. We show that by using electronic-structure based descriptions and simple ML models, the quality of data-driven predictions targeting first-principles calculations of these properties can be improved. There are also certain electronic behaviors which are not well-described by standard first-principles simulation approaches. In particular, standard approximate density-functional theory (DFT) models fail to reproduce the experimentally measured properties of some materials containing transition-metal elements due to a poor treatment of localized d and f electron orbitals. Corrective approaches, like DFT + Hubbard, can in many cases restore the expected behavior, but they require expensive first-principles calculations to determine the parameters controlling the strength of the corrections. In the second part of the thesis, we show that by using electronic-structure based equivariant neural network models, these Hubbard parameters can be accurately predicted orders of magnitude more quickly than with physical calculations. We also present ongoing efforts to extend this approach towards a universal model which could significantly facilitate the use of first-principles DFT + Hubbard more broadly. Finally, in the last part of the thesis, we describe preliminary work towards the prediction of electronic Hamiltonians, fundamental physical quantities describing the behavior of electrons, represented on Wannier function bases. This approach takes advantage of the locality of the Wannier functions to learn, in principle, their well-converged and compact Hamiltonians, providing indirect access to many relevant complex electronic properties.