We demonstrate engineering of carrier states in the conduction band (CB) and valence band (VB) of site-controlled InGaAs/GaAs quantum dots (QDs) grown into pyramidal recesses, by controlling their shape, size, and composition. QDs with CB level separation ranging from ∼15 to 70 meV are obtained, useful in applications based on intraband transitions, e.g., QD photodetectors and QD cascade lasers. Moreover, by varying the aspect ratio and composition of the QDs we are able to switch the polarization of the dominant interband transition, a feature of interest for producing single photon emitters and QD amplifiers with prescribed polarization states