The combination of several interesting characteristics makes metal-organic frameworks (MOFs) a highly sought-after class of nanomaterials for a broad range of applications like gas storage and separation, catalysis, drug delivery, and so on. However, the ever-expanding and nearly infinite chemical space of MOFs makes it extremely challenging to identify the most optimal materials for a given application. In this work, we present a novel approach using deep reinforcement learning for the inverse design of MOFs, our motivation being designing promising materials for the important environmental application of direct air capture of CO2 (DAC). We demonstrate that our reinforcement learning framework can successfully design MOFs with critical characteristics important for DAC. The reinforcement learning framework uniquely integrates two separate predictive models within its structure, uncovering two distinct subspaces in the MOF chemical space: one with high CO2 heat of adsorption and the other with preferential adsorption of CO2 from humid air, with few structures having both characteristics. Our model can thus serve as an essential tool for the rational design and discovery of materials for different target properties and applications.