The urban context is often simplified or neglected in Building Energy Models (BEMs) due to the difficulties of taking accurately into account all the heat fluxes emanating from the environment. Oversimplifying the urban context can impact the accuracy of the BEM predictions. Nevertheless, several approaches can be used to allow for the impact of the urban environment on the dynamic behavior of a building, its heating and cooling demands, and thermal comfort. This state of the art review provides a critical overview of the different methods currently used to take into account the urban microclimate in building design simulations. First, both the microclimate and building models are presented, focusing on their assumptions and capabilities. Second, a few examples of coupling, performed between both modeling scales are analyzed. Last, the discussion highlights the differences obtained between simulations that take the urban context into consideration and those that simplify or neglect urban heat fluxes. The remaining scientific obstacles to a more effective consideration of the urban context impacting the BEMs are indicated.