The steady advance of computational methods makes model-based optimization an increasingly attractive method for process improvement. Unfortunately, the available models are often inaccurate. An iterative optimization method called "modifier adaptation" overcomes this obstacle by incorporating process information into the optimization framework. This paper extends this technique to constrained optimization problems, where the plant consists of a closed-loop system but only a model of the open-loop system is available. The degrees of freedom of the closed-loop system are the setpoints provided to the controller, whereas the model degrees of freedom are the inputs of the open-loop plant. Using this open-loop model and process measurements, the proposed algorithm guarantees both optimality and constraint satisfaction for the closed-loop system upon convergence. A simulated CSTR example with constraints illustrates the method.