Attentional modulation is needed when a decision maker is faced with too much information to process. In this paper we discuss the use of attention in control applications where usually higher order system dynamics are involved as typical benchmarks considered in cognitive experiments. Previous works have implemented various forms of attention and active perception for selecting the necessary data, reducing the dimensionality of the input space, tuning the sensors, etc. However, decision making can depend not only on objective sensory data, but also upon mental states or subjective inputs. Specifically, we examine attention for the evaluation of selected action by various critics, as well as sensory inputs. The complexity of each situation and consequent variety of evaluation indices or local goals, which could be compatible or contrary to each other, can confuse the agent. Therefore, the agent needs to attend to the dependable critics which can be altered regarding the state. The attention to different critics or goals can be established by forming different architecture in the agent’s mind, which can assess the domination of a critic or an aggregation of them. Also, the ascendancy of critics can be judged by the rate of failure. In other words, the agent who feels unsatisfied while being unsuccessful to satisfy the particular goal, attends to that goal. In cases where there are no adequate knowledge about the ascendancy of different critics, and only the desired behavior is known, the attention can be learned by feedbacks of a super critic which evaluates the agent’s behavior. For instance, in real world applications human experts can play the role of super critic.