This article investigates the optimal containment control problem for a class of heterogeneous multi-agent systems with time-varying actuator faults and unmatched disturbances based on adaptive dynamic programming. Since there exist unknown input signals in each leader, distributed observers are utilized to estimate trajectories in the convex hull spanned by leaders. The containment control problem is then transformed into an optimal tracking problem. To compensate for the actuator faults and unmatched disturbances, a novel performance index function is designed. We prove that the optimal control policy can ensure that the tracking error system of each follower is uniformly ultimately bounded. The online policy iteration algorithm is implemented using critic neural networks to obtain the optimal control policy. A numerical example is provided to demonstrate the effectiveness of the proposed control policy.