Prediction is a vital component of motion planning for autonomous vehicles (AVs). By reasoning about the possible behavior of other target agents, the ego vehicle (EV) can navigate safely, efficiently, and politely. However, most of the existing work overlooks the interdependencies of the prediction and planning module, only connecting them in a sequential pipeline or underexploring the prediction results in the planning module. In this work, we propose a framework that integrates the prediction and planning module with three highlights. First, we propose an ego-conditioned model for causal prediction, with the introduced edge-featured graph transformer model, the impact the ego future maneuver poses to the target vehicles is demonstrated. Second, we develop a motion planner based on 'dynamic voxels' in the spatio-temporal domain, enabling the time-to-collision criterion evaluation and the optimal trajectory generation in continuous space. Third, the prediction and planning modules are coupled in a closed-loop and efficient form. Specifically, taking each maneuver as a cluster, representative trajectory primitives are generated for conditional prediction, and conversely, prediction results are used to score the primitives as guidance, which alleviates the duplicated callback of the prediction module. The simulations are conducted in overtaking, merging, unprotected left turns, and also scenarios with imperfect social behaviors. The comparison studies demonstrate the better safety assurance and efficiency of the proposed model, and the ablation experiments further reveal the effectiveness of the new ideas.