We consider a multiuser OFDM system in which users want to transmit videos via a base station. The base station knows the channel state information (CSI) as well as the rate distortion (RD) information of the video streams and tries to allocate power and spectrum resources to the users according to both physical layer CSI and application layer RD information. We derive and analyze a condition for the optimal resource allocation solution in a continuous frequency response setting. The optimality condition for this cross layer optimization scenario is similar to the equal slope condition for conventional video multiplexing resource allocation. Based on our analysis, we design an iterative subcarrier assignment and power allocation algorithm for an uplink system, and provide numerical performance analysis with different numbers of users. Comparing to systems with either only physical layer or only application layer information available at the base station, our results show that the user capacity and the video PSNR performance can be increased significantly by using cross layer design. Bit-level simulations which take into account the imperfection of the video coding rate control, the variation of RD curve fitting, as well as channel errors, are presented.