This paper considers a single source transmitting data to one or more receivers/users over a shared wireless channel. Due to random fading, the wireless channel conditions vary with time and from user to user. Each user has a buffer to store received packets before they are drained. At each time step, the source determines how much power to use for transmission to each user. The source's objective is to dynamically allocate power in a manner that minimizes total power consumption and packet holding costs, while satisfying strict buffer underflow constraints and a joint power constraint in each slot. The primary application motivating this problem is wireless media streaming. For this application, the buffer underflow constraints prevent the user buffers from emptying, so as to maintain playout quality. In the case of a single user, a state-dependent modified base-stock policy is shown to be optimal with linear power-rate curves, and a state-dependent finite generalized base-stock policy is shown to be optimal with piecewise-linear convex power-rate curves. When certain technical conditions are satisfied, efficient methods to compute the critical numbers that complete the characterizations of the optimal control laws in each of these cases are presented. The structure of the optimal policy for the case of two users is then analyzed.