We investigate the Rabi oscillation of an atom placed inside a quantum cavity where each mirror is formed by a chain of atoms trapped near a one-dimensional waveguide. This proposal was studied previously with the use of Markov approximation, where the delay due to the finite travel time of light between the two cavity mirrors is neglected. We show that Rabi oscillation analogous to that obtained with high-finesse classical cavities is achieved only when this travel time is much larger than the time scale that characterizes the superradiant response of the mirrors. Therefore, the delay must be taken into account and the dynamics of the problem is inherently non-Markovian. Parameters of interest such as the Rabi frequency and the cavity loss rate due to photon leakage through the mirrors are obtained.