Real-time video services are usually delay sensitive and have strict constraints on the transmission reliability, which poses challenges to live video streaming over multi-hop wireless networks, since the unpredictable packet losses and network congestions caused by time-varying wireless channels greatly degrade the received video quality. To address this, in this paper, we propose a reinforcement learning (RL)-based opportunistic routing (OR) scheme for wireless video streaming with high-reliability and low-delay requirements. It can exploit the broadcast nature of the wireless shared medium and path diversity through OR to improve the transmission reliability, and find the low-delay paths between the source-destination pair dynamically for video packets through the RL module embedded in each relay node. Specifically, we design for the OR a new path-cost metric called the expected anypath delay (EAD), to estimate the end-to-end delay of a packet between the current relay node and the destination. The EAD is dynamically measured and updated over time, thereby reflecting the changes of link quality and the congestion level at the relay node. Moreover, we utilize the ACK message to piggyback the EAD of each relay node to its previous-hop node. Based on the local communication of the EADs from the neighbors, each node in the network can iteratively and independently run the RL module to update its own EAD value. Then, the next-hop forwarder node on a low delay route can be determined by assigning higher relay priority to the candidate forwarder nodes with lower EADs in OR. Simulation results show that the proposed RLOR algorithm can achieve a proper tradeoff between the transmission reliability and latency, so as to support the low-delay transmission of wireless video streams with high received video quality.