Fueled by recent advances in deep neural networks, reinforcement learning (RL) has been in the limelight because of many recent breakthroughs in artificial intelligence, including defeating humans in games (e.g., chess, Go, StarCraft), self-driving cars, smart-home automation, and service robots, among many others. Despite these remarkable achievements, many basic tasks can still elude a single RL agent. Examples abound, from multiplayer games, multirobots, cellular-antenna tilt control, traffic-control systems, and smart power grids to network management.