A new plenoptic sensor inspired from the visual system of flying insects is introduced. This sensor is referred to as the Panoptic camera. This system has two distinguished features. First, it is an omnidirectional camera capable of recording light information around it. Second, it is a polydioptric system where each CMOS facet has a distinct focal plane; hence the full system is a multiple aperture camera. An FPGA based solution is proposed for real time application deployment. The omnidirectional vision reconstruction algorithm of the Panoptic camera is elaborated and implemented in real time. A built system along with its captured results are presented. Panoptic device is targeted for applications such as depth map estimation, 3D cinematography, ego motion and reconstruction from motion.