Existing works for depth estimation and image deblurring in the presence of depth-dependent blur work with the assumption of a multi-layered scene wherein each layer is modeled in the form of a fronto-parallel plane. In this work, we attempt to relax these constraints by considering more generalized settings of a 3D scene with piecewise planar structure, i.e., a scene that can be modeled as a combination of multiple planes with arbitrary orientations. To this end, we first propose a novel approach to estimate the normal of a planar surface from a single motion-blurred image. We then extend this idea and develop an algorithm for automatic recovery of the number of planes, the parameters corresponding to each plane, and camera motion from a single motion-blurred image of a multi-planar 3D scene. Finally, we propose a first-of-its-kind approach to recover the planar geometry and latent image of the scene by adopting an alternating minimization framework built on our findings. Experiments on synthetic and real data reveal that our proposed method achieves state-of-the-art results on the dual problem of depth recovery and image deblurring.