Robust, Generalizable, and Reproducible Computational Imaging: A Lensless Perspective
Computational imaging combines innovative camera designs with physics-informed algorithms to enable new sensing capabilities, modalities, and applications. Such cameras can relax optical design constraints and enable encoding of higher-dimensional data. Mask-based lensless cameras are one such design, which replaces the focusing lens with a thin mask, and image formation is shifted from the optics to digital post-processing. Classical methods recover images by solving optimization problems involving data fidelity and regularization terms. While incorporating machine learning can significantly enhance image quality, the reliance of such approaches on training data introduces new limitations. These include sensitivity to slight deviations in imaging conditions, such as signal-to-noise ratio (SNR) or illumination changes, and poor generalization to new configurations, such as changing the mask or sensor. Moreover, while custom hardware setups can already be difficult to replicate, introducing deep learning only exacerbates existing reproducibility challenges in computational imaging. The goal of this thesis is to improve the robustness, generalizability, and reproducibility of lensless cameras and related reconstruction algorithms.
To address robustness and generalizability limitations, this thesis introduces a physics-informed modular imaging framework. Theoretical analysis highlights the impact of model mismatch and noise, informing a robust design that demonstrates resilience to variations in imaging conditions. Our modular approach can handle practical challenges in the deployment of lensless cameras, namely variations in the SNR and in external illumination. We also show that components trained on one lensless system can be reused with others, enabling generalization to scenarios where acquiring new training data is challenging.
Building on model mismatch insights and using generalizable imagers, we explore security and authentication applications for lensless cameras. By leveraging a programmable mask for optical encryption at image acquisition, we achieve encryption strengths that can rival digital standards. Additionally, we demonstrate how a programmable mask enables robust authentication, as each mask pattern leaves a unique fingerprint on the image. When combined with a lensed system, lensless measurements can serve as analog certificates, providing a unique solution for verifying image authenticity and combating deepfakes.
With regards to reproducibility, a comprehensive toolkit is open-sourced - LenslessPiCam. It contains all aspects of lensless imaging, from the mundane-yet-necessary (camera assembly, calibration, data collection, benchmarking) to research-related tasks (mask fabrication and imaging with classical methods and machine learning). Accessible components and methods are prioritized (e.g. Python, Raspberry Pi parts and 3D printing) such that the cost of our prototypes amount to approximately 100 USD, e.g. our DigiCam system based on a programmable mask. With LenslessPiCam, the barriers for lensless camera replication and adoption can be substantially reduced.
The work in this thesis extends beyond the application of lensless cameras. The insight on model mismatch may benefit other forms of computational imaging to improve robustness. Moreover, the developed toolkit serves as an opportunity to discuss practices and challenges in open science that are of relevance to the broader computational imaging community.
EPFL_TH11016.pdf
Main Document
Published version
openaccess
N/A
69.62 MB
Adobe PDF
5b93f77b72a496b74947c3402f2027e4