Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. EPFL thesis
  4. A Real-Time Multi-Camera Depth Estimation ASIC with Custom On-Chip Embedded DRAM
 
doctoral thesis

A Real-Time Multi-Camera Depth Estimation ASIC with Custom On-Chip Embedded DRAM

Narinx, Jonathan Emmanuel D  
2019

The capability to process high-resolution videos in real-time is becoming more important in a wide variety of applications such as autonomous vehicles, virtual reality or intelligent surveillance systems. The high-accuracy and complex video processing algorithms needed in these applications led to increased challenges for the system design, due to the amount of computations to be processed instantly. Furthermore, video processing algorithms operate on large amounts of data, but storing this data in dense off-chip memories leads to difficulties to meet bandwidth requirements. Hence, embedded memories are usually required to temporally store data on-chip, close to the processing units. However, on-chip embedded memories often dominate most of the silicon real-estate and power budget of modern video processing system-on-chips. Considering the current trend toward videos of higher resolutions and faster frame rates, these challenges are expected to dramatically increase in the future. One of the most important kernels required in modern video processing systems is the depth perception, since depth information is needed for many advanced video processing algorithms. Depth maps can be created using stereo-matching, which denotes the problem of finding dense correspondences in pairs of images. However, computing high-quality depth maps in real-time, on high-resolution images at high-frame rate is challenging due to the computational complexity of stereo-matching algorithms. Furthermore, their need for large memories and bandwidth limits the performance of depth estimation units, increases their power consumption, and renders them challenging for system integration. In this thesis, we develop task-specific solutions from the algorithmic level to the circuit level that accelerate the computation operations and data transfers, and optimize the on-chip data storage of such depth estimation units. First, we present hardware oriented stereo-matching algorithms and their hardware implementations, tailored to increase parallelism while using only on-chip memory to produce high-quality, high-resolution depth maps. Based on that, we propose a multi-camera depth map estimation ASIC implemented in 28nm, which is capable of computing in real-time up to 2K resolution depth maps at 32fps with up to 256-pixel disparity range using two/three cameras. Our design achieves the highest reported disparity range capability at the lowest power consumption and highest frame rate, while computing high-quality depth maps. It also features a stream-in/out interface for easy integration in existing vision systems. Despite having optimized the complexity of the stereo-matching process, a considerable share of the proposed ASIC area and power budget is consumed by the on-chip memory. To address this issue, we focus next on how data can be stored effectively on-chip. An emerging on-chip memory alternative to conventional SRAM is the logic-compatible GC-eDRAM, due to its high-density, low-power, and inherent two-ported operation. In this thesis, we propose a single-well mixed 3T gain-cell implementation in 28nm FD-SOI. Based on this concept, a custom 24kbit GC-eDRAM macro suitable for modern real-time video processing units was fabricated in 28nm FD-SOI, resulting in the highest density logic-compatible embedded memory reported in the literature, with improved data retention time compared to conventional 3T gain-cells, and lower static power compared to conventional SRAM.

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

EPFL_TH7163.pdf

Access type

openaccess

Size

59.65 MB

Format

Adobe PDF

Checksum (MD5)

e778e0a1d58201bf05dda937b01df904

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés