Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. How to Win a Hot Dog Eating Contest: Distributed Incremental View Maintenance with Batch Updates
 
conference paper not in proceedings

How to Win a Hot Dog Eating Contest: Distributed Incremental View Maintenance with Batch Updates

Nikolic, Milos  
•
Dashti Rahmat Abadi, Mohammad  
•
Koch, Christoph  
2016
SIGMOD

In the quest for valuable information, modern big data applications continuously monitor streams of data. These applications demand low latency stream processing even when faced with high volume and velocity of incoming changes and the user’s desire to ask complex queries. In this paper, we study low-latency incremental computation of complex SQL queries in both local and distributed streaming environments. We develop a technique for the efficient incrementalization of queries with nested aggregates for batch updates. We identify the cases in which batch processing can boost the performance of incremental view maintenance but also demonstrate that tuple-at-a-time processing often can achieve better performance in local mode. Batch updates are essential for enabling distributed incremental view maintenance and amortizing the cost of network communication and synchronization. We show how to derive incremental programs optimized for running on large-scale processing platforms. Our implementation of distributed incremental view maintenance can process tens of million of tuples with few-second latency using hundreds of nodes.

  • Files
  • Details
  • Metrics
Loading...
Thumbnail Image
Name

sigmod2016-cr.pdf

Type

Publisher's Version

Version

Published version

Access type

openaccess

Size

1.07 MB

Format

Adobe PDF

Checksum (MD5)

abc41f2582a889726ecd5e6b60538964

Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés