Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. Hailstorm: Disaggregated Compute and Storage for Distributed LSM-based Databases
 
conference paper

Hailstorm: Disaggregated Compute and Storage for Distributed LSM-based Databases

Bindschaedler, Laurent  
•
Goel, Ashvin
•
Zwaenepoel, Willy
March 16, 2020
Proceedings of the 25th ACM International Conference on Architectural Support for Programming Languages and Operating Systems
25th ACM International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS ’20

Distributed LSM-based databases face throughput and latency issues due to load imbalance across instances and interference from background tasks such as flushing, compaction, and data migration. Hailstorm addresses these problems by deploying the database storage engines over a distributed filesystem that disaggregates storage from processing, enabling storage pooling and compaction offloading. Hailstorm pools storage devices within a rack, allowing each storage engine to fully utilize the aggregate rack storage capacity and bandwidth. Storage pooling successfully handles load imbalance without the need for resharding. Hailstorm offloads compaction tasks to remote nodes, distributing their impact, and improving overall system throughput and response time. We show that Hailstorm achieves load balance in many MongoDB deployments with skewed workloads, improving the average throughput by 60%, while decreasing tail latency by as much as 5X. In workloads with range queries, Hailstorm provides up to 22X throughput improvements. Hailstorm also enables cost savings of 47-56% in OLTP workloads.

  • Files
  • Details
  • Metrics
Type
conference paper
DOI
10.1145/3373376.3378504
Author(s)
Bindschaedler, Laurent  
Goel, Ashvin
Zwaenepoel, Willy
Date Issued

2020-03-16

Publisher

ACM

Published in
Proceedings of the 25th ACM International Conference on Architectural Support for Programming Languages and Operating Systems
ISBN of the book

978-1-4503-7102-5

Total of pages

16

Subjects

Hailstorm

•

disaggregation

•

compute

•

storage

•

distributed

•

database

•

key-value store

•

skew

•

compaction offloading

•

RocksDB

•

MongoDB

•

TiKV

•

TiDB

•

YCSB

•

TPC-C

•

TPC-E

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
LABOS  
Event nameEvent placeEvent date
25th ACM International Conference on Architectural Support for Programming Languages and Operating Systems - ASPLOS ’20

Lausanne, Switzerland

March 16-20, 2020

Available on Infoscience
February 22, 2020
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/166440
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés