Repository logo

Infoscience

  • English
  • French
Log In
Logo EPFL, École polytechnique fédérale de Lausanne

Infoscience

  • English
  • French
Log In
  1. Home
  2. Academic and Research Output
  3. Conferences, Workshops, Symposiums, and Seminars
  4. When to Hedge in Interactive Services
 
conference paper

When to Hedge in Interactive Services

Primorac, Mia  
•
Argyraki, Katerina  
•
Bugnion, Edouard  
February 16, 2021
Proceedings Of The 18Th Usenix Symposium On Networked System Design And Implementation
18th USENIX Symposium on Networked Systems Design and Implementation

In online data-intensive (OLDI) services, each client request typically executes on multiple servers in parallel; as a result, “system hiccups”, although rare within a single server, can interfere with many client requests and cause violations of service-level objectives. Service providers have long been fighting this “tail at scale” problem through “hedging”, i.e., issuing redundant queries to mask system hiccups. This, however, can potentially cause congestion that is more detrimental to tail latency than the hiccups themselves. This paper asks: when does it make sense to hedge in OLDI services, and how can we hedge enough to mask system hiccups but not as much as to cause congestion? First, we show that there are many realistic scenarios where hedging can have no benefit—where any hedging-based scheduling policy, including the state-of-the-art, yields no latency reduction compared to optimal load balancing without hedging. Second, we propose LÆDGE, a scheduling policy that combines optimal load balancing with work-conserving hedging, and evaluate it in an AWS cloud deployment. We show that LÆDGE strikes the right balance: first, unlike the state of the art, it never causes unnecessary congestion; second, it performs close to an ideal scheduling policy, improving the 99th percentile latency by as much as 49%, measured on 60% system utilization—without any difficult parameter training as found in the state of the art.

  • Files
  • Details
  • Metrics
Type
conference paper
Author(s)
Primorac, Mia  
Argyraki, Katerina  
Bugnion, Edouard  
Date Issued

2021-02-16

Publisher

USENIX ASSOC

Publisher place

Berkeley

Published in
Proceedings Of The 18Th Usenix Symposium On Networked System Design And Implementation
ISBN of the book

978-1-939133-21-2

Total of pages

15

Start page

373

End page

388

Subjects

tail latency

•

online data intensive services

•

cloud computing

Editorial or Peer reviewed

REVIEWED

Written at

EPFL

EPFL units
NAL  
DCSL  
Event nameEvent placeEvent date
18th USENIX Symposium on Networked Systems Design and Implementation

Virtual Conference

April 12-14, 2021

Available on Infoscience
February 16, 2021
Use this identifier to reference this record
https://infoscience.epfl.ch/handle/20.500.14299/175305
Logo EPFL, École polytechnique fédérale de Lausanne
  • Contact
  • infoscience@epfl.ch

  • Follow us on Facebook
  • Follow us on Instagram
  • Follow us on LinkedIn
  • Follow us on X
  • Follow us on Youtube
AccessibilityLegal noticePrivacy policyCookie settingsEnd User AgreementGet helpFeedback

Infoscience is a service managed and provided by the Library and IT Services of EPFL. © EPFL, tous droits réservés