Size-aware Sharding For Improving Tail Latencies in In-memory Key-value Stores

This paper introduces the concept of size-aware sharding to improve tail latencies for in-memory key-value stores, and describes its implementation in the Minos key-value store.
Size-aware sharding distributes requests for keys to cores according to the size of the item associated with the key. In particular, requests for small and large items are sent to disjoint subsets of cores. Size-aware sharding improves tail latencies by avoiding that a request for a small item gets queued behind a request for a large item.
Minos uses hardware dispatch for all requests for small items, which form the very large majority of all requests, to achieve high throughput, and achieves load balancing by adapting the number of cores handling requests for small and large items to their relative presence in the workload.
We compare Minos to three state-of-the-art designs of in-memory KV stores. Compared to its closest competitor, Minos achieves a 99th percentile latency that is up to 20 times lower. Put differently, for a target 99th percentile latency equal to 10 times the mean service time, Minos achieves a throughput that is up to 7.4 times higher.


Published in:
Proceedings Of The 16Th Usenix Symposium On Networked Systems Design And Implementation, 79-93
Presented at:
16th USENIX Symposium on Networked Systems Design and Implementation, Boston, MA, Feb 26-28, 2019
Year:
Jan 01 2019
Publisher:
Berkeley, USENIX ASSOC
ISBN:
978-1-931971-49-2
Laboratories:




 Record created 2019-07-31, last modified 2019-12-05


Rate this document:

Rate this document:
1
2
3
 
(Not yet reviewed)