Scalable Far Memory: Balancing Faults and Evictions
Page-based far memory systems transparently expand an application's memory capacity beyond a single machine without modifying application code. However, existing systems are tailored to scenarios with low application thread counts, and fail to scale on today's multi-core machines. This makes them unsuitable for data-intensive applications that both rely on far memory support and scale with increasing thread count. Our analysis reveals that this poor scalability stems from inefficient holistic coordination between page fault-in and eviction operations. As thread count increases, current systems encounter scalability bottlenecks in TLB shootdowns, page accounting, and memory allocation. This paper presents three design principles that address these scalability challenges and enable efficient memory offloading. These principles are always-asynchronous decoupling to handle eviction operations as asynchronously as possible, cross-batch pipelined execution to avoid idle waiting periods, and scalability prioritization to avoid synchronization overheads at high thread counts at the cost of eviction accuracy. We implement these principles in both the Linux kernel and a library OS. Our evaluation shows that this approach increases throughput for batch-processing applications by up to 4.2× and reduces 99th percentile latency for a latency-critical memcached application by 94.5%.
École Polytechnique Fédérale de Lausanne
Yale University
École Polytechnique Fédérale de Lausanne
École Polytechnique Fédérale de Lausanne
Yale University
Yale University
Yale University
École Polytechnique Fédérale de Lausanne
2025-10-12
New York, NY, USA
979-8-4007-1870-0
136
152
REVIEWED
EPFL
| Event name | Event acronym | Event place | Event date |
SOSP '25 | Seoul, Republic of Korea | 2025-10-13 - 2025-10-16 | |