Virtual Memory (VM) is a critical programming abstraction that is widely used in various modern computing platforms. With the rise of datacenter computing and birth of planet-scale online services, the semantic and capacity requirements from memory have evolved dramatically and pose various challenges for VM. The traditional VM implementations cannot scale with the increasing memory capacity present in modern datacenter servers, and the adoption of heterogeneous memory hierarchies stresses the synchronization mechanisms in VM implementations.

In this thesis, we aim to holistically reinvent the VM implementation using OS, architecture, and microarchitecture co-design, which can help towards solving the above challenges. Our redesign of VM is based on the following key insights: (i) we can use the existing abstraction of Virtual Memory Areas (VMAs) to optimize address translation while providing an easy adoption path for the OS developers and ensuring that the application programming model does not require any changes; (ii) we can divide the traditional address translation into a lightweight and heavyweight step where the lightweight step is required for all memory accesses and thus mandates specialized microarchitectural support, while the heavyweight step is invoked infrequently and can be supported using the existing resources present in the cache hierarchy without requiring significant additional silicon; and (iii) instead of encapsulating complicated functionalities completely in the microarchitecture, we can partly offload it to the OS using a simple OS + microarchitecture co-design, while helping the OS developers by providing formal specifications to ensure correctness.

We realize the above insights by introducing an intermediate address space called Midgard between the virtual and physical address spaces. The Midgard address space is used to address the cache hierarchy and coherence domain, while ensuring that the physical address space is only required when accessing the physical memory device. Introduction of the Midgard address space enables lean VMA-based virtual-to-Midgard address translation that provides fast access control and access to cache hierarchy and requires little microarchitectural support. The page-based Midgard-to-physical address translation is only required for capacity management when accessing physical memory, and can be performed using the resources already present in the cache hierarchy instead of requiring specialized microarchitectural support. To handle the exceptions generated by the delayed Midgard-to-physical address translation without requiring significant silicon resources, we introduce a novel OS + microarchitecture co-design to partly offload complicated microarchitectural functionality to be easily performed in the OS, and provide formalism to help the OS developers to ensure correct behavior. We evaluate the address translation in Midgard using full-system trace simulation and show that Midgard can eliminate address translation overhead by using incoming large-capacity cache hierarchies. We also build a full-system RISC-V prototype with Linux to model exception handling behavior for Midgard, and show that the resulting design is both performant and correct. Overall, Midgard reinvents the traditional VM implementation and allows it to scale with the increasing memory capacity requirements in modern datacenter servers.