  Multi-socket machines with 1-100 TBs of physical memory are becoming
prevalent. Applications running on multi-socket machines suffer non-uniform
bandwidth and latency when accessing physical memory. Decades of research have
focused on data allocation and placement policies in NUMA settings, but there
have been no studies on the question of how to place page-tables amongst
sockets. We make the case for explicit page-table allocation policies and show
that page-table placement is becoming crucial to overall performance. We
propose Mitosis to mitigate NUMA effects on page-table walks by transparently
replicating and migrating page-tables across sockets without application
changes. This reduces the frequency of accesses to remote NUMA nodes when
performing page-table walks. Mitosis uses two components: (i) a mechanism to
enable efficient page-table replication and migration; and (ii) policies for
processes to efficiently manage and control page-table replication and
migration. We implement Mitosis in Linux and evaluate its benefits on real
hardware. Mitosis improves performance for large-scale multi-socket workloads
by up to 1.34x by replicating page-tables across sockets. Moreover, it improves
performance by up to 3.24x in cases when the OS migrates a process across
sockets by enabling cross-socket page-table migration.
