Performance of Remote Memory Paging Over the Ethernet

In our first experiment we evaluate four methods for paging:

• NO_RELIABILITY, which uses only main memory of other workstations as a paging device. In this experiment two remote memory servers were used. The measurements were done on an (almost) idle Ethernet to ensure repeatability.
• PARITY_LOGGING, which uses 4 servers plus a parity server, all devoting 10% overflow memory.
• MIRRORING, which uses one primary memory server and one mirror memory server.
• DISK, which uses the local DEC RZ55 disk for paging. In this case the page transfer requests go directly from the DEC OSF/1 kernel to the disk driver without the intervention of our pager.

The completion time of the applications is plotted in figure 2. We see that in all cases the use of remote memory results in significant performance improvements. For example, for the GAUSS application, the NO_RELIABILITY results in 96% faster execution time than DISK. Even for the MVEC application which performed very little paging, NO_RELIABILITY results in 32% faster execution time. The reliability methods induce some runtime overhead as expected but still perform much better than DISK. PARITY_LOGGING results in 40.4% faster execution time for QSORT and in 59.86% faster time for GAUSS. MIRRORING also performs better than DISK for all applications except MVEC, since MVEC performs many pageouts and almost no pageins.

In order to evaluate the use of remote memory for a more realistic application, we measured the completion time of a kernel build after modifying the code of our device driver. As can be seen in figure 2, NO_RELIABILITY performs 26.56% better than disk, PARITY_LOGGING performs 24.65% better and MIRRORING performs just 9.7% better. We see that PARITY_LOGGING performs very close to NO_RELIABILITY. As the number of the remote memory servers used increases, the difference in performance between NO_RELIABILITY and PARITY_LOGGING becomes lower.

Our performance results suggest that paging to remote memory over an Ethernet interconnection network is simply faster than paging to the disk. Even though both the disk and the Ethernet have similar data transfer rates, remote memory does not suffer from seek and rotational latency as DISK does.

Our experimental results verify that even when the network data transfer rate is as low as the disk transfer rate, the performance of remote memory is significantly higher than the performance of disk. Moreover the performance requirements of reliability are surprisingly small. Since architecture trends suggest that modern high speed networks provide much higher data transfer rates than modern disks, the performance improvements of remote memory over disk are bound to increase.