Both speed and functionality.
The Virtual Memory Story
All modern operating systems like Windows, Linux, and MacOS use virtual memory. Virtual memory is a neat, hardware assisted way of lying to your computer about how much memory there is. Your CPU has a Memory Management Unit (MMU), which maps the memory used by each program into actual physical memory. This gives two instant advantages: every program starts at the same location in memory, from the point of of that program, and all programs are isolated from one another. Program A will not have access to the memory of program B.
But let’s say that you have an 8GB computer and both program A and program B each allocate a 4GB memory buffer. That’s too much, of course, but it’s also the case that the OS designers know that sometimes, programmers ask for much more than they’ll ever need. So each of those 4GB blocks will be allocated in virtual memory, but not necessarily actually allocated. Every program (every process) has its own memory mapping, broken up into small “pages” of memory. The MMU table can be updated for these allocations, but some or all of those pages marked as not actually being allocated.
So then, as the programs run, memory that isn’t in memory will flag a processor trap when a processor attempts to access it. That trap jumps to the operating system’s memory manager. If there’s still available memory, some of that will be allocated in physical RAM and assigned to the page that trapped, and for efficiency’s sake, probably a few more. One immediate benefit of this is that the program, thanks to memory management, can get a contiguous block of memory, up to the requested 4GB in this case, even if a contiguous block doesn’t exist in real, physical memory.
But what happens if there is no memory left? One option would be, of course, to just flag that as an out-of-memory error and terminate the program. But long ago, operating system designers realized that plenty of programs load code that’s only ever used once or used rarely. Users can load up programs that may sit idle, consuming resources for little value. So the memory management tables in the system also track which MMU pages have been used at all, which have been used recently, etc. And this allows the operating system to unload some unused memory, to fulfill this need for more RAM for program A.
Now, if the memory pages being reclaimed contain executable code, that’s code that was loaded from the program’s on-disc image. That page can simply be marked as not-in-memory, and that RAM re-claimed. However, if the memory contains data, that has to be put somewhere. That somewhere is a pretty much always on-disc somewhere. Windows systems have a “swap file”, while Linux system typically have a “swap partition”. Either way, there’s a chunk of disc somewhere, whether you use HDD or SSD, that’s there for virtualizing your RAM.
The Performance Hit of VM
So there is a small overhead in basic memory management, or so it would seem. After all, every time your program takes a trap into the memory manager, that’s a chunk of time eaten up that would have been a single instruction. So in a way, yeah, it’s making thing slower.
But the real slowdown is going to disc rather than RAM. It doesn’t matter if you have an old parallel IDE drive or the very latest M.2 SSD on four PCIe channels, your drive is many, many times slower than your DRAM. And so the very first win of having more memory is avoiding page faults, these traps to the memory manager that need to swap in and out to disc.
The Performance Win of VM
Consider also that virtual memory didn’t just magically show up one day in a modern OS. OS designers, application programmers, etc. have been living with virtual memory for over fifty years. There are huge performance benefits with virtual memory.
One is simply squeezing that application that would not otherwise fit. Back in the 1990s, I was working in a company that made a multimedia presentation application. This application ran on MS-DOS, but shuffling lots of photos and video around, it could be a challenge. MS-DOS, of course, didn’t have virtual memory. But in those days, if you ran an MS-DOS program on IBM’s OS/2, it used the OS/2 memory manager and voila! Virtual memory with paging. So presentations that would not run under MS-DOS would run under OS/2 on the same limited computer platform in those days.
Some other features are used, whether paging or not. As the memory manager knows what memory was allocated, it can reclaim it all without the need for applications to spend coding effort reclaim it. That is not an excuse for sloppy programming, but it goes along with that protection factor — a dead program’s resources can be found and returned to the system. Other aspects of program design evolved to take advantage of virtual memory systems as well.
And the operating systems themselves evolved. Since a memory managed system can adapt to the available RAM, modern operating systems don’t simply leave unused RAM lying around, they allocate it when not used otherwise to speed up the system. Larger memory buffers, chunks of code that might be needed later, etc. can be allocated without fear of choking out user programs.
This is not unlimited, of course. I have a system here with 64GB DRAM, running a browser, a CAD program, and a few utilities, and I’m using 21.8GB. This would easily run on a 16GB system, it would work well enough on an 8GB system. But the optimizations the OS is making to speed things up would be lost, so I’d have a slower system. It’s possible in the 8GB case I would see enough memory paging that I’d noticably slow down even more.
More RAM, More Capable… On the Fringe
Naturally, in speaking about large memory, we’re speaking about 64-bit systems. The common “rule of thumb” for memory is at least 2GB per CPU core, so today, 8GB on a 4-core system, 16GB on a 6-core system like mine (12GB is not an option). That’s based on the old 32-bit limit in most operating systems of 2–3 GB per user process and some supposition that you’re not running too many different programs all at once that actually consume CPU time (those sitting idle, of course, can get booted out of memory without much of a slowdown). But that’s just a minimal guideline.
And some things just don’t function in smaller chunks of memory, but given the size of RAM on average computers these days, the number of these applications shinks regularly. Before my last PC upgrade, in 2013, I had a PC with 16GB RAM, and it was holding me back. One of the things I do is large photo composites, and the main program I use for this is pretty memory hungry. I’m stitching together dozens of 16Mpixel or 20Mpixel images, and I was hitting a wall, running out of memory even with a fairly large page file.
So I built my new PC with 64GB DRAM, the most the main board would handle, and proceeded to run some of those photo composites… and immediately ran out of memory. So that was on me — I tried for two at the same time. Going one at a time, I’m pretty good these days.