Beyond the Bootcamp: How does software look at memory?

Welcome back to another edition of Beyond the Bootcamp!

Today, we’re going to take a look at how the software side interfaces with memory. We’ll quickly review what the hardware side looks like then jump right into how programming languages like Java or Ruby deal with memory.

How does the hardware look at memory?

Recall the diagram from the first edition of this module:

As our program is executing, the operating system is interfacing with the hardware on the program’s behalf to find the right bits of data to successfully execute the program. It first takes a look at the process register in the CPU to see if the data is on it and if data is not found, it will try to find that data in the subsequent layer all the way until hitting file-based memory. So how do languages interface with this hardware then?

How does software interact with memory?

Here’s a diagram illustrating what the software sees:

This might look a little overwhelming at first glance, so let’s break it down. On the left hand side, we have the address space. Addresses are represented in hexadecimal. Low addresses start at 0 and work their way up to the largest address, 0XFFFFFFFF (2^n-1 where n depends on the system architecture you’re running).

The main two pieces we’re going to talk about in this module are the stack and the heap.

On the right hand side on the very top, we have what’s called the stack. The stack is responsible for storing local variables that can be stored on CPU registers such as integers, floats, doubles, and longs. In your programming career, you might have experienced stack overflow errors when programming using recursion. That’s where this comes from!

For simple variables like ints and longs, these variables are copied onto the stack. So when changeNum(f) is called, the integer f is copied onto the stack. When f is changed in changeNum, that integer is a different integer than the one declared in the main method. This copying of parameters onto the stack is often referred to as call by value.

Contrast this with the heap. The heap serves as a place for a program to allocate pieces of data that can be mutated outside of the current method. An example of this would be arrays. Let’s take a look at a code snippet to demonstrate:

In Java, whenever we call new, we’re explicitly telling Java to set aside some address space for this array on the heap that we want to make reference to in software. When we pass nums into setToAllOnes, under the hood, it sends in a reference to a memory address. The entire nums array is not copied over onto the stack. This is often referred to as call by reference because rather than calling copied over values, we’re calling a reference to a memory address.

In other languages such as Ruby or Python where there are no types, even though as programmers we don’t explicitly call new, the runtime is doing this under the hood for us. Observe:

Changing a number:

Changing an array:

While this scheme sounds great and all, a natural question that might come up is can we run out of memory? What happens if we run out of memory? Does the program crash? We’ll be unpacking all of these questions and more!

Can we run out of memory?

Throughout the execution of the program, memory is constantly allocated onto the heap. What happens if you have a long running application like a server that’s constantly taking in requests, querying the database, and using a lot of memory?

Back in the days when programming languages like C were first created, programmers would manually have to release references to free memory. Unfortunately, when programmers aren’t responsible and don’t free memory, they create memory leaks.

Memory leaks occur when space on the heap is not used anymore by the process but is accidently marked as used. This causes a huge problem because if there’s a long running process that’s not releasing memory, the program will continue to run until the process runs out of memory. If the process runs out of memory, it will start to write chunks of memory onto disk and swap in bits in a process called swapping.

If we look back at our hierarchy, we can see that disk is at the bottom of the pyramid which means it’s very slow to access. To make sure we write performant applications, we should always try to make sure we’re not using memory when we don’t need it. So how do programming languages avoid running out of memory?

Introducing garbage collection!

In most modern languages (think Python, Javascript, Ruby, and Java) garbage collection refers to the act where a process reclaims memory. Contrast this to other languages such as C and C++ don’t have garbage collection and rely on the programmer to free memory themselves.

In C, there’s a special method called malloc which stands for memory allocation. It’s similar to Java’s new construct except C programmers have to call free in order to release memory. In this approach, the programmer has more control of when memory is freed, but it comes at a cost of possibly having memory leaks if a programmer forgets to call free to avoid running out of memory.

In languages with garbage collection however, the language is responsible for figuring out if there are references to that memory address. The language effectively keeps a mapping to each reference to the memory address. Once it reaches 0, it can safely release that memory address from the heap. Let’s take a look at this piece of code as an example:

After myFancyMethod exits, the Java garbage collector sees that there are no more references to the array, foo. So it goes ahead and releases the memory address associated with foo off of the heap!

Awesome! So we’ll never have to worry about memory then right?

Not so fast! While it’s nice that the language will manage the memory for us, we still need to understand how it works so we don’t end up causing unnecessary memory leaks. In the next edition, we’ll take a look at how garbage collection can go awry. Until next time!