Smash The Stack: An Introduction

By Guy Nguyen-Phuoc on March 13, 2019

(By: Guy Nguyen-Phuoc on March 13, 2019)


Buffer Overflows have been the bane of cyber security for more than 45 years. With no sign of stopping, fledgling security researchers should be familiar with the concept. However, concepts such as: a Stack, Buffer, and Memory Addresses might be lacking when most guides simply show how to preform a Buffer Overflow over explaining why a Buffer Overflow is possible.


The goal of this paper is to present an entry level program to tests the inner workings of what entails a Buffer Overflow.

STEP 1 (Installing VS Code):


Once your download for VS Code is complete, please change your directory to the downloads folder and run these commands:

  1. apt install ./“filename.deb”
  2. updatedb

You can now search for VS Code in the “Show Applications” menu.

STEP 2 (Setting up the Code):

Part 1

Using the code in Figure 1, please save to your preferred Directory.

Running code in C

Figure 1

Part 2

Change directories to the saved file and run, as seen in Figure 2 :

  1. gcc “filename.c” -o “newfilename”
  2. chmod 777 “newfilename”
  3. ./”newfilename”

Running code in C

Figure 2

STEP 3 (Buffer Overflow?):

As you can see in Figure 1, the buffer size for the string is 20 characters. What would happen if we added more than 20?

Run the following:

  • python -c “print(‘A’ * 21)” | ./”newfilename”

As we can see in Figure 3 the code executes normally.

Running python code in Kali Linux

Figure 3

Now try the following:

  • python -c “print(‘A’ * 21)” | ./”newfilename”

Running python code in Kali Linux

Figure 4

Segmentation Fault! We have preformed a Buffer Overflow!

STEP 4(What is a Buffer Overflow?):

To understand a Buffer Overflow, one must understand how memory works. Figure 5 show a diagram of Application Memory.

Heirarchy of application memory

Figure 5 (by 0xRick)

When an application runs it is loaded into memory, as seen in Figure 5. Static / Global contains the memory addresses for variables that are not limited to a certain function. The heap contains dynamic memory. What is most important to us is the stack, which contains function calls and local variables.

Part 1 (Memory Addresses)

When a program is executed, the instruction set is stored in to memory addresses, shown in Figure 6.

Register value for buffer overflow example

Figure 6

Each address represents an instruction or collection of data.

Part 2 (Buffer)

A Buffer is simply an allotted amount of space in memory addresses. In our case the char “username”, show in Figure 1, is a buffer of 20 bytes allotted to a string.

Part 3 (Stack)

The Stack is a buffer space allocated to the variables used in a function, ours is the main function. In Figure 7, you can see a rough layout of how a function is stored in a stack. A key feature of a stack is that it handles memory for us, rather than manually setting aside memory addresses for every variable. It also uses a LIFO (Last In First Out) model.

Hierarchy of memory address

Figure 7 (by of Syracuse)

Part 4 (The Overflow)

Now that we know some components of a Buffer Overflow (Stack, Memory Address and Buffer) we can look at what causes it. In short, variable allocated in the stack overflows (exceeds the predetermined buffer limit) that it overwrites the adjacent memory addresses. These addresses may be needed for further jobs in the program, crashing the program if the correct data is overwritten.

  • python -c “print(‘A’ * 64)” | ./”newfilename”

The command above wrote 64 characters into our buffer, “username”. It is certainly higher than the limit set for it, “20” so we can expect it to overwrite other areas of the stack.

Run the following:

  1. gdb “newfilename”
  2. break *main
  3. run
  4. c
  6. info registers

In Figure 8, we can see the RBP (Register Base Pointer) as been overwritten with 0x41, the ascii value for ‘A’.

Showing information registers using gdb

Figure 8

STEP 5 (Why do we care?):

So far, we have only seen our Buffer Overflow crashing the program, but the danger lies in the registers RBP and RSP (Register Stack Pointer). Using a Buffer Overflow, you can manipulate the buffer in a way to change the values of these registers, RBP and RSP, to point to other locations. These locations can be other functions, thus revealing sensitive data, or even reverse shells.

STEP 6 (How Do We Mitigate It?):

Change ‘scanf‘ in the program to ‘fgets‘, as seen in Figure 9.

Code snippet in C

Figure 9

Compile the code as shown in Figure 2 and run the following command to test the Buffer.

  • python -c “print(‘A’ * 64)” | ./”newfilename”

Congratulations, you have mitigated the overflow!

Running python code in Kali Linux

Figure 10

STEP 7 (Home Work):

With the successful implantation of, fgets, we have eliminated the vulnerability. However, we only tested boundary Buffer Overflows, is there another vulnerability in our script? In addition, why does scanf allow a Buffer Overflow and not fgets?



[1] NIST, “Computer Security Technology Planning Study,” Oct. 1972.

[2] NVD, “CVE-2018-17439 Detail,” Sep. 2018.


[4] 0xrick, “Why Do Buffer Overflows Happen,” Dec. 2018.

[5] of Syracuse, “Buffer-Overflow Vulnerabilites and Attacks,” Oct. 2013.