3.1
Stack/buffer overflow
Stack and buffer overflows are common programming errors that can have serious consequences on the security and stability of a system.
Definition
Stack overflow occurs when a function calls itself recursively or when an attempt is made to store more data than can be handled in the memory allocated to the program's stack.
Local variables, return addresses and other data related to function calls are stored on the stack. If a function calls itself repeatedly without terminating, or if too much data is included on the stack, the stack can grow beyond the space allocated, causing an overflow. Overwriting other memory areas not included in reserved memory may affect the values of a function's return pointer or local variables. This can lead to unexpected behaviour or even arbitrary code execution by an attacker. This type of vulnerability is common in programming languages such as C and C++, where incorrectly performed manual memory management can lead to errors.
Definition
Buffer overflow occurs when an attempt is made to write more data to a buffer (a region of memory) than the buffer can hold. This occurs when data is copied from one location to another without first verifying the length of the data or setting appropriate limits.
Attackers often exploit this vulnerability by overwriting the buffer contents with malicious data, which can lead to unexpected behaviour, data corruption or even arbitrary code execution. Buffer overflow is one of the most common forms of exploitation in computer security and can be extremely dangerous if not properly controlled.
Low-level languages leave the control and integrity checks of memory accesses in the hands of the developer. This often leads to bugs that can be exploited by attackers, allowing them to execute malicious code, such as shellcode, and thus gain control of the system.
Interesting
To avoid stack and buffer overflows, it is crucial to write robust code that performs memory bounds checking and correctly validates the data being manipulated in the program. To mitigate overflow risks, it is recommended to use stack monitoring techniques, stack and buffer bounds checking, as well as validating program data inputs.
The example below illustrates how a stack overflow can occur (see figure 2). Consider the following code written in C language, which copies the contents of the parameter entered via keyboard into the local variable of the func function, called buffer, which has a memory length of 20 bytes. The func function prints on the screen the message "Welcome <name>", where name represents the string entered by the user from the keyboard:
+

Fig. 2. Example of stack overflow
In this case, the program lacks a verification to ensure that the length of the string entered by the user does not exceed 100 characters. This opens the door to a stack overflow, since the reserved stack size (100 bytes) is exceeded, resulting in writing to previously unreserved memory areas. This inappropriate writing can result in changes to the contents of these areas, possibly causing errors in program execution. With this vulnerability, an attacker could insert malicious code (so-called shellcode) into the buffer, overwriting the return address of the function to point to the start of the buffer. The shellcode consists of a set of instructions written in assembly language, a low-level language that allows specific tasks to be performed. This enables attackers to execute arbitrary code in the compromised application. It is worth noting that the assembly code used has to be created by the attacker in a way that it is compatible with the system under consideration.
To mitigate buffer overflows, the following prevention mechanisms are recommended:
- Stack controlled values or Canary values: It consists of a random or predefined value that is placed on the stack. This value acts as a protection, being placed between the local variables and the return address of the function on the stack. Its purpose is to detect any buffer overflow, checking if it has been modified before the function finishes its execution. If a change in the value is detected, a condition exception or error is generated. This exception can be caught and handled by the software to avoid the vulnerability or to log the incident for analysis.
- Address Space Layout Randomization (ASLR): It is a security technique that randomly changes the memory location of certain components of the operating system and applications, making difficult to exploit vulnerabilities by changing the base address of the memory each time the program is started. However, if the attacker can find a way to jump to a known memory address and execute arbitrary code, he can evade ASLR.
- NOP sleds: are a technique used in vulnerability exploitation to overcome security mitigations such as ASLR. NOP sleds work by adding a sequence of NOP (no operation) instructions to the beginning of the malicious code (shellcode). Since these NOP instructions do not perform any operation, the processor simply ignores them and moves on to the next instruction. The goal is to create a "sled" of NOP instructions before the malicious code, so that, if the attacker can divert the execution flow to any part of the sled, it will eventually reach the malicious code. The use of NOP sleds increases the probability of success in jumping to the shellcode address, even when the exact location of the shellcode in memory is randomized due to ASLR. This is because the attacker does not need to jump exactly to the beginning of the shellcode; he simply needs to go anywhere in the NOP sled and, from there, the execution flow will slide until it reaches the malicious code.
- Non-executable stacks: Most current operating systems and compilers have incorporated additional security measures to defend against buffer overflow attacks, including stack overflow. Among these measures are the marking of the stack as non-executable, known as NX (Non-eXecutable) bit, as well as the use of data execution prevention, abbreviated as DEP (Data Execution Prevention).
By marking the stack as non-executable, the execution of code that could be found on the stack is avoided. This acts as a barrier against malicious code execution attacks that could be inserted into the stack via a buffer overflow. In addition, some compilers and operating systems have implemented additional protections, such as Stack Canaries. This technique adds special values to the stack to detect any overflow attempts. Also, the use of safe functions, such as strcpy_s instead of strcpy, which consider the size of the target buffer when making copies of strings to avoid overflows, is encouraged.
Thus, when an attempt is made to execute code in a region marked as non-executable, the operating system responds by generating a segmentation fault and finalizes the program execution. This action protects the integrity of the system and prevents the execution of malicious code.
Despite having marked the stack as non-executable, attackers can still introduce malicious code into the system using advanced techniques. One such technique is Return Oriented Programming (ROP), which is widely used in systems that implement security measures such as the NX bit.
On Linux systems, a common way to implement ROP is through the concept of Return to LibC. LibC is a standard library on Unix systems that contains essential C language functions. In this approach, attackers exploit vulnerabilities in a program to manipulate the execution flow and direct it to existing code fragments in LibC. They then modify these fragments to perform malicious actions.
Although code memory regions are not executable, loaded libraries, such as LibC, are executable. Therefore, attackers can take advantage of these memory areas to execute pre-existing code fragments, known as gadgets, and perform unwanted actions.
Interesting
It is important to note that this approach is not limited to LibC alone and can be applied to other loaded libraries or even to the program executable itself. Attackers can build gadget chains using return instructions and function calls sequentially to achieve their goals.
Finally, Table 2 summarizes the primary actions to mitigate stack and buffer overflow vulnerabilities.
Stack overflow | Buffer overflow |
Increase stack size | Stack canaries |
Use static analysis tools | Address Space Layout Randomization |
Dynamic memory allocation | NOP sleds (secure code practices) |
Check stack usage | Non-executable stacks (Data execution prevention) |
Code reviews and testing | Code reviews and testing |