The GCC provides a stack protection mechanism that can be compiled into a binary.

Using the -fstack-protector flag during compilation makes GCC supply every function with a buffer of at least 8 bytes. -fstack-protector-strong or -fstack-protector-all can be used to weaken this constraint and protect more or even all functions¹⁾.

Below follows a description of GCC's stack protection features.

Right after the stack frame is created, a protection value is pushed to the stack. There are several schemes to generate this value²⁾, e.g. GCC uses a random value provided by the Linux kernel³⁾. This so-called „stack canary“ is located between the local variables and the meta information on the stack. If no overflow occurred during the execution of the function, the value of the stack canary is unchanged. In case the value changed, execution is aborted.

The behavior of the stack protector is tested with a simple strcpy().

canary/canary.c

// gcc -g -O0 -m32 -fstack-protector canary.c
#include <stdio.h>
#include <string.h>
 
int main(int argc, char *argv[])
{
    char buffer[8] = {0};
 
    if(argc != 2)
    {
        printf("A single argument is required.\n");
        return 1;
    }
 
    strcpy(buffer, argv[1]);
    return 0;
}

Trying to overflow the buffer causes the execution to terminate.

$ ./a.out AAAAAAAAAAAAAAAAAAAAAAAA
*** stack smashing detected ***: <unknown> terminated
[1]    13372 abort (core dumped)  ./a.out AAAAAAAAAAAAAAAAAAAAAAAA

Decompiling the generated binary reveals implementation details of the GCC stack protector. First, a random value is read from gs:0x14. This is a special read-only memory region providing a fixed random number per execution which is saved on the stack.

 5d1:   65 8b 1d 14 00 00 00    mov    ebx,DWORD PTR gs:0x14
 5d8:   89 5d f4                mov    DWORD PTR [ebp-0xc],ebx

After the actual function body, the random value is read again and compared to the saved one. If the values are not equal, an error is raised.

 62d:   65 33 0d 14 00 00 00    xor    ecx,DWORD PTR gs:0x14
 634:   74 05                   je     63b <main+0x8e>
 636:   e8 85 00 00 00          call   6c0 <__stack_chk_fail_local>

As the canary is located between local variables and the meta information on the stack, overflows targeting the variables can not be detected by this mechanism.

Right after the space for local variables is allocated, the parameters of the function are copied to the top of the stack⁴⁾.

The usage of stack canaries secures the meta information located on the stack. However, there is still the possibility to overflow from an array into other local variables. As protecting every single parameter would be quite cumbersome and inefficient, modern compilers like the GCC reorder variables to protect them from being overwritten. This can be achieved by simply locating arrays below (at higher memory addresses) other local variables⁵⁾⁶⁾.