User space run-time memory error detection tools: Comparison

In this post, I will provide comparison on various memory tools I tried so far - Valgrind's memcheck, AddressSanitizer, MALLOC_CHECK_. The tests I tried on different tools is located at https://github.com/babuneelam/valgrind_tests/blob/master/vg_tests.c

Here is the comparison chart.

	Valgrind’s memcheck	ASAN	MALLOC_CHECK_
uninitialized use	✓
Read/Write after free	✓
buffer overrun	✓	✓	✓
Read/Write bad stack location (underrun)	✓
Memory Leak	✓
c++related test (ignore this)	✓
Overlapping memcpy	✓
double free	✓	✓	✓
Read/Write invalid stack location (overrun)	✓
Read/Write invalid global data location (overrun)
Performance hit	10-30x	?	?

Hope this helps.

AddressSanitizer: Detect user space run-time memory errors

AddressSanitizer(ASAN) Overview

ASAN helps detect user space memory errors when compiled with ASAN gcc flag. To understand the type of memory errors, I used the same tests I used to explore Valgrind's memcheck. I will also provide details of errors MALLOC_CHECK_ catches in "Usage Example" section below.

ASN gcc flag is available in gcc version 4.8 and above.

ASAN Internals
https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerAlgorithm

ASAN Usage

• Step1: Recompile the program enabling gdb & turning off compiler optimizations:

gcc -g -fsanitize=address -O1 prog.c -o <prog_name>

• Step2: Run the program:

<prog_name> <prog_args>

ASAN Usage Example

I used the code at https://github.com/babuneelam/valgrind_tests/blob/master/vg_tests.c to explore 10 of the 11 tests:

1. uninitialized free
2. Read/Write after free
3. buffer overrun
4. Read/Write bad stack location (underrun)
5. Memory Leak
6. c++related test (ignore this)
7. Overlapping memcpy
8. double free
9. pass unaddressable memory to system call
10. Read/Write invalid stack location (overrun)
11. Read/Write invalid global data location (overrun) 

Here are the results:

root@babu-VirtualBox:~/tools/address_sanitizer_tests# gcc -g -fsanitize=address -O1 vg_tests.c -o as_tests

root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 1

0

*num = 0

root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 2

root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 3

=================================================================

==4110== ERROR: AddressSanitizer: heap-buffer-overflow on address 0xb5e00ff8 at pc 0x8048942 bp 0xbf943fd8 sp 0xbf943fcc

WRITE of size 4 at 0xb5e00ff8 thread T0

    #0 0x8048941 (/home/babu/tools/address_sanitizer_tests/as_tests+0x8048941)

    #1 0x8048c1b (/home/babu/tools/address_sanitizer_tests/as_tests+0x8048c1b)

    #2 0xb5fa0904 (/lib/i386-linux-gnu/libc-2.17.so+0x19904)

    #3 0x80487b0 (/home/babu/tools/address_sanitizer_tests/as_tests+0x80487b0)

0xb5e00ff8 is located 0 bytes to the right of 40-byte region [0xb5e00fd0,0xb5e00ff8)

allocated by thread T0 here:

    #0 0xb6151854 (/usr/lib/i386-linux-gnu/libasan.so.0.0.0+0x16854)

    #1 0x8048918 (/home/babu/tools/address_sanitizer_tests/as_tests+0x8048918)

    #2 0xb5fa0904 (/lib/i386-linux-gnu/libc-2.17.so+0x19904)

Shadow bytes around the buggy address:

  0x36bc01a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

  0x36bc01b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

  0x36bc01c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

  0x36bc01d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

  0x36bc01e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

=>0x36bc01f0: fa fa fa fa fa fa fa fa fa fa 00 00 00 00 00[fa]

  0x36bc0200:fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

  0x36bc0210: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

  0x36bc0220: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

  0x36bc0230: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

  0x36bc0240: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa

Shadow byte legend (one shadow byte represents 8 application bytes):

  Addressable:           00

  Partially addressable: 01 02 03 04 05 06 07

  Heap left redzone:     fa

  Heap righ redzone:     fb

  Freed Heap region:     fd

  Stack left redzone:    f1

  Stack mid redzone:     f2

  Stack right redzone:   f3

  Stack partial redzone: f4

  Stack after return:    f5

  Stack use after scope: f8

  Global redzone:        f9

  Global init order:     f6

  Poisoned by user:      f7

  ASan internal:         fe

==4110== ABORTING

root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 4

root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 5

root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 6

root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 7

root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 8

ASAN:SIGSEGV

=================================================================

==4118== ERROR: AddressSanitizer: SEGV on unknown address 0xfffffff4 (pc 0xb6102d6e sp 0xbf8b6c40 bp 0xfffffff4 T0)

AddressSanitizer can not provide additional info.

    #0 0xb6102d6d (/usr/lib/i386-linux-gnu/libasan.so.0.0.0+0x6d6d)

    #1 0xb61127b3 (/usr/lib/i386-linux-gnu/libasan.so.0.0.0+0x167b3)

    #2 0x8048ae4 (/home/babu/tools/address_sanitizer_tests/as_tests+0x8048ae4)

    #3 0x8048c34 (/home/babu/tools/address_sanitizer_tests/as_tests+0x8048c34)

    #4 0xb5f61904 (/lib/i386-linux-gnu/libc-2.17.so+0x19904)

    #5 0x80487b0 (/home/babu/tools/address_sanitizer_tests/as_tests+0x80487b0)

==4118== ABORTING

root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 9

Please type a bunch of characters and hit enter.

wer

root@babu-VirtualBox:~/tools/address_sanitizer_tests#
root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 10
root@babu-VirtualBox:~/tools/address_sanitizer_tests#
root@babu-VirtualBox:~/tools/address_sanitizer_tests# ./as_tests 11
root@babu-VirtualBox:~/tools/address_sanitizer_tests#

As we can see, just like MALLOC_CHECK_ tool, ASAN caught only buffer overrun and double free. Considering that ASAN homepage mentioned that ASAN detects use after free, I am surprised that my use after free test(test number - 2) went undetected by ASAN!!

References:
https://code.google.com/p/address-sanitizer/

MALLOC_CHECK_: Detect User space run-time memory errors

MALLOC_CHECK_ overview

MALLOC_CHECK_ a glibc tool that can be used to find memory errors in linux user space C/C++ applications.To understand the type of memory errors, I used the same tests I used to explore Valgrind's memcheck. I will also provide details of errors MALLOC_CHECK_ catches in "Usage Example" section below.

MALLOC_CHECK_ Usage

• Step1: Modify the program with the following code snippet:

         if (mallopt(M_CHECK_ACTION, 7) != 1) {

           fprintf(stderr, "mallopt() failed");

           return(-1);

         }

The number 7 n mallopt argument indicates response level. Different response levels:

0  Ignore error conditions; continue execution (with undefined results).

1  Print a detailed error message and continue execution.                    

2  Abort the program.

3  Print detailed error message, stack trace, and memory mappings, and abort the program.

5  Print a simple error message and continue execution.

7  Print simple error message, stack trace, and memory mappings, and abort the program.

• Step1: Recompile the program enabling gdb & turning off compiler optimizations:

gcc -g prog.c -o <prog_name>

• Step2: Run the program in with MALLOC_CHECK_ environment variable set:

MALLOC_CHECK_=<response_level> <prog_name> <prog_args>

From my experiments, I found that the response level provided in source code overrides the response level provided here. Despite this overriding, this environment variable had to be passed at command line for source code change to show the impact. However, when I provided 0 here, this overrode the source code response level. 

MALLOC_CHECK_ Usage Example

I used the code at https://github.com/babuneelam/malloc_check_tests/blob/master/mc_tests.c to explore 10 of the 11 tests:

1. uninitialized use
2. Read/Write after free
3. buffer overrun
4. Read/Write bad stack location (underrun)
5. Memory Leak
6. c++related test (ignore this)
7. Overlapping memcpy
8. double free
9. pass unaddressable memory to system call
10. Read/Write invalid stack location (overrun)
11. Read/Write invalid global data location (overrun) 

Here are the results:

root@babu-VirtualBox:~/tools/malloc_check# gcc -g mc_tests.c -o mc_tests

root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 1

10

*num = 0 

root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 2

root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 3

i[-1] = 49 

free(): invalid pointer

======= Backtrace: =========

/lib/i386-linux-gnu/libc.so.6(+0x767f1)[0xb75b57f1]

/lib/i386-linux-gnu/libc.so.6(cfree+0x9b)[0xb75b914b]

./mc_tests[0x8048676]

./mc_tests[0x8048878]

/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xb7558905]

./mc_tests[0x80484e1]

======= Memory map: ========

08048000-08049000 r-xp 00000000 08:01 1076878    /home/babu/tools/malloc_check/mc_tests

08049000-0804a000 r--p 00000000 08:01 1076878    /home/babu/tools/malloc_check/mc_tests

0804a000-0804b000 rw-p 00001000 08:01 1076878    /home/babu/tools/malloc_check/mc_tests

09852000-09873000 rw-p 00000000 00:00 0          [heap]

b750a000-b7525000 r-xp 00000000 08:01 262149     /lib/i386-linux-gnu/libgcc_s.so.1

b7525000-b7526000 r--p 0001a000 08:01 262149     /lib/i386-linux-gnu/libgcc_s.so.1

b7526000-b7527000 rw-p 0001b000 08:01 262149     /lib/i386-linux-gnu/libgcc_s.so.1

b753e000-b753f000 rw-p 00000000 00:00 0 

b753f000-b76ed000 r-xp 00000000 08:01 263093     /lib/i386-linux-gnu/libc-2.17.so

b76ed000-b76ef000 r--p 001ae000 08:01 263093     /lib/i386-linux-gnu/libc-2.17.so

b76ef000-b76f0000 rw-p 001b0000 08:01 263093     /lib/i386-linux-gnu/libc-2.17.so

b76f0000-b76f3000 rw-p 00000000 00:00 0 

b7708000-b770c000 rw-p 00000000 00:00 0 

b770c000-b770d000 r-xp 00000000 00:00 0          [vdso]

b770d000-b772d000 r-xp 00000000 08:01 263069     /lib/i386-linux-gnu/ld-2.17.so

b772d000-b772e000 r--p 0001f000 08:01 263069     /lib/i386-linux-gnu/ld-2.17.so

b772e000-b772f000 rw-p 00020000 08:01 263069     /lib/i386-linux-gnu/ld-2.17.so

bfd17000-bfd38000 rw-p 00000000 00:00 0          [stack]

Aborted (core dumped)

root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 4

root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 5

root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 7

root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 8

Segmentation fault (core dumped)
free(): invalid pointer
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(+0x767f1)[0xb75f47f1]
/lib/i386-linux-gnu/libc.so.6(cfree+0x9b)[0xb75f814b]
./mc_tests[0x8048800]
./mc_tests[0x8048963]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0xb7597905]
./mc_tests[0x8048561]
======= Memory map: ========
08048000-08049000 r-xp 00000000 08:01 1076878    /home/babu/tools/malloc_check/mc_tests
08049000-0804a000 r--p 00000000 08:01 1076878    /home/babu/tools/malloc_check/mc_tests
0804a000-0804b000 rw-p 00001000 08:01 1076878    /home/babu/tools/malloc_check/mc_tests
08418000-08439000 rw-p 00000000 00:00 0          [heap]
b7549000-b7564000 r-xp 00000000 08:01 262149     /lib/i386-linux-gnu/libgcc_s.so.1
b7564000-b7565000 r--p 0001a000 08:01 262149     /lib/i386-linux-gnu/libgcc_s.so.1

b7565000-b7566000 rw-p 0001b000 08:01 262149     /lib/i386-linux-gnu/libgcc_s.so.1

b757d000-b757e000 rw-p 00000000 00:00 0

b757e000-b772c000 r-xp 00000000 08:01 263093     /lib/i386-linux-gnu/libc-2.17.so

b772c000-b772e000 r--p 001ae000 08:01 263093     /lib/i386-linux-gnu/libc-2.17.so

b772e000-b772f000 rw-p 001b0000 08:01 263093     /lib/i386-linux-gnu/libc-2.17.so

b772f000-b7732000 rw-p 00000000 00:00 0

b7748000-b774b000 rw-p 00000000 00:00 0

b774b000-b774c000 r-xp 00000000 00:00 0          [vdso]

b774c000-b776c000 r-xp 00000000 08:01 263069     /lib/i386-linux-gnu/ld-2.17.so

b776c000-b776d000 r--p 0001f000 08:01 263069     /lib/i386-linux-gnu/ld-2.17.so

b776d000-b776e000 rw-p 00020000 08:01 263069     /lib/i386-linux-gnu/ld-2.17.so

bfd64000-bfd85000 rw-p 00000000 00:00 0          [stack]

Aborted (core dumped)

root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 9

Please type a bunch of characters and hit enter.

wer

root@babu-VirtualBox:~/tools/malloc_check#
root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 10
root@babu-VirtualBox:~/tools/malloc_check#
root@babu-VirtualBox:~/tools/malloc_check# MALLOC_CHECK_=7 ./mc_tests 11
root@babu-VirtualBox:~/tools/malloc_check#

As we can see, MALLOC_CHECK_ caught only buffer overrun and double free. 

Reference:
http://man7.org/linux/man-pages/man3/mallopt.3.html