Friday, March 28, 2014

Kspace: Stack corruption and overflow tests & results


Stack Overflow Experiments & Results

TBD.

Stack Corruption Experiments & Results

 TBD.
Posted by at No comments:

Uspace: Stack corruption and overflow tests & results


Stack Overflow Experiments & Results

TBD.

 Stack Corruption Experiments & Results

 I used the following code to experiment user space stack corruption features.

 
        #include <stdio.h>
        #include <string.h>

        void corrupt_stack(char* str, char *buf1)
        {
            char buf[4];
            strcpy(buf, str);  <-- If input str is longer than 4 bytes, results in stack corruption
            strcpy(buf1, str);   
        }

        int main(void)
        {
            char buf1[100];
            char buf2[100];

            printf("Enter the buffer: \r\n");
            scanf("%s", buf2);
            corrupt_stack(buf2, buf1);
            printf("Survived Stack corruption \r\n");
            return 0;
        }

Stack corruption experiment when -fstack-protector is disabled:
root@babu-VirtualBox:~/tools/stk_corruption# gcc -fno-stack-protector stk_corrupt.c -o stk_corrupt
root@babu-VirtualBox:~/tools/stk_corruption# ./stk_corrupt
Enter the buffer:
sdsadsjsdfsdjfsdfjlskfj
Segmentation fault (core dumped)
root@babu-VirtualBox:~/tools/stk_corruption# 

Stack corruption experiment when -fstack-protector is enabled:

root@babu-VirtualBox:~/tools/stk_corruption# gcc -fstack-protector stk_corrupt.c -o stk_corrupt
root@babu-VirtualBox:~/tools/stk_corruption# ./stk_corrupt
Enter the buffer:
asd
Survived Stack corruption
root@babu-VirtualBox:~/tools/stk_corruption#
root@babu-VirtualBox:~/tools/stk_corruption#
root@babu-VirtualBox:~/tools/stk_corruption# ./stk_corrupt
Enter the buffer:
asdfgg
*** stack smashing detected ***: ./stk_corrupt terminated
Aborted (core dumped)
root@babu-VirtualBox:~/tools/stk_corruption# 

So, -fstack-protector has helped to clearly identify the segmentation fault is due to stack overflow.

Posted by at No comments:

Detect the source of stack corruption & overflow


TBD.

  • Discuss kernel guard page mechanism


Posted by at No comments:

Thursday, March 27, 2014

Hardening against stack corruptions


Stack is hardened against corruption by protecting the sanity of stack using gcc flags.

  • -fstack-protector: When compiled with this option, gcc 
    • Adds additional protective code to vulnerable functions. This code adds a canary variable before the return function address. With such protection, a buffer overrun is likely corrupt the canary first before being able to reach the return function address. So, the protective code verifies the sanity of canary before returning. If canary is corrupt, then the program is instead aborted with error indicating stack corruption.
    • re-orders stack variables so that buffers are placed after pointers & hence reducing the possibility of corrupted pointers corrupting other memory.

This mechanism is applied only to vulnerable functions - the functions with character array >= 8 bytes. This value can be tuned using --param=ssp-buffer-size=N option.

The cost of this protective mechanism is reduced performance and increased binary size .

This flag is available in gcc from 4.1 version. If you have a gcc version less than 4.1, then stack protection feature should be backported. you can apply the ProPolice/StackGuard patches. I have never done such patching. 

History of -fstack-protector:
IBM implemented this mechanism as StackGuard &introduced it as patches to gcc version 2.7. Then IBM improved this idea in ProPolice by reorganizing stack variable to detect pointer corruption. Then, RedHat re-implemented all this to introduce -fstack-protection (and -fstack-protection-all discussed below) in gcc version 4.1

  • -fstack-protector-all: This is same as -fstack-protector, but applied to all function rather than just vulnerable functions.
  • -fstack-protector-strong: -fstack-protector is available only for functions with character arrays & -fstack-protector-all is an overkill for performance. -fstack-protetion-strong extend this protective mechanism to functions with any type of arrays & length: structs/unions/char/.. So, essentially, this option increases the scope of protection mechanism to more vulnerable functions without having to enable it on every function (the way -f-stack-protector-all does) & hence provides better balance of stack corruption detection and performance cost. This option is contributed by Google to gcc & is available in gcc version 4.9 onwards. If your gcc version is <4.9, you can apply the path at http://gcc.gnu.org/ml/gcc-patches/2012-06/msg00974.html to your gcc. Looks like Google enabled this option in Chrome OS by default.

<<compare increase in function coverage & binary size with -fstack-protector vs -fstack-protector-string>>> - TBD.

How to build user space applications stack protection flags?
gcc -fstack-protector <c_file> 

Some distribution like Ubuntu & Fedora Core enable this option by default in gcc.

How to disable stack protections flags enabled by default in user space applications?
gcc -fno-stack-protector <c_file> 

How to build kernel or kernel modules stack protection flags?
Linux kernel has this mechanism only for some architectures. For x86:
  • kernels < 2.6.19 don't have stack protection mechanism
  • -fstack-protection is available in linux kernels >= 2.6.19 
    • CONFIG_CC_STACKPROTECTOR: enables -fstack-protection. By default, stack protection is disabled
  • -fstack-protection-strong & -fstack-protection is available in linux kernels >=3.14
    • CONFIG_CC_STACKPROTECTOR_NONE: default value & doesn't enable any stack protection
    • CONFIG_CC_STACKPROTECTOR_STRONG: enables -fstack-protection-strong
    • CONFIG_CC_STACKPROTECTOR_REGULAR: enables -fstack-protector
For other architectures, check the diff for this compile flag at http://lxr.linux.no and attempt porting. I haven't done so yet & so I don't know the challenges involved. Looks like its NOT simply a matter of which version of gcc & gcc flags you compile the kernel with.
 


Do we need to enable in kernel to use it for kernel modules?

TBD.





References:
























Posted by



at





No comments:
  

















Labels:
,
,
,
,
,
,

















          

        

          

        

Wednesday, March 26, 2014


          

        









Monitor stack limits and usage at runtime in linux













User Space Process Stack


The stack size of a process is limited by two configuration values: soft limit and hard limit. At any time, a process can't exceed usage beyond soft limit. Hard limit is a ceiling for soft limit. An under privileged process can only lower it's hard limit, but can't increase. A privileged process can increase/decrease hard limit. Refer to setrlimit man page to understand how to change hard/soft limit during the execution of a process.


How to find system wide stack size limits applicable by default to a process?



root@babu-VirtualBox:~# ulimit -a | grep stack



stack size              (kbytes, -s) 8192



root@babu-VirtualBox:~# ulimit -s



8192



root@babu-VirtualBox:~# 



How to find the stack size limits of a particular process?



root@babu-VirtualBox:~# cat /proc/2338/limits 



Limit                     Soft Limit           Hard Limit           Units     



Max cpu time              unlimited            unlimited            seconds   



Max file size             unlimited            unlimited            bytes     



Max data size             unlimited            unlimited            bytes     



Max stack size            8388608              unlimited            bytes     



Max core file size        0                    unlimited            bytes     



Max resident set          unlimited            unlimited            bytes     



Max processes             unlimited            unlimited            processes 



Max open files            1024                 4096                 files     



Max locked memory         65536                65536                bytes     



Max address space         unlimited            unlimited            bytes     



Max file locks            unlimited            unlimited            locks     



Max pending signals       6149                 6149                 signals   



Max msgqueue size         819200               819200               bytes     



Max nice priority         0                    0                    



Max realtime priority     0                    0                    



Max realtime timeout      unlimited            unlimited            us        



root@babu-VirtualBox:~#


I wrote a script (located at https://github.com/babuneelam/stk_corruption/blob/master/process_stk_limits.sh) that helps to list the stack size limits of all user-space processes currently running in the system.



root@babu-VirtualBox:~/tools/stk_corruption# ./process_stk_limits.sh 



Current soft/hard limits of various processing running in the system now:



PID     Soft Limit       Hard Limit     Units   Proc_name



973     8388608        unlimited        bytes  /sbin/getty-838400tty6



968     8388608        unlimited        bytes  /sbin/getty-838400tty3



967     8388608        unlimited        bytes  /sbin/getty-838400tty2



959     8388608        unlimited        bytes  /sbin/getty-838400tty5



953     8388608        unlimited        bytes  /sbin/getty-838400tty4



719     8388608        unlimited        bytes  /usr/sbin/dnsmasq....



7088   8388608        unlimited        bytes  /bin/bash./process_stk_limits.sh



.



.



.



.



root@babu-VirtualBox:~/tools/stk_corruption# 


How to find current allocated stack size/usage of a process?



root@babu-VirtualBox:~/tools/stk_corruption# cat /proc/2338/status | grep -i vmstk



VmStk:      136 kB

root@babu-VirtualBox:~/tools/stk_corruption# 







I wrote a script (located at https://github.com/babuneelam/stk_corruption/blob/master/process_stk_usage.sh) that helps to list the stack size usage of all user-space processes currently running in the system.



root@babu-VirtualBox:~/tools/stk_corruption# ./process_stk_usage.sh 



Current soft/hard limits of various processing running in the system now:



PID     Current_Usage         Proc_name



392     156kb rsyslogd-c5  



1043   148kb /usr/bin/X-core:0-auth/var/run/lightdm/root/:0-nolistentcpvt7-novtswitch   



973     136kb /sbin/getty-838400tty6      



968     136kb /sbin/getty-838400tty3      



967     136kb /sbin/getty-838400tty2      



959     136kb /sbin/getty-838400tty5      



953     136kb /sbin/getty-838400tty4      



8714   136kb /bin/bash./process_stk_usage.sh  



6906   136kb /usr/bin/atop-a-w/var/log/atop/atop_20140327600      



679     136kb /usr/sbin/cups-browsed      



666     136kb /usr/lib/policykit-1/polkitd--no-debug      



656     136kb NetworkManager   



596     136kb /usr/sbin/modem-manager  



.



.



.



.



root@babu-VirtualBox:~/tools/stk_corruption# 





How to find the current stack trace of a uspace process?



root@babu-VirtualBox:~/tools/stk_corruption# cat /proc/2469/stack



[<c1bb0da6>] unix_stream_recvmsg+0x416/0xce0
[<c1a8151b>] sock_aio_read+0x17b/0x200
[<c12f89c1>] do_sync_read+0x81/0xe0
[<c12f9ddf>] vfs_read+0x1ff/0x260
[<c12fa0e5>] SyS_read+0x85/0x110
[<c1c77acd>] sysenter_do_call+0x12/0x28
[<ffffffff>] 0xffffffff
root@babu-VirtualBox:~/tools/stk_corruption#









Kernel Space Thread Stack




The size of a kernel side stack for either user space thread or a stack for kernel thread is fixed at compile time & is determined by THREAD_SIZE macro. This value is different for different architectures. Historically, this was 4K, but now this is 8K (http://lxr.linux.no/linux+v3.13.5/arch/x86/include/asm/page_32_types.h#L21). However, I have worked on embedded system which increased the kernel stack size to 16K. I am not aware of the need to have such huge stack - TBD.


How to find stack size limit of a kernel thread at run time?

I don't know of any command line tool that outputs kernel's stack size in linux. Unfortunately, the command /proc/<pid>/status, when used on kernel threads, is not showing stack usage info (as it shows for user space processes).


How to find the run-time stack usage of different kernel threads?

This information is not available by default in linux kernel. The flag CONFIG_DEBUG_STACK_USAGE should be set to "y" or enabled. After enabling, recompile the kernel & reload the image. Then, below are the steps to find the run time stack usage of different kernel threads:







root@babu-VirtualBox:~/tools/stk_corruption# dmesg -c

root@babu-VirtualBox:~/tools/stk_corruption# echo t > /proc/sysrq-trigger







root@babu-VirtualBox:~/tools/stk_corruption# dmesg > dmesg_stk_usage



root@babu-VirtualBox:~/tools/stk_corruption# cat dmesg_stk_usage | grep " S " | cut -d ' ' -f 2-

                               PC               Stack  PID    Parent





dconf worker



S



f73de43c  



6440



2342



1792



0x00000000





gdbus        



S



f73e3540



6440



2343



1792



0x00000000





update-notifier 



S



f73de43c



6072



2423



1792



0x00000000





dconf worker



S



f73ec43c



6224



2426



1792



0x00000000





gdbus           



S



f73f1540



6200



2427



1792



0x00000000





gmain        



S



f73de43c  



6196



2428



1792



0x00000000





deja-dup-monito



S



e401e208



6196



2474



1792



0x00000000





dconf worker



S



f73ec43c  



6212



2477



1792



0x00000000





gdbus      



S



f6c01b00



6216



2478



1792



0x00000000





kworker/u4:2



S



eddb6480 



5652



3648



2



0x00000000





kworker/1:2



S



00000001



7260



3822



2



0x00000000





kworker/0:0



S



00000000



7260



5170



2



0x00000000





update-manager



S



f73de43c  



6052



5827



1601



0x00000000





dconf worker



S



f73ec43c



6224



5828



1601



0x00000000





gdbus



S



f73f1540  



6224



5829



1601



0x00000000





kworker/u5:2



S



f6467ed4



7232



5913



2



0x00000000





dhclient



S



f73ec43c  



6264



6038



656



0x00000000





kworker/u4:0



S



 00000001



7260



6810



2



0x00000000





atop 



S



00000008



6560



6906



1



0x00000000





oneconf-service



S



f73ec43c  



6216



9798



1601



0x00000000





gmain        



S



00000000



6440



9799



1601



0x00000000





dconf worker



S



f73de43c  



6224



9800



1601



0x00000000





gdbus



S



f73e3540  



6440



9802



1601



0x00000000 





cupsd          



S



f73de43c  



6364



9932



1



0x00000000





dbus 



S



f5e7fe08  



6368



9935



9932



0x00000000







root@babu-VirtualBox:~/tools/stk_corruption# 


How to find the current stack trace of a kernel thread?



root@babu-VirtualBox:~/tools/stk_corruption# cat
/proc/132/stack



[<c14a0707>] kjournald2+0x2f7/0x490



[<c10e6985>] kthread+0x115/0x160



[<c1c77a37>]
ret_from_kernel_thread+0x1b/0x28



[<ffffffff>] 0xffffffff



root@babu-VirtualBox:~/tools/stk_corruption#





How to find know which kernel threads are building up towards stack overflow situation?




    

  • One way I can think of: Combine the techniques I wrote above. Have a scripting tool that logs stack usage & the current stack of all kernel threads at regular intervals to the disk. And another scripting tool that helps report this raw data in a intuitive manner. Such a tool would help to proactively look for potential kernel stack overflows building up in the system. However, such a tool can reduce system performance depending on how frequently the stack usage profile information is collected by the tool.
    • 






IRQ Stack





TBD


    

  • CONFIG_DEBUG_STACKOVERFLOW - TBD. Prints message when free stack space drops below a certain limit.. kernel runs a little bit slower due to the proactive additional checks done to ensure that stack doesn't overflow, but catches most overflows. However, these checks are done only in IQR contexts, but not in kernel threads!!
    • 



References:

http://lxr.linux.no/linux+v3.13.5/Documentation/x86/x86_64/kernel-stacks














Posted by



at





No comments:
  

















Labels:
,
,
,
,
,
,

















        

      









Subscribe to:
Posts (Atom)





































































UA-48797665-1