Creating and Applying a Patch File

Recently, I went over the patch file creation & application process in linux. In this post, I will cover a few linux commands useful for creating a patch file & applying a patch file.


Creating a Patch File

Sometimes, we need to create a patch file & distribute it to customers, partners, etc.

To create a patch, we need a copy of original source code directory & the modified source code directory. Lets say we are starting to modify the sources at /home/babu/patch_tests//valgrind_tests. Before any modifications, its better to take a copy to /home/babu/patch_tests//valgrind_tests_new & then modify the sources in /home/babu/patch_tests//valgrind_tests_new.

Now, to create the patch file, execute this command:

diff –Naur    <old_dir>  <new_dir>      >   file.patch

Example: 

babu@babu-VirtualBox:~/patch_tests$ diff -Naur valgrind_tests valgrind_tests_new > vg.patch

babu@babu-VirtualBox:~/patch_tests$ cat vg.patch
diff -Naur valgrind_tests/vg_test.c valgrind_tests_new/vg_test.c
--- valgrind_tests/vg_test.c        2014-09-10 21:26:39.473561882 -0700
+++ valgrind_tests_new/vg_test.c            2014-09-10 21:29:41.966273636 -0700
@@ -4,6 +4,12 @@

 int global_array[100] = {-1};
 +/* Testing patch functionality */
+void patch_test()
+{
+  return 1;
+}
+
  void
 unintialized_use()
 {
babu@babu-VirtualBox:~/patch_tests$

Now, remove the first line in the patch - which displays the command. After the removal the file should be like this:

babu@babu-VirtualBox:~/patch_tests$ cat vg.patch

valgrind_tests/vg_test.c        2014-09-10 21:26:39.473561882 -0700
+++
valgrind_tests_new/vg_test.c            2014-09-10 21:29:41.966273636 -0700
@@ -4,6 +4,12 @@

 int global_array[100] = {-1};
 +/* Testing patch functionality */
+void patch_test()
+{
+  return 1;
+}
+

  void
 unintialized_use()
 {
babu@babu-VirtualBox:~/patch_tests$

Applying the Patch File

Lets say we are receiving patch files & need to apply them.

Now, to apply the patch file, execute this command:

patch -p1 <source file> < <patch specific to source file>

Example:

For the earlier example I discussed above, I duplicated the unchanged sources to  /home/babu/patch_tests//valgrind_tests_patched. Now, to patch vg.patch to this duplicated sources, here is what I did:

babu@babu-VirtualBox:~/patch_tests$ patch -p1 valgrind_tests_patched/vg_test.c < vg.patch
patching file valgrind_tests_patched/vg_test.c
babu@babu-VirtualBox:~/patch_tests$ diff -Naur valgrind_tests valgrind_tests_patched/
diff -Naur valgrind_tests/vg_test.c valgrind_tests_patched/vg_test.c
--- valgrind_tests/vg_test.c   2014-09-10 21:26:39.473561882 -0700
+++ valgrind_tests_patched/vg_test.c         2014-09-10 21:49:07.039469608 -0700
@@ -4,6 +4,12 @@
 
int global_array[100] = {-1};
 +/* Testing patch functionality */
+void patch_test()
+{
+  return 1;
+}
+

void
unintialized_use()
{
babu@babu-VirtualBox:~/patch_tests$

I will try to cover patching of multiple files in a directory some other time.


Hope this helps.

VirtualBox Error - Kernel driver not installed

I have been VirtualBox in my macbook pro. But, I stopped using it in last 3-4 months. And when I tried to use it today again, it threw up the following errors:

Error in Virtual Box window:

Failed to open a session for the virtual machine MyUbuntu.
Virtual machine 'MyUbuntu' has terminated unexpectedly during startup. 

NS_ERROR_FAILURE (0x80004005)
Machine
IMachine {<<<some big number here>>>}

Error in an additional popup window:

Kernel driver not installed (rc=-1908)
Make sure the kernel module has been loaded successfully.

Instead of debugging how to fix this issue, I just updated the virtualbox with an available newer version (Virtual Box on top left --> Check for updates). 

After update the error went away. None of my existing VMs or config was impacted due to the upgrade.


Hope this helps.

Internet Browsing from pubilc computers - Safety & Privacy

At times, I had to access Internet from public computers in place such as libraries, non-profit organizations etc. My family in India accesses Internet primarily from public kiosks.  In this post, I will cover a few basic internet safety & privacy tips that can be helpful when accessing internet from such public computers.

Internet Safety & Privacy - Importance

Following are some harmful effects of using Internet without caution:

• Identity theft
• The recent security breach - Heartbleed - turned me extremely cynical about internet safety. 

While I am not going to offer solution here for complex security issues such as Heartbleed,  I will provide some basic guidelines to protect yourself from even simpler but potentially harmful internet security issues.

• Privacy

According to me, it is utmost important to be cautious about safety & privacy of personal data while accessing internet from public computers.

Internet Safety & Privacy - Guidelines & How tos

I will cover 'how to's for IE 9 & Chrome Version 35. If my time permits, I will try adding 'how to's for other browsers as well. For other versions of IE & Chrome, the instructions may be more or less similar, but its best to search over internet for your specific browser versions, if different from IE 9 & Chrome v 35.

• Before you begin your Internet browsing session
• Choose proper Location: Ensure that you avoid exposing you computer monitor to public. That way, your personal information is not visible to accidental watchers.
• Switch to Private Browsing: Most browsers support private browsing. If we operate in private browsing mode, the browsing history is not recorded in the local system. Any temporary files downloaded is not retained in the local system?
• IE 9
• Tools->Safety->InPrivate Browsing
• Chrome v35
• Chrome Menu (  ) --> New Incognito window
• Disable Automatic Password Storage (or even prompting for such storage):
• IE 9
• Tools->Internet Options->Content->Auto Complete Settings : Uncheck the option "Usernames and password on forms"
• Chrome v35
• Chrome Menu (  ) -->Settings --> "Show Advanced Settings" --> "Passwords and Forms" section --> Uncheck the checkbox - "Offer to save your web passwords. "
• Before you end your Internet browsing session
• Delete locally saved files - if you happen to save any files to desktop or other location during your browsing session.
• Log out from any websites you logged in during your session
• Delete temporary internet files & browsing history
• IE 9
• Tools->Safety->Delete Browsing history (check “Temporary Internet Files”)
• Chrome v 35
• Chrome Menu (  ) -->Tools --> Clear browsing data --> A dialog box opens: Select "beginning of the time" from drop down box, check all the check boxes --> Click "clear browsing data" 

Hope this helps.


Disclosure: 
I learnt some of the above high level ideas while volunteering as a credit coach at United Way Silicon Valley & also advised my coachees to follow these ideas.

ldd/objdump - finding shared library dependencies

What is ldd?
ldd stands for 'list dynamic dependencies'. It is a shell script that displays shared libraries required by a unix/linux program or a shared library.

ldd internals

Copy-pasted from http://linux.die.net/man/1/ldd :

" 
In the usual case, ldd invokes the standard dynamic linker with the LD_TRACE_LOADED_OBJECTS environment variable set to 1, which causes the linker to display the library dependencies. Be aware, however, that in some circumstances, some versions of ldd may attempt to obtain the dependency information by directly executing the program. Thus, you should never employ ldd on an untrusted executable, since this may result in the execution of arbitrary code. A safer alternative when dealing with untrusted executables is:

$ objdump -p /path/to/program | grep NEEDED

"

LDD source code: http://stuff.mit.edu/afs/sipb/project/phone-project/bin/arm-linux-ldd
(is this latest?)

ldd command line options 
 

babu@babu-VirtualBox:~$ ldd --help

Usage: ldd [OPTION]... FILE...

      --help              print this help and exit

      --version           print version information and exit

  -d, --data-relocs       process data relocations

  -r, --function-relocs   process data and function relocations

  -u, --unused            print unused direct dependencies

  -v, --verbose           print all information

For bug reporting instructions, please see:

<https://bugs.launchpad.net/ubuntu/+source/eglibc/+bugs>.

babu@babu-VirtualBox:~$ 

Sample Usage

Here are a few sample outputs:

babu@babu-VirtualBox:~$ ldd /bin/lsmod

            linux-gate.so.1 =>  (0xb77ad000)

            libkmod.so.2 => /lib/i386-linux-gnu/libkmod.so.2 (0xb7765000)

            libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb75b1000)

            /lib/ld-linux.so.2 (0xb77ae000)

babu@babu-VirtualBox:~$ 

babu@babu-VirtualBox:~$ ldd /lib/i386-linux-gnu/libc.so.6

            /lib/ld-linux.so.2 (0xb76e4000)

            linux-gate.so.1 =>  (0xb76e3000)

babu@babu-VirtualBox:~$ 

In the above outputs, the first field displays the shared library name & the second field (after ==>) displays the path of the shared library. In case the shared library is not found in the system, it displays "not found" in this field.

Objdump way of finding dependencies
 

babu@babu-VirtualBox:~$ objdump -p /bin/lsmod | grep NEEDED  

  NEEDED               libkmod.so.2

  NEEDED               libc.so.6

babu@babu-VirtualBox:~$

As we can see, objdump too is useful, but it misses out other details ldd displays. So, I prefer ldd over objdump to view shared library dependencies.

References:

http://linux.die.net/man/1/ldd

Toolchain basics

Recently, I got an opportunity to explore toolchain basics. In this post, I will restrict my definitions/terms to C/Linux based development environment though toolchain term is used even in other development environments.

What is a toolchain?

A toolchain is used to build binaries for target system. In addition, toolchain helps in inspecting binaries, debugging, profiling  etc etc.

What are the different components of a toolchain?

• Mandatory:
• GNU binutils: a set of development tools required to build and manage binaries - assembler, linker, strip, nm, objdump, readelf, ar, addr2line, gprof ..
• Compiler: gcc
• libc: glibc or eglibc or uClibc
• Optional:
• Debugger: gdb

References:
http://elinux.org/Toolchains

http://www.gnu.org/software/binutils/

gdb: Debugging core dump of a user space application

What is a core dump?

A core dump is a snapshot of useful information of the process - such as memory contents, registers etc. Core dump files are represented in ELF format.

Triggers that generate core dump:

• A core dump is automatically triggered (depending of system core dump configuration) in the event of some fatal error of a process.
• A core dump can be generated manually using linux command gcore.
• A core dump can be generated manually using gdb facilities.
• A core dump can be generated programmatically from inside a process.

Core dump can be used by developers offline to debug fatal errors of the process, inspect process state even for non-critical software issues.

The offline debugging ability offered by core dumps helps:

• debug software issues which are infrequently reproducible
• debug software issues where access to the affected devices is limited/unavailable (such as customer environment)

Are core dumps generated by default?

Depends on the configuration. System should be configured to generate core dumps. The default behavior depends on the default system configuration for core dump file.

How to configure a system to generate a core dump in the event of fatal errors?

Set the maximum size of core files created using the following command:
 

ulimit -c <max_core_file_size>

When set to 0, core files are not generated. When core file being generated exceeds the above size, then core file would be truncated to the above size. To set core file size to unlimited, use the following command:

ulimit -c unlimited

You can check the current value using the following command:

root@babu-VirtualBox:~/tools/core_dump# ulimit -a | grep core
core file size          (blocks, -c) 0
root@babu-VirtualBox:~/tools/core_dump# 

The core file location & format can be configured using the following command:

echo "/tmp/core_files/core.%p" > /proc/sys/kernel/core_pattern

If no directory is specified (as provided above - /tmp/core_files), the generated core file is stored in the current directory of the process.

%p above specifies the format of the core file name. Here are all the allowed format specifiers:

%p: pid  of the process dumped 
%u: uid of the process dumped
%g: gid of the process dumped
%s: signal number causing core dump
%t: timestamp of core dump. specified in seconds since seconds since 0:00h, 1 Jan 1970
%h: hostname (uname command output)
%e: executable filename

The file location & format configured above is not retained across reboots. To keep the configuration even across reboots, the above config should be done in /etc/sysctl.conf. Add "kernel.core_pattern=/tmp/core_files/core.%p" to /etc/sysctl.conf.

The currently configured core file location & format can be viewed using the following command:

root@babu-VirtualBox:~/tools/core_dump# cat  /proc/sys/kernel/core_pattern
core
root@babu-VirtualBox:~/tools/core_dump#

or

root@babu-VirtualBox:~/tools/core_dump# sysctl -a | grep core_
kernel.core_pattern = core
kernel.core_pipe_limit = 0
kernel.core_uses_pid = 1
root@babu-VirtualBox:~/tools/core_dump# 

In addition to the above commands, I found two more core dump related configuration facilities:

root@babu-VirtualBox:~/tools/core_dump# cat /proc/sys/kernel/core_pipe_limit 

0

root@babu-VirtualBox:~/tools/core_dump# cat /proc/sys/kernel/core_uses_pid 

1 <-- Impact of a non-zero value is same as %p format specifier in /proc/sys/kernel/core_pattern 

root@babu-VirtualBox:~/tools/core_dump# 

or

root@babu-VirtualBox:~/tools/core_dump# sysctl -a | grep core_

kernel.core_pattern = core

kernel.core_pipe_limit = 0

kernel.core_uses_pid = 1

root@babu-VirtualBox:~/tools/core_dump# 

I am yet to explore the purpose of these configuration facilities. 

How to manually generate a core dump?

• CLI method:

root@babu-VirtualBox:~/tools/core_dump# ls

latencytop  latencytop.c  latencytop.o  Makefile

root@babu-VirtualBox:~/tools/core_dump# pidof latencytop 

2832

root@babu-VirtualBox:~/tools/core_dump# gcore 2832

0xb76e5424 in __kernel_vsyscall ()

Saved corefile core.2832

root@babu-VirtualBox:~/tools/core_dump# ls

core.2832  latencytop  latencytop.c  latencytop.o  Makefile
root@babu-VirtualBox:~/tools/core_dump# 

Note: This method generated core file irrespective of ulimit configuration.

or

root@babu-VirtualBox:~/tools/core_dump# ls
ex  latencytop  latencytop.c  latencytop.o  Makefile
root@babu-VirtualBox:~/tools/core_dump# kill -s SIGSEGV `pidof latencytop`  (or SIGABRT instead of SIGSEGV)
root@babu-VirtualBox:~/tools/core_dump# ls
core.3669  ex  latencytop  latencytop.c  latencytop.o  Makefile
root@babu-VirtualBox:~/tools/core_dump# 

Note: This method generated core file only if ulimit is configured to a non-zero size.

• GDB method:

root@babu-VirtualBox:~/tools/core_dump# ls

latencytop  latencytop.c  latencytop.o  Makefile

root@babu-VirtualBox:~/tools/core_dump# pidof latencytop 

2832

root@babu-VirtualBox:~/tools/core_dump# gdb -p 2832

GNU gdb (GDB) 7.6.1-ubuntu

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "i686-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>.

Attaching to process 2832

Reading symbols from /home/babu/tools/core_dump/latencytop...done.

Reading symbols from /lib/i386-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug/lib/i386-linux-gnu/libc-2.17.so...done.

done.

Loaded symbols for /lib/i386-linux-gnu/libc.so.6

Reading symbols from /lib/ld-linux.so.2...Reading symbols from /usr/lib/debug/lib/i386-linux-gnu/ld-2.17.so...done.

done.

Loaded symbols for /lib/ld-linux.so.2

0xb76e5424 in __kernel_vsyscall ()

(gdb) generate-core-file

Saved corefile core.2832

(gdb) quit

A debugging session is active.

            Inferior 1 [process 2832] will be detached.

Quit anyway? (y or n) y

Detaching from program: /home/babu/tools/core_dump/latencytop, process 2832

root@babu-VirtualBox:~/tools/core_dump# 

root@babu-VirtualBox:~/tools/core_dump# ls

core.2832  latencytop  latencytop.c  latencytop.o  Makefile

root@babu-VirtualBox:~/tools/core_dump# 

Note: This method generated core file irrespective of ulimit configuration.

How to programmatically generate a core dump from inside a process?

• Invoke abort where ever core dump should be generated. Ofcourse, the process gets terminated after generated the core. Another point to remember - System should have been configured to generate core dumps. Otherwise, abort will not generate core.
• Another approach to generate core dump without terminating the process is to fork the process, let the child invoke abort & let the parent continue with normal execution.

Note: This method generated core file only if ulimit is configured to a non-zero size.


Steps to ANALYZE core dump file offline

gdb is used to analyze core dump file. After using the steps required to load the core file into gdb, all gdb commands can be used to inspect/debug the core file. 

Step1:
Identify the name of the application from the core file.

If %e not configured in /proc/sys/kernel/core_pattern, then the core file name itself indicates the application that generated the core. If not, then we can use the following command to get the application name:

root@babu-VirtualBox:~/tools/core_dump# file core.5240 
core.5240: ELF 32-bit LSB core file Intel 80386, version 1 (SYSV), SVR4-style, from './latencytop'
root@babu-VirtualBox:~/tools/core_dump# 

or

root@babu-VirtualBox:~/tools/core_dump# strings core.5240 | tail -n 1
./latencytop
root@babu-VirtualBox:~/tools/core_dump# 

Step2:
To debug a core file using gdb, we need binaries with debug symbols. However, in embedded systems, debug disabled or debug-symbol-stripped binaries are used in QA environment and customer deployments. So, its most likely that we first need to get a debug enabled application binary and shared/static libraries. We also need to ensure that the code from which we build debug binary is same as the code of the binaries on which the software issue is observed.

To demo the core dump tool, let me use the application latencytop (https://github.com/babuneelam/gcov_uspace_tests/tree/master/latencytop). I have enabled gdb flags in the Makefile as well. And I then generated a core file using kill command.

Step3:
Feed the core file & debug-enabled/unstripped binary to gdb to debug the core file:

root@babu-VirtualBox:~/tools/core_dump# gdb latencytop 

GNU gdb (GDB) 7.6.1-ubuntu

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "i686-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>...

Reading symbols from /home/babu/tools/core_dump/latencytop...done.

(gdb) set solib-search-path /lib/i386-linux-gnu/  <--- this can be a list of colon separated PATHs

(gdb) core-file core.3678 

[New LWP 3678]

warning: Can't read pathname for load map: Input/output error.

Core was generated by `./latencytop'.

#0  0xb7725424 in __kernel_vsyscall ()

(gdb) 

(gdb) bt

#0  0xb7725424 in __kernel_vsyscall ()

#1  0xb7612740 in __nanosleep_nocancel ()

    at ../sysdeps/unix/syscall-template.S:81

#2  0xb7612563 in __sleep (seconds=0) at ../sysdeps/unix/sysv/linux/sleep.c:137

#3  0x08048d81 in main (argc=1, argv=0xbfd2a354) at latencytop.c:173

(gdb) list latencytop.c:173
168
169 //abort();
170
171    while ((iterations == 0) || (count++ < iterations)) {
172
173        sleep(delay);
174
175        e = NULL;
176        if (pid) {
177            if (tid) {
(gdb) 

solib-search path provides the location of the shared libraries the binary uses. Without providing info about where the shared libraries are, the stack trace wouldn't display the symbol names that help corner down the crashing code path. In this case, I am setting shared library path to /lib/i386-linux-gnu/. However, i may different in different systems. For embedded systems where the development & build machines are different, developers need to spend significant effort in setting up the library path. This would be even more challenging if the build system were not to place all shared libraries in common build directory. 

How would the stack trace be if we don't feed the location of libraries to gdb during core dump analysis:

root@babu-VirtualBox:~/tools/core_dump# gdb latencytop

GNU gdb (GDB) 7.6.1-ubuntu

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "i686-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>...

Reading symbols from /home/babu/tools/core_dump/latencytop...done.

(gdb) set solib-search-path /lib1/   <--- Providing incorrect shared library path for demonstration purpose

(gdb) core-file core.3678 

[New LWP 3678]

warning: Can't read pathname for load map: Input/output error.

warning: Could not load shared library symbols for 2 libraries, e.g. /lib/i386-linux-gnu/libc.so.6.

Use the "info sharedlibrary" command to see the complete listing.

Do you need "set solib-search-path" or "set sysroot"?

Core was generated by `./latencytop'.

#0  0xb7725424 in __kernel_vsyscall ()

(gdb) bt

#0  0xb7725424 in __kernel_vsyscall ()

#1  0xb7612740 in ?? ()     

#2  0xb7612563 in ?? ()

#3  0xbfd2a0bc in ?? ()

#4  0x00000000 in ?? () <--- symbol names are not displayed!! As we can see, this is not limited to just the symbols of shared library, but even impacted symbols of the binary !!

(gdb) 

How would the stack trace be if we don't compile the binary with debug (-g) flag or strip the debug symbols:

root@babu-VirtualBox:~/tools/core_dump# gdb latencytop

GNU gdb (GDB) 7.6.1-ubuntu

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "i686-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>...

Reading symbols from /home/babu/tools/core_dump/latencytop...(no debugging symbols found)...done.

(gdb) set solib-search-path /lib/i386-linux-gnu/

(gdb) core core.5188 

[New LWP 5188]

warning: Can't read pathname for load map: Input/output error.

Core was generated by `./latencytop'.

Program terminated with signal 11, Segmentation fault.

#0  0xb77b5424 in __kernel_vsyscall ()

(gdb) bt

#0  0xb77b5424 in __kernel_vsyscall ()

#1  0xb76a2740 in __nanosleep_nocancel ()

    at ../sysdeps/unix/syscall-template.S:81

#2  0xb76a2563 in __sleep (seconds=0) at ../sysdeps/unix/sysv/linux/sleep.c:137

#3  0x08048d81 in main ()  <-- With debug flags enabled, we even got info about file & line number - "latencytop.c:173".. This is missing without gdb flags!!

(gdb) 

TBD

solib-absolute-prefix - TBD

Core dump tool - Internals

Following is the sequence of events/code-flows that create a core dump of a process:

• User space application encounters an fatal-error/exception. This leads kernel to raise a signal for this process. This signal could be raised manually using kill command or methods as well. 
• Kernel basically sets a flag in process descriptor indicating that a signal is raised & it's handling is pending. 
• Kernel then handles the signal. Signal handling is done by kernel every time the process is resumed for execution in user space. Today, kernel has this resuming opportunity in scheduler and interrupt-handler context. So, in these two places, kernel checks for pending signals of the process.
• If there are pending signals, then kernel invokes default signal handler: If the signal were compile-time configured to dump registers, then core dump is generated. User space process is not invoked at all in accomplishing this. If a custom signal handler were registered, then custom handling instead of core dump is done.
• The signals for which core dump is compile-time configured can be obtained from http://lxr.linux.no/linux+v3.12.6/include/linux/signal.h#L331.
• signal.c generates core dump in do_coredump function. To accomplish this, this function in turn invokes the registered binary format(say ELF)'s core dump function handler as can be seen here.

References:
http://ss64.com/bash/ulimit.html
http://man7.org/linux/man-pages/man2/getrlimit.2.html
http://man7.org/linux/man-pages/man5/core.5.html