Tuesday, January 7, 2014

gcov/lcov - C & Linux based Dynamic Code Coverage Tools


Gcov & Lcov - Overview


gcov is a GNU code coverage tool. gcov provides three basic functionalities:

  • Compile time instrumentation facility
    • This functionality is built into gcc & can be enabled with a couple of gcc options. So, if you have gcc installed, you get this portion automatically.  
  • Run time coverage data collection engine
    • For user space applications, a gcc packaged library "libgcov" provides this functionality & hence should be linked with applications. As this library comes pre-installed with gcc, you get this portion too automatically. 
    • However, for kernel space applications, data collection engine comes through additional kernel code which can be enabled by turning on a compile time kernel flags. So, to get run time coverage info for linux kernel or kernel modules, linux kernel should be recompiled with these additional kernel compilation flags.
  • Code coverage reporting Interface:  
    • For this functionality, GNU provides a command line utility 'gcov'. However, gcov's cli based user interface is not intuitive. 
    • For a more intuitive and GUI based interface, lcov can be used to interpret gcov's coverage information. Let me also delve a bit into lcov's origins. LTP(Linux Test Project), an open source initiative, developed lcov. LTP's goal is to develop a comprehensive test suite for Linux OS. To identify areas of improvement in LTP test suite, LTP uses code coverage as one of the criteria. To help aid in code coverage analysis, LTP developed lcov tool by extending gcov.

gcov is pretty easy to use & gives great results. 
 
User Space Applications - Usage Steps 


Here are the usage steps:
  • Instrument Application(s) Code
    • Recompile the sources with gcov compilation flags: -fprofile-arcs -ftest-coverage. Link the binaries too with LDFLAGS: -fprofile-arcs -ftest-coverage. 
    • These two flags are synonymous with "--coverage" flag and also imply "-lgcov" during linking. 
    • libgcov provides the gcov's profiling functionality for user space applications.In my VM, I found libgcov installed at /usr/lib/gcc/i686-linux-gnu/4.8/64/libgcov.a, /usr/lib/gcc/i686-linux-gnu/4.8/libgcov.a and /usr/lib/gcc/i686-linux-gnu/4.8/x32/libgcov.a. So, as I mentioned earlier, libgcov comes along with gcc installation.
    • Post-compilation & linking, a .gcno file is created in the respective build directory for every c file instrumented with gcov.
  • Execute Application(s)' Test Suite(s)
  • Fetch run time coverage info: After the test case is executed, dump the gcov run time time coverage information. 
    • Run time coverage information is not dumped until the process is terminated. However, most of the embedded applications are background daemons which never terminate. So, I wrote a custom script - https://github.com/babuneelam/gcov_scripts/blob/master/dum_gcov. This script basically attaches to the process through gdb & flushes gcov run time data & then detaches from gdb. 
    • For every c file instrumented with gcov, a .gcda file is generated at run time.
  • Merge run time coverage info to Application(s)' build directories
    • For environments where build and testing are done in the same device, this step can be skipped as gcov dumps each application's run time data automatically into corresponding application's build directory. 
    • In case of embedded systems, the build server would be different from test machine. So, the gcov run time information should be copied from test machine to build server into applications' build directories. I have automated the steps of extraction & merge using a script - https://github.com/babuneelam/gcov_scripts/blob/master/fetch_n_merge_gcov. One can use this script from build machine in any build sub-directory to update corresponding directory's run time coverage info from the test machine. 
  • Convert raw gcov coverage info into lcov's format: At top build directory, run lcov tools - geninfo & genhtml. These tools convert raw gcov coverage information to such a format that a browser can be used present the data to the user.
  • Create intuitive Code coverage report: Any web browser can be used to view lcov data as a report. lcov report includes only those applications instrumented with gcov. So, engineers should NOT be hasty in concluding the coverage information based on just lcov report, but should cautiously check whether all the code expected to be compiled with gocv flags is compiled so. Any code that is not compiled with gcov flags should be categorized "unknown coverage".

Kernel or Kernel modules - Usage Steps

Here are the usage steps:
  • Enable run time coverage data collection engine:
    • To get code coverage for kernel modules, recompile the kernel with CONFIG_GCOV_KERNEL=y. To get code coverage even for entire linux kernel, recompile with CONFIG_GCOV_PROFILE_ALL=y. 
  • Instrument Application(s) Code

To enable profiling for specific files or directories, add a line similar to the following to the respective kernel Makefile:         
               For a single file (e.g. main.o):                 
                       GCOV_PROFILE_main.o := y         

               For all files in one directory:
                       GCOV_PROFILE := y 
To exclude files from being profiled even when CONFIG_GCOV_PROFILE_ALL is specified, use:
                       GCOV_PROFILE_main.o := n        
               and:
                       GCOV_PROFILE := n        

    • Post-compilation & linking, a .gcno file is created in the respective build directory for every c file instrumented with gcov.
  • Execute Application(s)' Test Suite(s)
  • Fetch run time coverage info: After the test suite(s) is executed, fetch run time coverage information: 
    • Run time coverage information of linux kernel or kernel modules can be accessed any time in debugfs at /sys/kernel/debug/gcov/. 
    • For every c file instrumented with gcov, a .gcda file is generated at run time. For a specific file, look for /sys/kernel/debug/gcov/path/to/compile/dir/file.gcda.
  • Merge run time coverage info to Application(s)' build directories
    • Run time coverage should be merged with the kernel or kernel module's build directory. Additionally, in case of embedded systems, data should be extracted from test machine to build machine. 
    • I have automated the steps of extraction & merge using a script - https://github.com/babuneelam/gcov_scripts/blob/master/fetch_n_merge_gcov. One can use this script from build machine in any build sub-directory to update corresponding directory's run time coverage info from the test machine. This works for both local test machine scenario or remote test machine scenario.
  • Convert raw gcov coverage info into lcov's format: At top build directory, run lcov tools - geninfo & genhtml. These tools convert raw gcov coverage information to such a format that a browser can be used present the data to the user.
  • Create intuitive Code coverage reportAny web browser can be used to view lcov data as a report. lcov report includes only those applications instrumented with gcov. So, engineers should NOT be hasty in concluding the coverage information based on just lcov report, but should cautiously check whether all the code expected to be compiled with gocv flags is compiled so. Any code that is not compiled with gcov flags should be categorized "unknown coverage".

Hope this helps.


References:



No comments:

UA-48797665-1