When I worked with Cisco's product development, I used an in-house code coverage tool. Recently, I also came across c/linux based code coverage tools - gcov/lcov. In this post, I will cover basics of Dynamic Code Coverage.
Purpose of Dynamic Code Coverage
Dynamic Code coverage tools help to understand the amount of code executed when a test/test suite is executed. Using the code coverage information, the effectiveness of test/ test suite can be improved through test suite optimizations. By achieving the code coverage standards an organization sets internally for itself, the confidence in the product/feature quality is improved. Code coverage is typically used as a pre-condition at three stages of product development cycle:
- Engineering releases typically use certain coverage criteria as a pre-condition before a product/release is rolled out to customers.
- When new features are developed internally, new feature testing & automation engineers use code coverage to prove the effectiveness of the test suite used for testing the feature. Usually, meeting an internal code coverage standard would be a pre-condition to move the feature from experimental branch to a mainline branch & add the respective test suite test suite to regression test suites.
- Development engineers too use code coverage during their functional/unit testing so that they can proactively ensure the quality of their code deliverable to QA teams.
So primarily , code coverage is a Quality Assurance tool.
Secondarily, code coverage can help in understanding the execution frequency of different code blocks & hence provide an opportunity to profile/optimize the code.
Typical Steps in Obtaining Dynamic Code Coverage
- Instrument code: The code for which coverage info is sought is instrumented at compile time: Every line of source code is tracked using a counter to count the number of times the line is executed/traversed, every branch is tracked with branch path tracking counter, etc etc..
- Execute test suite(s): When such instrumented code is tested, runtime coverage information is generated during or after completion of execution.
- Collate all pieces of code coverage info: source code, compile time instrumentation info, runtime coverage info.
- Create intuitive code coverage report: In this step, code coverage reporting tools combine knowledge of code, compile time instrumentation information and runtime coverage information to present code coverage details in an intuitive manner.
- Evaluate against organizational code coverage standards: If coverage standard are met, then move on to next stage in product development cycle. If not, improve test suites, Go back to Step1.
Dynamic Code Coverage Information - Categories
Following is some possible coverage information:
- Statement coverage: Is every statement covered?
- Condition coverage: Is every boolean condition evaluated to true/false?
- Branch coverage: Is every branch/path if/else and switch/case covered?
- Decision coverage: Is every decision covered?
- Function Coverage: Is every function covered?
- Function Entry/Exit coverage: Is every path of entering a function covered? Is every path of exiting a function covered?
Misconception about Dynamic Code Coverage
Engineering and QA teams might show a tendency to perceive the covered code as COMPLETELY safe or bug/error free. However, this perception is incorrect because doesn't imply testing logical aspects of the code.
The appropriate way of interpreting dynamic code coverage is
- to focus on uncovered code to verify the adequacy of test suites & improve test suites for increased coverage.
- to treat covered code information as just one input in assuring code quality and continue to leverage other available methods/techniques for further improvement in code quality
More..
In my upcoming posts, I am covering gcov/lcov - C & Linux based Dynamic Code Coverage Tools, How to - gcov/lcov for linux user space process and How to - gcov/lcov for linux kernel modules.
Hope this helps.
No comments:
Post a Comment