Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Call coverage #23

Open
MMelzig opened this issue Mar 2, 2017 · 2 comments
Open

Feature request: Call coverage #23

MMelzig opened this issue Mar 2, 2017 · 2 comments

Comments

@MMelzig
Copy link

MMelzig commented Mar 2, 2017

lcov is able to show:

  • line coverage
  • branch coverage
  • function coverage

I understand function coverage to which percentage functions are entered and thereby tested.

From my point of view it is very helpful to be able to measure to which percentage calls to function(s) has been tested. The call coverage.

An example for the difference and for the need of this metric. Let's assume we have one function A, which calls another function B. To know that function B is entered (at least once = function coverage) is one side. But when the function B is called from 3 different locations in function A and in each of these calls a mistake can be made e.g. with parameter order there is an additional need to know whether each of the existing calls are applied = Call coverage.

Best regards
MMelzig

@oberpar
Copy link
Contributor

oberpar commented Mar 3, 2017

Interesting idea. I haven't given call coverage much thought yet. It appears that gcov provides all the required data when running it with option -b (which LCOV already does for getting branch coverage). I can see the benefit of having this function added to LCOV, though I don't think I will be getting to implement this any time soon, but patches are of course welcome.

Let's think the interface through for a moment:

  • Call coverage data would need to be added to the .info file format. Something like CLDA:<line number>,<call number>,<taken>, based on gcov's "call x returned y" data
  • The data could be visualized either like function coverage data, or like branch coverage data. That is, either:
    a) There should be a page listing all call sites in a source code file similar to the page for function coverage. This could be a table with columns for Line number, call number, source code for that line, followed by execution count
    or
    b) Similar to the "Branch data" column in source code view, there should be another column "Call data" that shows coverage of calls in a line similar to how branch coverage data is shown, with colored [+ -] signs.
  • Page headers would need to show call coverage rates for each file and directory
  • The top level and directory level views should contain another column for call coverage rates per file
  • There should be an option to turn off call coverage, something like a lcov_call_coverage= directive in the lcovrc file

Thoughts and comments on this are welcome.

@MMelzig
Copy link
Author

MMelzig commented Mar 6, 2017

The thoughts concerning the interface are really complete. But the part "b) Similar to the "Branch data" column in source code view, there should be another column "Call data" that shows..." can blow up the report. A simple line can have several calls as well as each condition in an e.g. if statement can include calls.

My intention for this feature request is to give tester and quality manager an overview of the reached call coverage together with statement and branch coverage at the end of a development / test phase. I expect that the analysis of missing test cases is done on the basis of the statement and branch coverage mainly. If there is a lack of call coverage this can be found in the not covered statements and branches, which should not be that much when we come to the end of the test.

So for my intention an implementation of the annotation in the code can be postponed until somebody really ask for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants