Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Incorrect Coverage Metrics for single-line if-statements
arjo_reich
#1 Posted : Friday, May 29, 2015 3:23:29 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 5/29/2015(UTC)
Posts: 1
Location: United States of America

So, the Code Cleanup Format at the place I'm at removes braces and inlines any single-lined if-statement. The other day I noticed that NCrunch counts both the evaluation and the "true-part" of a single-lined if-statement is counted as covered when it's not. Since a picture is worth a thousand words, I figure the following might be easier to understand...

https://goo.gl/photos/sV4B5MU1WcW5d3VL6
Remco
#2 Posted : Saturday, May 30, 2015 12:49:44 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,373

Thanks: 705 times
Was thanked: 878 time(s) in 835 post(s)
Hi, thanks for sharing this.

This is a known limitation of NCrunch that exists because the tool simplifies coverage down to individual lines, rather than multiple statements per line, as is often done with other coverage tools.

The limitation exists for two reasons:

1. The UI of NCrunch is designed for only line-level resolution. This could be overcome to some extent by splitting the markers (for example, you could have a 'half green' marker), though this is also likely to create different kinds of confusion.

2. It's hard to implement with a tool like NCrunch. I mean, insanely hard. This is because the core engine has extremely tight resource constraints due to the requirement to merge code coverage data from multiple test runs very quickly. Allowing lines of code to have multiple blocks of tracked coverage would require all routines involved to perform a multi-dimensional merge and storage of coverage data. The handling of code coverage in NCrunch is by far the most complex area of the product and introducing such a further dimension of complexity would likely be impossible to maintain without risking serious problems in the coverage handling. Excuses aside, I simply do not yet know how to implement it.
richardmoss
#3 Posted : Wednesday, May 25, 2016 7:10:14 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/1/2012(UTC)
Posts: 6

Was thanked: 2 time(s) in 2 post(s)
Apologies for resurrecting such an old thread, but I have encountered the same issue with regards to multiple branches on the same line not being picked up by NCrunch (mostly null coalescing or the null conditional in newer code or the odd if bool || bool), followed by MSTest hell over the missing blocks in a formal test run (I am required to have 100% code coverage, no if's, no but's).

In your reply above you mentioned (aside from the insane complexity of the feature) that you didn't actual have a solution. Since then have you come up with any ideas or do you have any plans for integrating this in the future? Is it possible to use Rosyln for this sort of stuff or is that total overkill? I will sanity check my tests when I think I'm done in future by putting them through MSTest informally, but as the whole point of NCrunch is that it is always there running your tests and giving you the metrics without any sort of manual intervention, it would be really useful if it could be more accurate in regards to code coverage.

Thanks;
Richard Moss
Remco
#4 Posted : Wednesday, May 25, 2016 1:07:34 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,373

Thanks: 705 times
Was thanked: 878 time(s) in 835 post(s)
Hi Richard,

I'm sorry, but there has been no change in the way this is handled by NCrunch. If I can find a feasible way to implement more detailed resolution of coverage in the tool in future, it will be implemented.
richardmoss
#5 Posted : Wednesday, May 25, 2016 3:33:43 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/1/2012(UTC)
Posts: 6

Was thanked: 2 time(s) in 2 post(s)
Remco;8774 wrote:
Hi Richard,

I'm sorry, but there has been no change in the way this is handled by NCrunch. If I can find a feasible way to implement more detailed resolution of coverage in the tool in future, it will be implemented.


Ah well, no worries. Thanks anyway!

Regards;
Richard Moss
jnm236
#6 Posted : Monday, April 23, 2018 6:11:03 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/25/2015(UTC)
Posts: 57
Location: United States of America

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
I was caught out by this too.
The ability to see coverage, erring on the side of caution if necessary, seems very important to me. Marking the whole line as covered rather than waiting for all branches is not cautious from my point of view.

I appreciate that it's difficult, but I don't feel safe without it.
1 user thanked jnm236 for this useful post.
Remco on 4/23/2018(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.055 seconds.