Code coverage flakiness

I'm running the latest version of NCrunch but I'm still seeing some unreliablity in terms of code coverage indicators (typically saying lines aren't covered when they are). I've also seen a green X icon where it seems to have been confused about what used to be an exception.

In the latest instance it was a very simple project but using SpecFlow+NUnit. It seems to fix itself when I tell NCrunch to reset and haven't figured out a reliable repro yet.

Any thoughts? Thanks!

J

Thanks for sharing the details about this.

I've uncovered a mapping problem that can cause code coverage instability and code coverage loss in a similar way to what you describe. It occurs when an unopen code file is changed on disk before more work is done (and tests are run) using NCrunch in the IDE. This could happen as the result of a VCS action, or by a refactoring operation (i.e. 'Rename'). A fix is being prepared for the 1.45 maintenance release due out in a few weeks time.

What I'm curious about is whether the issue you've experienced is a result of the above, or if it's another mapping problem entirely. One way we can tell is by checking for NCrunch log file exceptions when you notice the code coverage is out of alignment. When this next happens to you, can you submit a bug report with a link back to this forum post in the description? I'll then see if there is a clear indication of whether the problem has been solved, or whether this is a new one!


Cheers,

Remco

For anyone interested, a fix for this problem has been introduced with the newly released version of NCrunch (v1.45).