Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Code coverage flakiness
James Crowley
#1 Posted : Sunday, March 10, 2013 8:32:28 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 12/4/2012(UTC)
Posts: 2
Location: United Kingdom

I'm running the latest version of NCrunch but I'm still seeing some unreliablity in terms of code coverage indicators (typically saying lines aren't covered when they are). I've also seen a green X icon where it seems to have been confused about what used to be an exception.

In the latest instance it was a very simple project but using SpecFlow+NUnit. It seems to fix itself when I tell NCrunch to reset and haven't figured out a reliable repro yet.

Any thoughts? Thanks!

J
Remco
#2 Posted : Sunday, March 10, 2013 8:44:08 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Thanks for sharing the details about this.

I've uncovered a mapping problem that can cause code coverage instability and code coverage loss in a similar way to what you describe. It occurs when an unopen code file is changed on disk before more work is done (and tests are run) using NCrunch in the IDE. This could happen as the result of a VCS action, or by a refactoring operation (i.e. 'Rename'). A fix is being prepared for the 1.45 maintenance release due out in a few weeks time.

What I'm curious about is whether the issue you've experienced is a result of the above, or if it's another mapping problem entirely. One way we can tell is by checking for NCrunch log file exceptions when you notice the code coverage is out of alignment. When this next happens to you, can you submit a bug report with a link back to this forum post in the description? I'll then see if there is a clear indication of whether the problem has been solved, or whether this is a new one!


Cheers,

Remco
Remco
#3 Posted : Saturday, March 30, 2013 10:19:13 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
For anyone interested, a fix for this problem has been introduced with the newly released version of NCrunch (v1.45).
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.030 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download