Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Code Coverage doesn't update on full rebuild.
Options
Previous Topic Next Topic
Micah71381
#1 Posted : Wednesday, January 1, 2014 9:55:39 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/18/2013(UTC)
Posts: 27
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
I have two unit test projects. One is a legacy project that is being phased out, the other is a new project with tests written in a more sound way. During normal development, all unit tests are run (both projects). When I am working on improving my unit test coverage, only the new project is run and the legacy project is disabled via NCrunch configuration (Ignore this component completely).

When I disable the legacy project, the NCrunch Metrics window still displays code coverage metrics from when both projects were enabled. If I open a class that has 0% coverage by the new test project and 60% coverage by the legacy project (which is ignored) I see all black dots for line coverage, suggesting that those lines are in fact not covered.

If I clean all build artifacts/caches from my solution directory (using git clean -dfx to wipe out anything that isn't checked in) and reload my solution with the legacy test project ignored, then the NCrunch Metrics window once again reports the correct numbers (0% coverage on classes that aren't covered by the new stuff). This lasts until I un-ignore the legacy test project and then I am back in the bugged state again.

Executing a Resynchronize does not resolve the issue.
Back to top
Remco
#2 Posted : Wednesday, January 1, 2014 11:01:33 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,982

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for reporting this issue.

I suspect this may be a hole in the synchronisation around the changing of the 'Ignore this project completely' setting. When this setting is turned on, NCrunch will completely avoid loading the project and analysing it for tests, but the solution state for the project (including its tests) would persist until the cache file is deleted. I've noted this down for a future fix.

If you are running V2 and you wishing to isolate the code coverage from your new test project, you may wish to try using the Tests Window to select all fixtures in the new project, then right click and choose to 'Show coverage for selected tests only'. This will give you a much sharper picture of your new code coverage without needing to use the 'Ignore this project completely' setting, which was never really designed with this sort of thing in mind.

Cheers,

Remco
Back to top
Micah71381
#3 Posted : Thursday, January 2, 2014 1:16:39 AM(UTC)
Rank: Member

Groups: Registered
Joined: 10/18/2013(UTC)
Posts: 27
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
I didn't know about v2. I installed it and attempted to do what you suggested and it works but when I add a new test I have to repeat the process again since it is not automatically included in the things to show coverage for by default. This is definitely better than my previous solution (disabling the legacy project) but it still leaves me wanting more.
Back to top
Remco
#4 Posted : Thursday, January 2, 2014 11:50:16 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,982

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Thanks for the feedback. I'm glad it's an improvement for you. I may also be able to suggest another way to narrow your metrics - by using ignored tests.

If you right-click on your legacy test project and choose to ignore all the tests within it, the code coverage for these tests will be purged and they won't show inside the metrics window. If the tests run fast enough, you could simply ignore/unignore them as necessary to juggle your metrics reporting with your continuous execution of the old tests.

There are some rough plans to make improvements on engine modes that will allow them to configure criteria for metrics and code coverage, but this is still a way out at the moment. I hope that when these features are implemented they may give you further options in this area.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.035 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download