Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Listed tests that cover code not correct
chrisohio
#1 Posted : Friday, April 15, 2016 1:44:25 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/15/2016(UTC)
Posts: 2
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
This also causes the lines of code to show green in Ncrunch even when NCrunch says the unit tests are failing. I have included a sample project. Try making all the tests pass then make a different test fail then look at the test coverage in the non-test code file.

samplesourcecode
Remco
#2 Posted : Friday, April 15, 2016 5:56:16 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this code sample.

In this instance, the code coverage data shown by NCrunch is physically correct, because it represents the true path of execution of each test.

Note that the ClassInitialize attribute doesn't work when applied on base classes under MSTest, so it is not executed by the fixture. Moving this method onto the MappingTests class will give you a more meaningful result (NCrunch will report Mapper.cs as covered by MappingTests.*, which is the fixture itself).

Because you've worked around this limitation by calling the ClassInit method directly from your inherited TestInitialize, the coverage of contents of this method thus falls under the actual tests themselves, rather than their fixture. Because the entry to the method is gated by a static boolean flag (_classInitCalled), the method will only be executed by the first test to execute within the test runner's application domain. Every test executed later within the same domain will simply ignore the method because the boolean flag has already been set.

You can actually reproduce this behaviour using a debugger. Try running the whole fixture with a debugger attached and a breakpoint set on the ClassInit method - you'll notice the method gets called only once, and only by the first test to run. NCrunch is recording and reporting this exactly as it happens.

The confusion here stems from the difference between what I like to called 'physical' code coverage, and 'logical' code coverage. Physical code coverage is representative of the actual code path executed by a test when it runs, where logical code coverage is representative of all elements of code that contribute to the execution result of a test. Differentiating between these types of code coverage requires an in-depth understanding of the code itself and how it is designed to work - such an understanding is currently computationally impractical for a tool such as NCrunch, so NCrunch's code coverage is physical.
chrisohio
#3 Posted : Friday, April 15, 2016 2:01:42 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/15/2016(UTC)
Posts: 2
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
Great response that helped me to better understand how NCrunch is working. I refactored my tests (of course while using NCrunch to give me instant feedback on my re-factoring!) and now I am getting the results I wanted to get originally. Thanks this is a great tool.
1 user thanked chrisohio for this useful post.
Remco on 4/15/2016(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.030 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download