Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Incorrect coverage detection with NUnit TestCaseSource
Options
Previous Topic Next Topic
VNicholson
#1 Posted : Thursday, December 19, 2013 1:43:41 AM(UTC)
Rank: Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 25
Location: New Zealand

Thanks: 7 times
Was thanked: 3 time(s) in 3 post(s)
I use a number of TestCaseSource based unit tests that use reflection to cover a set of common functionality in some of my classes. NCrunch 2.2.0.2 (in Visual Studio 2012) seems to have issues with coverage colouring and population of the 'Tests Covering' context menu for these tests.

In a single TestFixture I have two source private static IEnumerable<TestCaseData> properties and four test functions each referencing one of those properties with a TestCaseSource attribute. I get appropriate colouring and 'Tests Covering' menus for one of those four test functions, but the other three are wrong. The three that aren't working appear with the 'no coverage' colouring and while they have non-zero 'Tests Covering' menu headers, none of the tests are listed therein (see screenshot).

(- BROKEN LINK -)[img](- BROKEN LINK -)[/img][/url]

If I separate the four test functions into four separate test fixtures, this seems to work okay.

Is there a bug or known limitation here that means I can't have more than one function in a test fixture that is decorated with a TestCaseSource attribute?
Back to top
 |  Edit by user
Remco
#2 Posted : Thursday, December 19, 2013 2:13:49 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi, thanks for sharing this issue!

The only thing that springs to mind is that tests re-using the same TestCaseSource have a higher chance of being described/generated under the same name as each other. Do the tests show uniquely inside the NCrunch Tests Window? Because NCrunch needs a certain way to identify tests over multiple execution runs, it's important that tests are named uniquely.

Is there any chance you could share some sample code with me? A short compiling snippet showing two of the problem tests and the TestCaseSource they are using would be really great - I'm sure I could give a clear explanation of what is happening with this, and perhaps arrange a fix if possible.


Cheers,

Remco
Back to top
VNicholson
#3 Posted : Thursday, December 19, 2013 2:47:03 AM(UTC)
Rank: Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 25
Location: New Zealand

Thanks: 7 times
Was thanked: 3 time(s) in 3 post(s)
Thanks for the quick response Remco, your theory was spot on. I was using a Type name as the test name and because four test methods were sharing only two sources there was a lot of overlap going on. I refactored things so that I have four test case sources now, each with a unique name prefix and the coverage colouring and 'Tests Covering' menus are working as expected.
Back to top
 |  Edit by user
Remco
#4 Posted : Thursday, December 19, 2013 2:49:54 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Great stuff! Good work on the refactoring :)

Cheers,

Remco
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.030 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download