Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
NCrunch thinks a shared test file is a single item
Options
Previous Topic Next Topic
damianh
#1 Posted : Monday, May 5, 2014 10:03:47 AM(UTC)
Rank: Member

Groups: Registered
Joined: 5/5/2014(UTC)
Posts: 23
Location: Netherlands

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
(I've browsed the forums but could find anything related to this. Sorry if I missed it.)

If I have a Test class file that lives in one project but I include it in another project ( 'Add as link' ) and the test fails, NCrunch seems to think it is a single test file and/or messes up the groupings.

Sample repro soluton: https://github.com/damianh/NCrunchSharedTestsBug
Test failures as seen by resharper: (- BROKEN LINK -) - 2 failing tests
Test failures as seen by ncrunch: (- BROKEN LINK -) - 1 failing test

Anything wrong on my end, or is it a ncrunch bug?

Cheers.
Back to top
Remco
#2 Posted : Monday, May 5, 2014 10:32:25 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this issue.

The root of the issue is that you have two tests that exist under the same namespace, class and method. NCrunch uses a combination of these three items to uniquely identify the test. Because the test is being discovered twice, the second instance is being discarded by NCrunch. This is a result of limitations with the current design, which presently isn't easy to change/fix.

Are you seeing this issue as a result of cross-compiling the same file onto different platforms? If so, you may be able to use a compiler conditional to work around the issue. For example:

#if PLATFORM1
public class Class1
#else
public class Class1_2
#endif
{
[Test]
public void AssertTrueIsFalse()
{

}
}
Back to top
damianh
#3 Posted : Monday, May 5, 2014 10:46:34 AM(UTC)
Rank: Member

Groups: Registered
Joined: 5/5/2014(UTC)
Posts: 23
Location: Netherlands

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
Thanks for the quick response Remco.

No, I'm not use this to target different platforms, though I can see that as another use case. Here I have acceptance tests that I share across different projects but where each project has a different setup / fixture. Think of it as multiple different implementations of a service that if I write a test for one, I want it shared for all. Maybe me it's me, but I tend to have a lot of scenarios where I do this ( https://github.com/NEventStore ) and NCrunch is not working as desired, unfortunately.

I'll try the compilation symbols as a workaround for now, but that's not desirable long term. May I suggest you use the project (or assembly) in conjunction with namespace, class and method to uniquely identify a test?
Back to top
 |  Edit by user
Remco
#4 Posted : Monday, May 5, 2014 11:11:47 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for sharing the link to your project code. Your approach to this problem makes good sense.

I was going to suggest perhaps looking at alternative ways of structuring these tests to ensure their names are unique, for example, having a base class with several inheritors, each of which introduces the specialised code needed for each scenario. But looking at the major point of difference in your code (the app.config) and the number of classes involved, I think that the compiler conditional workaround is definitely the best way to go. You could use the conditional on the namespace containing these fixtures, which would save time+code. It's messy, but it will get you working.

I agree that the current method of identifying tests in NCrunch is not correct. I would like to change this also. As it's something quite fundamental, it's hard to make promises on this .. but I will look into it to see what the options are.
Back to top
damianh
#5 Posted : Monday, May 5, 2014 11:16:54 AM(UTC)
Rank: Member

Groups: Registered
Joined: 5/5/2014(UTC)
Posts: 23
Location: Netherlands

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
> You could use the conditional on the namespace containing these fixtures, which would save time+code. It's messy, but it will get you working.

Yep, got it working, and yep it's messy ;)

> I agree that the current...

That's reasonable (it's not a showstopper after all)
Back to top
1 user thanked damianh for this useful post.
Remco on 5/5/2014(UTC)
Remco
#6 Posted : Monday, May 12, 2014 9:32:10 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Back to top
damianh
#7 Posted : Tuesday, May 13, 2014 7:18:18 PM(UTC)
Rank: Member

Groups: Registered
Joined: 5/5/2014(UTC)
Posts: 23
Location: Netherlands

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
Can confirm that it works. Base on the repro solution, I now see 2 failing tests in the NCrunchTests window. Cheers!
Back to top
1 user thanked damianh for this useful post.
Remco on 5/13/2014(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.052 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download