Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

NCrunch does not ignore xunit theory data source
stanac
#1 Posted : Saturday, April 26, 2025 9:03:03 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/7/2019(UTC)
Posts: 9
Location: Serbia

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
Problem I'm having is that some of my tests are xUnit theory with data source of hundreds of cases. I am testing more or less all variations of configuration from production. Those variations are loaded from disk and theory itself is slow to execute even for a single case. Also loading from disk involves parsing and validation of data. For those reasons I am ignoring those theories in nCrunch configuration. Problem is that nCrunch is not ignoring data source, it will load all cases but not run the tests.

As an example nCrunch takes more than 10 seconds to execute following tests, even though "LargerThan1" theory is ignored.

Could you add configuration to ignore data source for ignored theories? Or maybe something like this already exists and I don't know about it.

Quote:

public class UnitTest1
{
[Fact]
public void Test1()
{
int sum = 1 + 1;
Assert.Equal(2, sum);
}

[Theory]
[ClassData(typeof(TestDataSource))]
public void LargerThan1(int value)
{
Assert.True(value > 1);
}
}

public class TestDataSource : IEnumerable<object[]>
{
public IEnumerator<object[]> GetEnumerator()
{
for (int i = 2; i < 13; i++)
{
Thread.Sleep(1000);
yield return [i];
}
}

IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
Remco
#2 Posted : Saturday, April 26, 2025 12:22:22 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,277

Thanks: 983 times
Was thanked: 1323 time(s) in 1227 post(s)
Hi, thanks for sharing this issue.

Can you confirm which version of Xunit you are running? Also, when you ignore these theories using NCrunch configuration, could you clarify which configuration option you are using?
stanac
#3 Posted : Tuesday, April 29, 2025 9:42:52 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/7/2019(UTC)
Posts: 9
Location: Serbia

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
I am using ncrunch 5.14.0.5, xUnit 2.9.3 and if it means anything xunit.runner.visualstudio 3.0.2.

By ignore configuration I mean right clicking on code coverage circle in code editor and clicking on Ignore covering test.
Remco
#4 Posted : Tuesday, April 29, 2025 11:06:59 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,277

Thanks: 983 times
Was thanked: 1323 time(s) in 1227 post(s)
Thanks for clarifying this.

The problem you're encountering is caused by a lack of clarity between test discovery and test execution.

Theories like the one you've described are generated using data that is encountered during test discovery. Test discovery is a special step that is performed by NCrunch just after your assembly is built. In the case of Xunit2, this discovery is performed in a runtime domain that physically needs to execute your code in order to work out which tests exist in your project.

Because you've added code into the discovery step that is slow to execute, the discovery step will always be slow and ignoring the tests using NCrunch's configuration will have no effect on this. The discovery process itself is a necessary step and can't be skipped, and it isn't possible to selectively stop parts of it without editing the code. The reasons for this are twofold:

1. To be able to skip tests, we need to know that they exist. We can't know they exist until they are discovered.
2. Most of this process is actually handled by the test framework (Xunit2 in this case), so NCrunch has only limited control over it.

The best workaround for this is to use a compiler condition to strip the attributes from the code involved, so that the Xunit discovery system will simply pass over it. For example:

Code:

#if !NCrunch
[Theory]
[ClassData(typeof(TestDataSource))]
#endif
public void LargerThan1(int value)
{
Assert.True(value > 1);
}


2 users thanked Remco for this useful post.
stanac on 5/5/2025(UTC), talbrecht on 5/5/2025(UTC)
stanac
#5 Posted : Monday, May 5, 2025 9:03:19 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/7/2019(UTC)
Posts: 9
Location: Serbia

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
Thanks and sorry for late reply. Conditional compilation solved my issue.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.040 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download