Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Uniqueness of xunit collection fixtures
jbartel
#1 Posted : Wednesday, December 9, 2020 7:19:12 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 7/1/2020(UTC)
Posts: 1
Location: Germany

Thanks: 1 times
Hi there,

I am currently facing a problem with an ICollectionFixture in xunit and I am not sure if this is an issue in the NCrunch engine or if I am simply missing some magic configuration I did not discover yet.

The fixture is shared with all test classes through a collection, so I would expect it to behave like a singleton. When executing with Visual Studio, it behaves as I would expect (only one instance of the fixture is instantiated throughout test execution).
But running with NCrunch, the fixture gets instantiated multiple times*). Parallel test execution is explicitly disabled. I could strip this behavior down to a project with only the following code:

Code:

[Collection("TestCollection")]
public class UnitTest1
{
    private readonly Fixture fixture;

    public UnitTest1(Fixture fixture)
    {
        this.fixture = fixture;
    }

    [Fact]
    public void Test1()
    {

    }
}

[Collection("TestCollection")]
public class UnitTest2
{
    private readonly Fixture fixture;

    public UnitTest2(Fixture fixture)
    {
        this.fixture = fixture;
    }

    [Fact]
    public void Test2()
    {

    }
}

public class Fixture
{
    private static int counter = 0;
    public Fixture()
    {
        int newCount = Interlocked.Increment(ref counter);
        if (newCount > 1)
            throw new Exception($"{newCount} instances? WTF???");
    }
}

[CollectionDefinition("TestCollection")]
public class TestCollectionDefinition : ICollectionFixture<Fixture>
{
}




Thanks for any support,
Julian

*) More details on NCrunch behavior based on the sample code:
- When starting all tests in Debug, it works.
- When all tests have the same state before execution, the execution works too.
- When one test is red, one is green, the Exception in the fixture's ctor is thrown on that test that previously was green. Further executing all tests lets both facts toggle between success and failure, meaning: The test, which was in failing state gets the first fixture instance, the succeeding test gets the second fixture instance. Feels like it depends if there are multiple execution queues/priorities of the tests?

P.S.: NCrunch version is 4.6.0.3.
Remco
#2 Posted : Wednesday, December 9, 2020 11:01:04 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,248

Thanks: 810 times
Was thanked: 1077 time(s) in 1022 post(s)
Hi, thanks for posting.

This is probably happening because of NCrunch's batching. NCrunch separates tests into tasks, which can be seen in the processing queue. Each task results into a new call into the test framework, with instructions to run a set of tests. In theory this could result in a test collection being instantiated multiple times in a given test process, as the test process is used for multiple test runs (see test runner re-use).

You have a couple of options here. Either use static variables to determine when the code in your singleton is being re-executed (thus ensuring it isn't), or try marking the tests involved with NCrunch.Framework.IsolatedAttribute.
1 user thanked Remco for this useful post.
jbartel on 12/14/2020(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.224 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download