Build/Test Issues

Fail tests when the fixture dispose throws

Started by aignas on 4,011 views

Hi,

I'm using XUnit with NCrunch (both are latest versions) and everything is really nice.

However, I recently had a hard to debug issue where the fixture was throwing in Dispose but the test was still marked as passed. Because it was not the behaviour on the build server, it was rather strange. I did not find an option to fail the tests if the fixture disposal throws, but it would be good to have something like this. For certain integration tests it is really important to have good disposal in the test fixtures, otherwise we may leak resources and crash not only NCrunch, but the infrastructure.

Something that illustrates what I mean:

public class TestFixture: IDisposable
{
    public void Dispose()
    {
        throw new NullReferenceException();
    }
}

public class TestsDependingOnFixture : IClassFixture<TestFixture>
{
    public TestsDependingOnFixture(TestFixture fixture)
    {
    }

    [Fact]
    public void ExampleTest()
    {
    }
}

public class TestFixtureTests
{
    [Fact]
    public void TestFixtureDisposal()
    {
        using(new TestFixture())
        {
            // should not throw
        }
    }
}

Edited

Hi, thanks for sharing this problem.

You've managed to find a hole in NCrunch's handling here.

Xunit reports this problem using a specialised message, and we haven't been listening for it. The result is that NCrunch ignores the exception and pretends that it doesn't exist.

A fix for this will be out with NCrunch v3.18. Thanks for taking the time to report this problem.

Post a reply

Log in to reply.