Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

[TestFixtureSetUp] runs twice with [TestCase] and [Test]
creade
#1 Posted : Thursday, January 23, 2014 12:16:53 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 1/23/2014(UTC)
Posts: 2
Location: United States of America

Thanks: 1 times
Hey all,

We have a test file that looks something like the following:

Code:

using NUnit.Framework;

namespace Tests.Unit
{
[TestFixture]
public class Twice
{
    private static bool initialized;
    
    [TestFixtureSetUp]
    public void Setup()
    {
        if (initialized)
        {
            Assert.Fail("fixture setup called multiple times");
        }
    
         initialized = true;
    }
    
    [Test]
    [TestCase("thisWillFail")]
    public void ShouldTestSomethingOnce(string willThisFail)
    {
         Assert.That("thisWillFail", Is.EqualTo(willThisFail));
    }
    
    [Test]
    public void ShouldTestSomethingElse()
    {
        Assert.That(1 + 1, Is.EqualTo(2));
    }
    
    }
}

What appears to matter is the use of the [TestFixtureSetup] alongside a mix of tests with and without [TestCase]. This will cause the setup to be run twice. These tests will pass (i.e. the setup will only be run once) from the nunit or resharper runners but fail (run twice) in ncrunch. Any advice in getting this to pass in ncrunch?

Thanks!
Remco
#2 Posted : Thursday, January 23, 2014 3:34:53 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,986

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for posting!

The behaviour of NCrunch will be slightly different in running this test compared with other test runners - although it isn't different quite in the way you may think it is.

Because NCrunch executes tests in batches, it tends to make multiple calls into a test process to execute different tests. Sometimes it may call into the same process multiple times to execute the same test, if asked to do so.

This is quite different from other test runners, that will tend to run all tests from start to finish inside one process, then tear the process down at the end. It's done to properly enable the parallel execution and distributed processing features, as well as to improve performance.

In this particular case, the test's TestFixtureSetUp method may be called multiple times, but only in the context of multiple test runs. For example, NCrunch may only call the TestFixtureSetUp method once when it tries to execute both tests as part of the same test run, but it will call the method twice if the ShouldTestSomethingOnce and ShouldTestSomethingElse tests are each executed in different test runs to service different tasks/batches.

Another way to explain this is if, for example, you were to try running your test using the NUnit UI twice. If you did this, NUnit would execute the Fixture Setup, then the test method. The second time you ran the test, it would once again execute the Fixture Setup, then the test method. NCrunch does exactly this, only it doesn't tear down the test runner process between the execution runs.

There's a detailed explanation of this issue here. The solution is basically to allow your test and its setup methods to potentially be called multiple times within the same process.

I hope this makes sense.


Cheers,

Remco
1 user thanked Remco for this useful post.
creade on 1/23/2014(UTC)
creade
#3 Posted : Thursday, January 23, 2014 5:02:18 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 1/23/2014(UTC)
Posts: 2
Location: United States of America

Thanks: 1 times
Hey Remco!

Thanks for getting back. This makes perfect sense and I'm sorry I missed it being laid out so specifically in the docs!
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.033 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download