Nunit theory and autofixture problems

Hi

I'm using NUnit theories together with Autofixture for creation of test data. My tests are working just fine in the standard nunit test runner (nunit-gui) but ncrunch is acting strange. Often reporting "This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures."

Any suggestions?

Thanks.

Hi, thanks for posting!

Do you have a small code sample you can share that describes the structure of this test?

Also, does changing the framework utilisation type for NUnit make any difference in the behaviour you're seeing?


Cheers,

Remco

I have created a small example that demonstrates the problem:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using NUnit.Framework;
using Ploeh.AutoFixture;

public class StackTheories
{
Stack stack;

[SetUp]
public void SetUp()
{
stack = new Stack();
}

[Datapoints]
public IEnumerable<int> Data
{
get
{
return new Fixture().CreateMany<int>();
}
}

[Theory]
public void PushPopIdentity(int a)
{
stack.Push(a);
var b = stack.Pop();

Assert.AreEqual(a, b);
}

}

public class Stack
{
readonly List<int> store = new List<int>();

public void Push(int a)
{
store.Add(a);
}

public int Pop()
{
var top = store[Top()];
store.RemoveAt(Top());
return top;
}

int Top()
{
return store.Count - 1;
}

}

Also, changing the framework utilization type to static, makes the test pass.

Thanks for the reproduction - this made it much easier to identify the problem.

Basically, this code is relying on dynamic test cases that make use of parameters of a random nature. NCrunch discovers and executes tests in separate physical steps, which means that the test names are not the same between the discovery step and the execution step. Because the names aren't consistent, NUnit/NCrunch fail to match up the tests and they don't get executed.

Unfortunately I don't see a way to solve this problem with a framework utilisation type of Dynamic Analysis, as there is no way to reconcile the random test name with the name that should be executed. I suggest avoiding the use of random values for theory datapoints. Where this is impossible, setting the framework utilisation type to static analysis (as you have done so) will allow you to work around the problem, although this will also have the effect of collapsing all the individual theory tests under one test name.

Cheers,

Remco

Thank you for clarifying. It makes sense.