Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Nunit theory and autofixture problems
floekke
#1 Posted : Saturday, April 20, 2013 9:55:32 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/19/2012(UTC)
Posts: 8

Hi

I'm using NUnit theories together with Autofixture for creation of test data. My tests are working just fine in the standard nunit test runner (nunit-gui) but ncrunch is acting strange. Often reporting "This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures."

Any suggestions?

Thanks.

Remco
#2 Posted : Sunday, April 21, 2013 12:36:53 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for posting!

Do you have a small code sample you can share that describes the structure of this test?

Also, does changing the framework utilisation type for NUnit make any difference in the behaviour you're seeing?


Cheers,

Remco
floekke
#3 Posted : Sunday, April 21, 2013 7:57:31 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/19/2012(UTC)
Posts: 8

I have created a small example that demonstrates the problem:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using NUnit.Framework;
using Ploeh.AutoFixture;

public class StackTheories
{
Stack stack;

[SetUp]
public void SetUp()
{
stack = new Stack();
}

[Datapoints]
public IEnumerable<int> Data
{
get
{
return new Fixture().CreateMany<int>();
}
}

[Theory]
public void PushPopIdentity(int a)
{
stack.Push(a);
var b = stack.Pop();

Assert.AreEqual(a, b);
}

}

public class Stack
{
readonly List<int> store = new List<int>();

public void Push(int a)
{
store.Add(a);
}

public int Pop()
{
var top = store[Top()];
store.RemoveAt(Top());
return top;
}

int Top()
{
return store.Count - 1;
}

}

Also, changing the framework utilization type to static, makes the test pass.
Remco
#4 Posted : Sunday, April 21, 2013 9:57:42 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Thanks for the reproduction - this made it much easier to identify the problem.

Basically, this code is relying on dynamic test cases that make use of parameters of a random nature. NCrunch discovers and executes tests in separate physical steps, which means that the test names are not the same between the discovery step and the execution step. Because the names aren't consistent, NUnit/NCrunch fail to match up the tests and they don't get executed.

Unfortunately I don't see a way to solve this problem with a framework utilisation type of Dynamic Analysis, as there is no way to reconcile the random test name with the name that should be executed. I suggest avoiding the use of random values for theory datapoints. Where this is impossible, setting the framework utilisation type to static analysis (as you have done so) will allow you to work around the problem, although this will also have the effect of collapsing all the individual theory tests under one test name.

Cheers,

Remco
floekke
#5 Posted : Sunday, April 21, 2013 11:53:39 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/19/2012(UTC)
Posts: 8

Thank you for clarifying. It makes sense.

Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.036 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download