Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Autofixture InlineAutoData attribute always causes NCrunch to mark existing tests as new
royboy23
#1 Posted : Friday, July 22, 2016 5:18:18 PM(UTC)
Rank: Member

Groups: Registered
Joined: 3/3/2014(UTC)
Posts: 21
Location: Germany

Thanks: 3 times
Hi,

I'm using NUnit 3.4.1 and Autofixture 3.49.0 to write parameterized tests such as this one:

[Theory]
[InlineAutoData(1)]
[InlineAutoData(2)]
public void Test(int someStableParameter,int someRandomParameter)
{
Assert.AreNotEqual(0,someParameter);
}

The problem is that NCrunch (2.23.0.2) always marks tests like these as "new". This causes existing tests to be marked as dirty all time, and it also makes the feature to pin new tests automatically useless since parameterized tests are always pinned.

Apparently the issue is that NCrunch assigns the test names such as Test(1,<randomValue>) and uses the same name determine if the test is new or not.

Is this a bug, or am I doing something wrong?

If it is a bug: Is there any workaround I could use?

Thanks,

Adrian
Remco
#2 Posted : Friday, July 22, 2016 11:33:50 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi Adrian,

This problem is caused by a technical limitation, so I guess you could say it's by design. This isn't to say that the design is a desirable outcome .. rather that there is nothing that can be done to 'fix' it.

For NCrunch to correlate data between tests, it needs to be able to uniquely identify them. For NUnit tests, this can only be done by the name of the test. This means that each test must have a unique and consistent name.

When using AutoFixture to generate random parameters for your test, what is technically happening under the hood is that an entirely new test is being generated every time the test discovery step is run. The test is new because its parameters are different, and therefore its name is also different. NCrunch has no way of knowing that the data related to the previously generated test is related to the new test, so it just discards the old data and creates a new test.

This is less of a problem for end-to-end test runners that don't store state between test runs.

The only way to solve this problem is to redesign the test so it does not use random data in its parameters.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.026 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download