Build/Test Issues

NCrunch and Autofixture NUnit AutoMoqData with parameters problem

Started by sebastijanp on 5,205 views

Hi,

When I write my test in the following form:

[Test]
[AutoMoqData]
public void When_GetAllResults_Then_ItShouldGetOnlyResults2(Fixture fixture, [Frozen] Mock<ICompetitionsService> competitionsServiceAsMock,
CompetitionsProvider sut)
{
IEnumerable<MatchModel> competitions = fixture.CreateMany<MatchModel>(4);
competitionsServiceAsMock
.Setup(x => x.GetResults(It.IsAny<string>(), It.IsAny<string>()))
.Returns(competitions.AsQueryable());

...

var result = sut.GetAll<MatchModel>(dataRequest);

int expected = 4;
result.Items.Count().ShouldBe(expected);
}


The test is not beeing able to run and I get the following message:

NCrunch was unable to locate this test during NUnit's execution run. This problem can be caused by tests with names that contain random elements, such as through the use of NUnit's RandomAttribute. Please ensure your test is named consistently or consider changing your 'Framework utilisation type for NUnit' solution-level configuration setting to 'StaticAnalysis'.


In Resharper this test runs normally and passes. Note that test written in other way, also passes in NCrunch, but of course, I want to use the above way, since it dramatically reduces the amount of code.

[Test]
public void When_GetAllResults_Then_ItShouldGetOnlyResults1()
{
var fixture = new Fixture();
fixture.Customize(new AutoMoqCustomization());
var competitionsServiceAsMock = fixture.Freeze<Mock<ICompetitionsService>>();
IEnumerable<MatchModel> competitions = fixture.CreateMany<MatchModel>(4);

competitionsServiceAsMock
.Setup(x => x.GetResults(It.IsAny<string>(), It.IsAny<string>()))
.Returns(competitions.AsQueryable());

...

var result = sut.GetAll<MatchModel>(dataRequest);

result.Items.Count().ShouldBe(4);
}


Will this be fixed (or is it already fixed) or is an advice that I have noticed here, not to use AutoMoqData, and have 10.000 lines of additional code? I was planning to buy it, but in this case of course, I have no use of it, since I am writing all of my test in this way.

Kind regards,
Sebastijan
Hi Sebastijan,

Thanks for sharing this issue.

This problem is a symptom of a structural issue that exists between NCrunch, NUnit and AutoFixture.

AutoFixture works by randomising the parameterised data being passed into tests.

NUnit takes this random data, and uses it as components of the physical test name.

NCrunch uses the name of the test to uniquely identify it during a run. Because NCrunch discovers tests in a separate step to executing them, the randomisation from AutoFixture means the test name is not consistent between steps. This makes it impossible to uniquely identify the test.

I'm afraid that no direct technical solution exists that can solve this problem. NUnit itself has an internal identifier than can uniquely identify a test between runs (even with the randomisation), but as this identifier is dependent on test discovery sequence and tends to change as new tests are added to the suite, NCrunch cannot make use of it.

From the side of NCrunch, the only suggestion I can provide is to try changing your 'Framework utilisation type for NUnit' (solution-level NCrunch configuration setting) to 'StaticAnalysis'. This will change the behaviour of NCrunch so that it will use static analysis to discover the test, removing the randomisation from the test name. I'm afraid I cannot guarantee that this will work as I'm not sure as to the internal mechanics of AutoFixture when this feature is enabled. You may lose some discovery metadata.
Hi,

I tried the same somehow in xunit and it works. So the problem is only permutation autofixture-nunit-ncrunch. Just wanted to let you know. Maybe packages.config of my libraries installed might help in resolving an issue in case there is an interest of a new customer:)

<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="AutoFixture" version="3.30.8" targetFramework="net45" />
<package id="AutoFixture.AutoMoq" version="3.30.8" targetFramework="net45" />
<package id="AutoFixture.AutoMoq.AutoSetup" version="1.3.0.0" targetFramework="net45" />
<package id="AutoFixture.NUnit2" version="3.30.8" targetFramework="net45" />
<package id="Moq" version="4.2.1506.2515" targetFramework="net45" />
<package id="NUnit" version="2.6.4" targetFramework="net45" />
<package id="NUnitTestAdapter" version="2.0.0" targetFramework="net45" />
<package id="Shouldly" version="2.5.0" targetFramework="net45" />
</packages>


Kind regards,
Sebastijan
Thanks. I wonder if this may be due to the way the test are being named between NUnit and xUnit. If the physical name of the test includes any random data (as might be provided by autofixture), then there will be problems. I would not be surprised if the autofixture implementation between xunit and nunit is quite different.

Post a reply

Log in to reply.