We now see the above warning in our code base, alarming many. I'm not sure if the warning is entirely correct as the core library of AutoFixture cannot modify test names. This feature is delegated to other packages, e.g. AutoFixture.Xunit2 or AutoFixture.NUnit3.

See the repo: https://github.com/AutoF...aAttribute.cs&type=

Maybe consider changing the warning to only show when one of the supporting packages is installed. My team uses Autofixture, but only within tests, never within theories (hence we don't have AutoFixture.Xunit2 installed).

Yes, it's true that AutoFixture won't cause problems if it isn't used for providing parameters to test cases or theories. I have a somewhat bias here in that many of the problems I chase are related to inconsistent test case parameters, so I assumed that this was the primary purpose of the library.

I've made a note to revise the warning to avoid alarming people that aren't using the library for parameter substitution. There isn't any feasible way to detect a difference in how the library is used, but I can at least change the wording so that the warning is more specific. If you're not experiencing any of the problems described in the warning link, you can safely ignore the warning.

Looking at the img you've linked, it seems that NCrunch is not discovering your tests. This is in no way related to the AutoFixture issues and has nothing to do with the warning being shown.

Something else is going wrong here. Do you have any other warnings being shown? Which version of Xunit are you running? Do your tests work with other runners? (i.e. VSTest, Resharper, etc)

Unfortunately there isn't a reliable way to do this, because when user code is used to create tests, anything can happen. For example, tests may be generated from an external resource (like a resource file or database), so it's possible for the list of tests to be different without any IL changes detected.

This is, also, completely under the power of the test framework itself. Proper integration with a test framework involves treating it as a service. The framework is interrogated for the list of tests to execute after an assembly is compiled. To try and assume which tests are in an assembly without asking the framework would be to step around this entirely, effectively re-implementing the framework and removing all forwards compatibility with future versions of it. It's a massive problem when you ask a framework for a list of tests that is different every time you ask for them even when no changes are made. If you have tests that change their shape every time they're discovered because of user code, there is nothing in NCrunch or the discovery system that can give you a consistent result.

If you look in the Processing Queue Window, do you see any 'Analyse Assembly' tasks in the list?

Also, would you be able to submit an NCrunch bug report after NCrunch has completed building your solution?

Thanks for sending this through. As you've discovered, the key issue here is an exception being thrown when NCrunch tries to invoke Xunit to discover tests:

System.InvalidCastException: Unable to cast object of type 'Xunit.Sdk.ReflectionAssemblyInfo' to type 'Xunit.Abstractions.IAssemblyInfo'.

These types are convertable under Xunit 2.2. The only reason I can think of why they would fail to convert is if you were using different and incompatible versions of Xunit libraries.

Can you try removing all Nuget and assembly references to Xunit from your test projects, ensure that any Xunit packages under your solution directory are removed, then try reinstalling Xunit via Nuget?

It's possible that the version of SpecFlow you're using isn't compatible with Xunit v2.2. This is likely to give you headaches in future, as it means that the behaviour of your test environment will be determined by the sequence in which assemblies are loaded by the JIT. v3.8 may work for you here because it uses a binding redirection to force the test environment to use v2.1 of xunit.abstractions, but this may yield other downstream issues for you. As frustrating as this problem may be, I do recommend resolving it to prevent wasted time in future. Issues like this can result in some truly bizarre errors in tooling.

Just so you are aware of what I found in my investigation:

I loaded an issue with Autofixture. https://github.com/AutoF.../AutoFixture/issues/786

This is the probable cause to my issue.

After performing numerous install and reinstall, I realized yesterday that I was on v 3.8.0.3 when I thought I was on 3.10.0.20. So I went through the same process I did last time.

But when I thought I did the same process, ncrunch would not discover the tests.

So this caused me to try changing settings on my work machine since I knew ncrunch discovered tests on my home machine.

So the first setting I changed was Pre-load all assembly into test environment from true to false.

This detected the tests as expected.

So I have to ask this previously worked to detect tests with this setting being true on v3.8.0.3.

I came here to post about this self same thing and found this thread. My current team doesn't use xunit autodata theories either, so the warnings are essentially false for us.

Good to know we can at least ignore them until the warning in NCrunch is refined.