Upgraded test projects to 4.7.2 - 0 tests discovered

Hi - I have a bit of a nightmare Solution (sorry!) which has a .NET Framework 4.7.2 test project, referencing some .NET Standard libraries and packages. It was 4.6.1, working fine locally - then trying to get Grid processing set up I had some problems with reference assemblies, so tried 4.7.1 and finally 4.7.2. However it is indeed not discovering any tests.

How can I diagnose this please? (I don't suppose there's an option for a screen-share session? :))

Thanks
Kieren

Hi Kieren,

Thanks for posting!

Is there any chance you could submit a bug report after you've had the discovery run fail? Perhaps the log file may contain some useful clues.

Thanks, I've submitted a report with this thread URL as the name. :)

Thanks for sending through the report. Looking through the logs, I wasn't able to find any significant issues reported by the engine, so we may need to dig a little deeper here.

If you look in the processing queue window, do you see any tasks performed for 'Analyse Assembly'? Can you confirm which test framework you are using?

Yes, the assembly is analysed (by all 3 machines in the grid: local, plus the other two). It's NUnit 3.7.1.

Are you able to try building a small NUnit 3.7.1 test project/solution on your machine? We need to see if this is specific to your environment or if its something in the solution. Assuming the test project works, we then need to figure out what the point of difference is between this project and your main solution. From there, it should hopefully be easy to identify a solution.

I've created a fresh .NET Framework 4.7.2 unit test project, referenced NUnit 3.7.1, and the test is discovered fine (both locally and on the grid).

The real project references around an additional 200 NuGet packages (lots of .NET Core stuff), as well as 16 other projects in the solution.

What would be the best next steps to try?

From here, the best approach is to transfer things over to your sample project, one step at a time.

For example, start by just copy/pasting your entire list of Nuget package references into the test project, then see if it still works. If not, we know it's something in the Nuget packages .. if it does work, it must be something else. So from there, you start transferring project references or source files.

It can sometimes be a time consuming approach, but the good thing about it is that you'll always identify the point of difference and therefore the reason for the failure. We use this approach very often in developing the product as the platform itself isn't always very good in its error reporting in the areas we've integrated with it.

Ok, that sounds good, thanks. A nice systematic approach.. I'll commit each step as I go into a repro repo.

I started doing this, but then had a hunch. I removed all 111 binding redirects from the test project, and tests were then all discovered!

I mean, it works, but I feel surprised that those redirects were doing nothing. The tests run very, very slowly (hence trying to set up a fleet of old machines to run on Distributed Processing).. but I'll have them all run though and see if any errors pop up, other than normal test failures.

In the meantime, is there anything you'd like me to do?

Nice work!

The more recent versions of VS tend to go a bit crazy with binding redirections. I've seen them appear in project templates and as a result of tooling changes. They are not always required. My best guess here is that you may have had a redirection that was messing with the test framework's ability to resolve its dependencies. Troubleshooting downstream logic caused by binding redirections is very time consuming and difficult, because the problems usually manifest themselves silently in the platform or in the test frameworks themselves. Thus I would be happy not to investigate this problem if you're happy with your solution to it. If you have any record of how these binding redirections were introduced to your test project, I'd be really interested to know this.