Hi there

We have been experimenting with amalgamating 20+ smaller solutions into one big solution with c. 750 projects given the advent of VS 2022.

It has worked fine so far but we are having trouble getting Ncrunch to execute tests. The tests executed fine in the previous individual, smaller solution files.

Ncrunch sucessfully builds everything, and executes some of the tests (about 3k out of 16k) - but then stops. The tests remain as pending tasks in the processing queue, and there are no active tasks.

I can remove all (13k+) pending tests and try to queue just a handful of them manually, but the situation is the same: the tasks are pending but never execute.

I can successfully re-queue and re-run the tests that previously executed ok.

How I debug this and find out why the tests are not running...?

Thanks

Thank you for the reply.

None of the build tasks have failed, but we do have two .netstandard 'helper' projects which reference NUnit for assertions but contain no tests (i.e. they are not actually test projects) that come up as an Analysis failure. But I think this should be ok (?) Or will NCrunch refuse to run tests that rely on these libraries?

The 'Required Capabilities' in the pending task list is all blank.

Anything else to check?

Nice work, that will fix it :)

The problem here isn't that these are test utility projects. It's perfectly fine to have projects that reference test frameworks but don't declare any tests. The problem here is that these projects are targeting a platform that cannot be directly instantiated at runtime (netstandard). We have no way of knowing whether a project contains tests without first loading it into a runtime environment, which we can't do for netstandard projects ... hence the failure. The engine should have given you a meaningful error about this, so I think you encountered behaviour as designed here. Great to hear you are up and running now.

This happens due to the structural behaviour of the engine. The build and analysis steps for a project are considered to be a dependency of all other projects that depend on it. Technically, the analysis step does not need to be a dependency (as it isn't required for building a test environment for referencing projects). However, in terms of sensible operation, we do still need to perform all analysis steps successfully before we can properly prioritise tests in the queue. Sensible prioritisation is VERY important, especially when big tests are involved. We don't want the engine kicking off big test runs when the overall list of tests is still a big unknown. For this reason, a failure encountered during analysis is given the same level of status as a build error.

It's possible to improve the above. We could implement a range of hacks and implicit logic to try and allow the engine to still process the rest of the work around analysis failures, but it really isn't worth it. There is a serious error being reported that requires attention. To properly use the product without alarm bells going off, you need to fix the error. Allowing the product to work around the error gives very little value when a decision about the netstandard project needs to be made anyway.