Having a problem with NCrunch where the majority of my tests are failing due to this error:
Assembly initialize/cleanup failed: System.Diagnostics.Contracts.__ContractsRuntime+ContractException: Assertion failed: Dependency.instance == null Cannot set dependency resolver once it has been set.
The code is being called from within the [AssemblyInitialize] method of my test project to set up a StructureMap dependency resolver. We use CodeContracts to ensure the dependency resolver is only set once (it's a Singleton/static instance):
public static void SetResolver(IDependencyResolver resolver)
{
Contract.Assert(Dependency.instance == null, "Cannot set dependency resolver once it has been set");
instance = resolver;
}
I am guessing the failure is to do with some kind of parallelization but have tried disabling parallel test execution, setting NCrunch CPU cores to 1, max number of processing threads to 1 but the problem still happens. If I click on a failed test and run it individually, it works so I know NCrunch is capable of running the test successfully, just not when all tests are being run even if they're supposed to run sequentially!
Any advice? I really want to use NCrunch but this is a big solution I'm working on for a large company and I can't change the structuremap settings (besides which the tests run fine through MSTest so technically there's nothing wrong with the project config).
Edit: it may be worth mentioning that my solution has 17 projects, 4 of which are test projects and each one will have a similar [AssemblyInitialize] routine to set up the structuremap.
NCrunch has difficulty with these tests because it uses test processes in a different manner to VS's MSTest runner.
The standard MSTest runner will spawn a test process, then run all the tests one by one until completion, then tear the process down.
NCrunch will spawn a test process, run several tests in the process as part of a single run, then call back to the process later with more tests to be run. Effectively NCrunch runs tests in batch and a side effect of this is that the Assembly initialize/cleanup can be called multiple times within the same process - once for each test run.
I can't think of any way you can reliably force NCrunch to avoid re-using test processes through configuration in a way that won't completely annihilate your testing performance. The best place to solve this problem is in your code. Try placing a static flag checking condition at the top of your AssemblyInitialize method to ensure it will not execute its contents more than once.
Thanks for the quick reply. Okay I will try and add some code to the Initialize and try to make sure it only happens once which as you say should fix the problem.
Is this something that could be fixed in a future release, or is it just an unfortunate side-effect of the continuous test process that NCrunch uses?
Unfortunately it's quite a difficult thing to change, as it is very much tied into the architecture NCrunch uses in order to run tests. The overhead of launching test processes and sending profiled data back and forth is very high, so the batching makes a big difference in the overall performance of NCrunch. Generally it doesn't cause problems, but occasionally tests tend to surface that weren't really written with this architecture in mind.
That's ok, it makes sense I guess. For any others experiencing the same issue, wrapping the offending calls within the [AssemblyInitialize] method inside a conditional branch and setting a static flag the first time the code is run will successfully prevent the problem from occurring.