Hi, thanks for sharing this.
The more detailed explanation to this is rooted within the architecture of the .NET platform and certain design decisions that have been made within NCrunch. The name of the configuration setting involved ('Copy referenced assemblies to workspace') is actually not ideal, but we don't have an acceptable way to change the name of the setting without impacting everyone, and so it persists.
When Visual Studio (or MSBuild) builds a project, it constructs a dependency tree of the project and all other projects it depends on. It then builds the entire tree, starting from the bottom. This effectively makes it impossible for you to build only the project you want to build without all its dependencies being passed through the system. MSBuild does have some level of optimisation to avoid rebuilding projects that have just been built, but this is very limited and there is still a considerable performance cost. So if you were to make a surgical change to a project at the top of a large dependency tree, you can expect the build to take quite a while even if you only changed a single line of code. Naturally, this has serious implications for a performance critical test runner like NCrunch.
So to get around this, we pull projects out of the solution and rewire their dependencies so that they are built in isolation. This has major performance advantages and often makes the difference between being able to return test results in a few short seconds as opposed to one minute plus. Unfortunately, we break a number of abstractions when we do this, and it only works if we avoid copying the DLLs of referenced projects into the build output directory of the project.
Why can't we copy DLLs of referenced projects into the build output directory? This is due mostly to file locking issues. When we have open test environments using the temporary workspace, they hold locks on the DLLs in the process. This stops us from updating and re-using the workspace easily when other changes are made to the codebase. More locked DLLs in the workspace means less flexibility for the engine which means higher disk consumption, longer spinup times and worse performance.
So basically, we have a system in place where solutions with a simple interproject dependency structure can be handled very efficiently by NCrunch with minimal resource consumption and hopefully little configuration required. Unfortunately, this starts to break down when additional toolsets become involved that require reference DLLs to coexist in the build output directory with the primary output file. Basically, if you were to write any code in your solution that tries to manage the location of reference DLLs by itself and will only work if they are adjacent to one of your other assemblies, then NCrunch won't work with this code unless 'Copy referenced assemblies to workspace' is turned on. In years gone by we mostly hit this problem with IOC containers and the like, but now we also have Microsoft.AspNetCore.TestHost, which micro-manages dependencies obsessively and took a huge effort from us to even have it work at all.
At the time NCrunch was first introduced, the concept of a solution being a buildable unit wasn't really as solid as it is now. Back then, pulling projects out of the solution and building them in isolation didn't upset too many toolsets as the build system was generally simpler and less extensible. Now, someone can just grab a Nuget package that completely restructures the whole build system in a way that we just can't efficiently organise NCrunch around. When packages or toolsets that do this become really popular, we make NCrunch detect them and turn the setting on to save people the trouble and confusion .. but sadly the performance benefits of the optimised system move out of reach.
