Hi Dave,

This approach cannot be made to work with NCrunch, because it relies on implicit behaviour describing the references between projects.

NCrunch works by shadowing projects individually and separating them from their parent solution. In the workspaces that NCrunch constructs, the project references are manipulated in order to build a test environment in the fastest way possible without needing to perform additional build steps. For this to work, NCrunch needs to be able to recognise references between the projects as making use of the ProjectReference MSBuild declaration. It cannot infer project references using the Reference declaration. Because of this, you'll find that NCrunch is referencing the MyLibrary.dll file that is sitting in your foreground solution, which cannot be correlated with its source project and therefore no code coverage exists.

Sorry, but you'll need to use a more conventional approach that does not involve referencing binary outputs from dependent projects. There's no way I can make this work for you in its current form.


Cheers,

Remco

NCrunch struggles with absolute file references, because they usually cause code to break the isolation of the sandbox (workspaces) NCrunch constructs to build your code and execute your tests.

In this particular case, the usage of an absolute path in an environment variable is causing NCrunch to use the same build output path as that used for your foreground solution. This creates two problems:

- Because the assemblies are being built to the same place as your foreground code, you'll get file locking issues. NCrunch uses the assemblies built from your projects in order to execute code. This requires launching and executing task runner EXEs which will hold file locks on any code files they make use of (as any CLR application does). When you try to run a build in Visual Studio, VS attempts to place the newly built assemblies over the top of the assemblies that NCrunch has locked.

- Because the assemblies from multiple projects are all being put in the same place, NCrunch will be unable to exert reliable control over which assemblies are loaded into a test environment. The CLR uses heavily implicit logic for loading assembly references. By forcing all references into the same place, NCrunch cannot intercept the assembly resolution process and you'll get weird issues around the versions of code inside assemblies used by the test process. You'll also get multiple warnings from the NCrunch engine about unexpected assembly resolution. This specific problem can be resolved by turning on the Preload Assembly References project-level configuration setting.

The solution to this issue is to avoid using absolute file path references. If this isn't compatible with your normal foreground build process, you can introduce an override into the project file to provide NCrunch with a relative OutputPath, while non-NCrunch builds still use the environment variable. For example:

<OutputPath Condition="'$(NCrunch)' == '1'">bin\</OutputPath>
<OutputPath Condition="'$(NCrunch)' != '1'">$(OUTDIRECTORY)</OutputPath>

You are correct in your understanding that it is very common for people to want to place all assemblies in the same place when a solution is built. In my experience, this is usually done using paths relative to the solution, for example:

<OutputPath>$(SolutionDir)bin\</OutputPath>

NCrunch can automatically handle file paths relative to the solution, as it accounts for the solution directory when padding out the workspace for each project.

Hi Dave,

Great to hear that did the trick. I'm sorry if I seemed a seemed a little purist earlier while trying to describe NCrunch's limits when handling a situation like this. Although I haven't seen a solution arrangement quite like what you've described, it it an interesting approach and it is not without merit.

I think the best way to look at the structural issue you are trying to solve is to split it into two problems.

The first problem is how to work with team members that store their code in different locations. I have seen many situations where people will simply put down a rule (as you've described) where everyone must have their code in the same place on disk. I agree that this is probably unnecessary and it removes flexibility from the development environment. It also creates an implicit expectation that is not obvious to someone simply pulling down the code to work on it. Generally I try to solve problems such as this one using the source control system. Dependencies between projects and solutions should usually be relative, as this is the best way to decouple a solution from its environment. Most source control systems contain features allowing you to 'check out' dependencies in relative locations to the project being worked on (i.e. SVN externals). Where this is impossible, you might find a structure similar to the one below can often work too:

All Solutions\
All Solutions\Boss\MainApp
All Solutions\IBMProjects\MainApp
All Solutions\Contractor\CommonCode

.. Assuming everyone checks code out of the repository at 'All Solutions', they can easily be kept in sync with the other projects/solutions with minimal effort. Source control updates can be performed against the root of the source tree, so everything stays up to date.

The second problem is dealing with having multiple solutions referencing a selection of projects. I.e. everyone on the team has their own solution, which contains only the projects they work on. Sometimes solutions may contain the same project, and these projects are all interdependent. Usually this means that you end up with a solution containing projects with <ProjectReference> declarations pointing at other projects outside the solution. This is actually a very complex situation that is difficult to find an ideal solution for. Most toolsets don't really handle this very well. NCrunch handles it by automatically resolving referenced projects outside the solution and including them with the engine.

Unfortunately, in order to make the layout work with other tools, people very often strip out the <ProjectReference> tags in the .proj files and replace them with fixed <Reference> tags pointing at the output DLLs of other projects - I guess the same way you have done for at least one of your projects. NCrunch then can't handle this as it doesn't realise you're referencing a project rather than some DLL, so the build doesn't work properly and code coverage is missing. Other (more serious) problems also go in hand with this solution, as MSBuild doesn't recognise the dependency and it may build the projects out of sequence. There's also a much greater risk of the DLL being referenced having been built using different configuration to the depending project (i.e. perhaps it is a DEBUG binary being included in a RELEASE build). Finally, it creates a further dependency on all the code being properly built before being checked into source control, as the DLL version may not be consistent with the source code.

Generally I recommend avoiding using multiple solutions where possible. If your team is working very closely together on different aspects of the same overall problem, they should probably be working in the same solution. If need be, you can set up Solution Folders to split the various projects out, making them easier to manage. Where codebases become unreasonably large to the point where loading all projects slows the IDE to a crawl, often I've found this to be symptomatic of a larger design issue where the solution itself can be broken up in to areas that are worked on less frequently, and can be split up into libraries that don't change often enough to be included with all their source code. Organisational limitations often tend to solve this problem, as large code bases are often split between multiple teams who take responsibility for different parts of the project (and can therefore have their own codebases separate from each other using formal integration points).

There are long term plans to introduce functionality to NCrunch that will solve the composite solution problem entirely. I can't share any details on this right now as it's all still in the planning stage, but it will be exciting when it happens :)

I hope this helps!


Cheers,

Remco