Hi, thanks for posting!

When you change your code, NCrunch will analyse the nature of the change and will rebuild the project that you changed. If the change was to any of the public interfaces of the project, then NCrunch will also build any projects that depend upon it. An exception to this is if you have turned on the 'Copy referenced assemblies to workspace' turned on for any of the projects in your solution - in which case these projects will almost always need to be rebuilt.

The running of your tests will depend upon the 'Engine mode' you've selected NCrunch to run in. There's more information about this in the wiki, but when set to run tests automatically, NCrunch will run ALL tests that make use of a changed project regardless of whether the impact detection picked them up or not. The impact detection is used more as a way to prioritise the sequence in which the tests are run as opposed to selecting them for execution.

The reasoning behind this is quite simple - no impact detection is perfect. The tests held within your solution could have external dependencies well outside the knowledge of NCrunch and as such you can't be sure whether all your tests are truely passing until every one of them is executed.

Of course there is another way to approach this. If you were prepared to accept that there was still a slight chance that a test could fail without the continuous test runner identifying the need for it to be run, then it is possible to run only the impacted tests and reduce the amount of work that needs to be done. NCrunch doesn't do this yet, but it is a high priority feature that will be added very soon.


Cheers,

Remco

Absolutely. NCrunch stores 5 different types of files on your disk:

globalconfig.crunch.xml - Stored in your AppData\Roaming\NCrunch directory. This contains user-specific global settings for NCrunch that are applied across all solutions (such as performance related config etc).
[solution name].ncrunchsolution.cache - Stored next to your .sln file. This contains cached data from your executed tests, including persistent code coverage information.
[solution name].ncrunchsolution.user - Stored next to your .sln file. This contains user-specific settings, such as the layout of the Tests Window, currently selected engine mode, etc.
[solution name].ncrunchsolution - Stored next to your .sln file. This contains solution-specific settings, such as test framework settings and the parallel execution toggle.
[project name].ncrunchproject - Stored next to each of your project files. This contains project-specific settings, such as build and test related options, additional files to include, etc.

I generally recommend to place the last two files under source control, and leave the rest as local files on each developer's machine. In this way, it's possible for all of your critical configuration to be shared and versioned alongside the codebase, while anything that would be computer or user specific can be locally customised.

Determining whether or not a change made to a file could impact the results of tests is actually (almost?) impossible to do reliably. It's for this reason that NCrunch works as more of a prioritisation engine than a test-excluding one. Although to us developers it may seem very obvious (hell, it's an XML comment!), the implications really are not that simple at all.

For example, you may have a build step in your project that reads the XML documentation in order to generate a resource file that is fed into one of your automated tests. By changing the XML documentation, you change the contents of the resource file and thus the test would need to run. I've never seen anyone do this before, but it's theoretically possible and NCrunch needs to consider it.

If you look under your NCrunch menu, you'll see an option that allows you to 'Set engine mode'. By default, NCrunch has a pre-built engine mode that will run impacted tests only. If you're operating in a codebase where you want to be exclusive with which tests are run, I suggest giving it a try. NCrunch's impact detection is far from perfect so you'll need to be careful about still performing a full test run before checking in your code .. although from your description it seems as though this may be what you are looking for.


Cheers,

Remco