Hi, thanks for sharing this issue.

Everything you've described is exactly as designed, and at first look it seems crazy and clumsy, but there is a good reason for it.

NCrunch is a fully continuous runner - it uses a processing queue containing a list of tests to execute that is fully dynamic. As you make changes to your codebase while NCrunch runs the tests, the ideal execution order of the tests changes, and NCrunch continues to update the queue.

This runs counter to the design of test frameworks like NUnit, where the framework is designed for a single end-to-end run of all (or a select few) tests in a given codebase. The only way NCrunch can make NUnit work with a dynamic execution order is by making small calls into the test framework, running the tests in batches. Depending upon the execution time of your tests, you may have a large number of batches to execute your 400 tests (look for the number of test tasks in the Processing Queue Window).

Because of the way it's designed, NUnit will execute a SetUpFixture for every call into the test environment, with the corresponding tear down happening on exit from the call.

The mechanics of this can make a test run take far longer than it needs to if there are expensive operations in the SetUpFixture. The good news is that it's pretty easy to design your code so this won't be a problem for you.

Add a static field to your SetUpFixture that prevents it from being run more than once for the same process (i.e. private static bool hasBeenRun). See here for an example: http://www.ncrunch.net/documentation/considerations-and-constraints_test-atomicity.

The TearDown code is a bit trickier. The problem with placing cleanup code in a teardown is that you actually can't be 100% sure that this code is run. There are a number of things that can stop a teardown from running (i.e. sudden process termination, power outage, out of memory exception, stack overflow exception, etc). Even without NCrunch in the picture, this makes the teardown a bad place to remove your database. Under NCrunch, teardowns just won't work for this sort of cleanup because you'll have no way of knowing when a test run has actually completed - and NCrunch won't know this either, because it's true continuous testing.

The best option here is to instead concentrate on cleanup up stale databases in the system at the start of the next test run, instead of making each run clean up after itself. Instead of naming your databases using a GUID, try naming them using the windows ID of the currently executing process (Process.GetCurrentProcess().Id). When a test run starts, you can query the DB server for a list of databases, and compare this against the list of running processes on the computer. Any test database that exists on the DB server that doesn't have a matching process under execution can be safely removed. This also has a nice advantage in giving you a way to view the database left over from a test run after a failure, as they won't get cleaned up until the next test run starts.

I would say that if you haven't yet managed to get all your tests running continuous (and distributed), you're missing out!



The Process ID is very, very useful when working with NCrunch .. as it's one thing that is guaranteed to be unique across multiple processes executing in parallel.



If a new process has started, then we do definitely want the boolean to be false and the setup code to be run again. A new process can broadly be one of three things:
1. A fresh execution run of a whole new version of the codebase (you've made a change)
2. A new parallel session (i.e. you already have one session running, NCrunch is starting another to add capacity)
3. You've told NCrunch to spin up a fresh new process to run some tests

In each case, you want the setup code to run, and you want a new database. You don't want separate test batches sharing the same database if they're running in parallel, as you'll likely see database locking issues.


[/quote]

Unfortunately not. I think the only way we're likely to see this in the near future is if NCrunch has its own test framework, but I'd prefer to avoid this as it will be fully proprietary and wouldn't work with anything else.