Reading through the docs again, could this work? For a test where the flag is on, use InclusivelyUses(feature1_off) to make it mutually exclusive to those tests where the flag is off by using.

Hi, thanks for posting.

When you specify InclusivelyUses, a test will be allowed to run only if no currently running test has declared ExclusivelyUses for the same resource.

So you could have a dozen tests making inclusive use of the same resource, and they could all run at the same time without constraints. However. while any of these tests are running, no test making exclusive use of the resource will be allowed to run. Likewise, if a test is making exclusive use of a resource, no tests making inclusive use of the resource can run. Thus InclusivelyUses is only useful in situations where ExclusivelyUses is also in use.

I'm not sure if I understand entirely what you are trying to achieve. You mention that you want certain sets of tests to be run at the same time. Why is this?

I can't think of a way to achieve such a setup with NCrunch right now. Can you tell me a bit more about why you need to have such a structure? Perhaps I can suggest an alternative approach.

This seems like quite a bit of micromanagement of concurrent execution. The intention of NCrunch is to try and to away with much of the micromanagement. Is there a reason tests with clashing flags can't be run together? Do they make use of an external dependency of some kind? (i.e. network socket, database, maybe something on the file system).

NCrunch never runs tests concurrently in the same process, so as long as the various features don't try to access the same external resources at the same time, they should be able to run concurrently without issues. The idea behind 'ExclusivelyUses' is that you can then label tests that do have a dependency on a specific external resource, so that the engine won't run them at the same time.

For example, you may have a feature that manipulates a shared database in a way that could interfere with other tests that are running. For every test with the flag for this feature enabled, you would add something like 'ExclusivelyUses("Database")'. In this way, you move the concurrency constraint away from the feature flags and over to the physical constraints underlying those features.

I accept this is somewhat of a paradigm shift away from how people are used to thinking of parallel execution (at least, in the way the frameworks themselves do it). The parallel features of test frameworks often involve launching several threads inside the same process, with each running a different test. We never do this. If you have static state in your codebase, this won't be a concurrency constraint for NCrunch.