Global use of isolated attribute

I head up a Scrum team which includes (- BROKEN LINK -)'The Art of Unit Testing' guidelines[/url] as part of our definition of done.

Part of this defines that a unit test should be isolated, it should be able to run by itself and not be affected by other unit tests.

We rely on nCrunch heavily to ensure our tests are passing and would also like to use it to ensure all our unit tests are isolated.

There is an isolated attribute that exists within nCrunch, however this would require us to add it to each test in our solution.

Is there a way to can force all our unit tests to run in isolation, at the solution/project/nCrunch level rather that needing to add this attribute to all our tests?

Thanks
Peter

Hi Peter,

It's possible to apply the IsolatedAttribute at assembly-level (i.e. in the AssemblyInfo.cs file), in which case it will be applied to all fixtures in the assembly. This isn't exactly what you're asking for, as the isolation will only be applied to fixture (not test), but it's a step closer and won't require the attribute to be declared everywhere.

However, I would recommend against doing this unless you have a very small test suite. For NCrunch to run a test in isolation, it needs to create a whole new process to execute the test. This massively increases the overhead of each test run. You may find that such an approach slows the engine down so much that it becomes unproductive.


Cheers,

Remco

Hello Remco

Thanks for your feedback.

I saw the performance note on the IsolatedAttribute documentation page, but I didn't realise the impact would be so severe.

Have you any recommendations how nCrunch can be used to enforce test isolation, without going this route?

Thanks
Peter

Hi Peter,

Do correct me if my understanding is incorrect, but I believe you're trying to come up with a mechanical solution that will ensure each test is regularly run in an isolated process away from other tests that might influence its execution via state within the application domain?

Unfortunately there's not much I can recommend in this situation, besides letting the test runner go through its normal operation. The key problem is that in theory, any test could make a change that impacts static members or process-wide state and therefore could influence the execution of other tests later run within the domain. The only way to mechanically prevent this would be to tear down the whole domain and reconstruct it for every test run (which is exactly what the Isolated attribute does).

If you have tests within your suite that are sequence or state dependent, you may find that NCrunch will very often report them as intermittent failures. This is because NCrunch can vary the sequence in which tests are run. In your case, you may find this very helpful as the tests that are sequence dependent will eventually fail while you work, and you can then try and figure out what may have happened.