I'm using NCrunch 3.2.0.3 and VS2015.


I need some way to exclude integration tests from running automatically because they take a lot of resources.
All my integration tests have the category "Integration" applied.
The only settings window that seemed suitable for this is the Customize Engine Modes window.

For some reason, there are eight configurations. Four say (This Computer) and four say (This Solution - Shared).
I want this to be a shared configuration, but it seems I have to choose between setting the engine mode to "Run impacted ... (This Computer)" or setting it to "Run impacted ... (This Solution - Shared)".
It would make more sense to me if I was only able to set it to "Run impacted ..." and NCrunch would merge the settings between "This Computer" and "This Solution - Shared."
So I'm really confused about that. Anyhow, I have it set to "Run impacted ... (This Computer)" right now. I'm also not sure if the Move Up/Move Down position in the list affects whether they combine?

Anyhow- I expanded Filters and changed the "Tests to execute automatically" filter from "When IsImpacted" to "When (IsImpacted AND DoesNotHaveCategory 'Integration')".
I also changed "Show coverage for tests" from empty to "When DoesNotHaveCategory 'Integration'" because I typically run the integration tests via Cake or ReSharper.

For whatever reason, this setting is simply not working. Integration tests still show coverage and still run automatically when impacted.



P.S. unrelated UI bug report: When you first open the Customize Engine Modes window and you click on any engine mode, the settings grid does not show any settings in bold. If you select another mode and then select the original mode, it shows correctly. It also begins showing correctly if you expand any line.

I've deleted the My Computer modes, so now I have the four shared modes left.

> To fully exclude all integration tests from automatic execution, you need to adjust the 'Tests to execute automatically' setting for each of the engine modes.

I should have mentioned that I had done this for all engine modes in the list.

> If the engine still seems to queue integration tests automatically, it's worth adding the 'Category' column to your Tests Window to ensure that the category for these tests has been detected correctly.

Yes, I should have mentioned that I did that! "Integration" is clearly showing up in the column.
Is there some type of dump I should take?

> If you only run integration tests via a different tool, the NCrunch 'ignored tests' feature might be a more appropriate feature for you.

Well, yes, and I was getting along this way for a while, but... I forget to ignore new integration tests.
Other times I do love to use NCrunch to run or debug a certain integration test when I'm in the IDE actually developing a test or feature. The code coverage indicators in non-test code are so handy for dropping to run or debug.

In the process of building a repro, I discovered that I was mistaking the cached failure state appearing in the Tests window for NCrunch continuing to run the test. For some reason, I was expecting the test to disappear from the Tests window.

I rely on the errored states in the NCrunch window all the time to track my progress. My integration tests tend to end up being remembered by NCrunch as failing. It's been so much noise that I've wanted to exclude them altogether (as in Ignore) except that running them from the code window is handy. Plus, you can't ignore based on category. So I'll change my approach.
I'll show code coverage for all tests, and I'll run the ones that will pass. (One integration test run by NCrunch will always fail because it has a timeout and NCrunch always takes about 9 seconds to do what takes 2 seconds in every other runner. I do not want to see that test in the NCrunch test window, ever. Ignoring is useful in this case. Perhaps I should open a separate issue on that.)