I have several integration test projects. All of their namespaces/project follow the exact same pattern:
Abc.Xyz.IntegrationTesting.Zyx.Cba
Where the name always includes ".IntegrationTesting."
I've created a custom engine mode as follows
project name does not match regex: \.IntegrationTesting\.
and
is impacted
And have also tried w/ a 3rd condition so it reads same as above, but re-adds the first filter:
project name does not match regex: \.IntegrationTesting\.
and
is impacted
and
project name does not match regex: \.IntegrationTesting\.
For some reason, there is always one project that is NEVER ignored. The project show up as tests to run in the Tests window with all the tests. However, all of the other .IntegrationTest. projects are correctly ignored.
I've double and triple checked the project names, namespaces, folder names, etc... Everything in this one project is spelled correctly. I've also checked the .user file in the project and it doesn't reference anything (just some empty xml). And the solution project/user file doesn't reference this project either.
Any ideas on how I can get NCrunch to exclude this one rogue project?
FWIW: I've disabled NCrunch multiple times. Toggled engine modes. Recreated the engine mode. Still the same issue. This has been going on for a few months and I finally just got annoyed enough to post here :)
Hi, thanks for sharing this issue and please accept my apologies for the frustration.
Is this a problem you're able to isolate with a sample project? It's not clear to me why this would be happening.
Something to be aware of is that when a custom engine mode excludes a test from automatic execution, it will still be shown as runnable inside the Tests Window (the 'ignore tests' option is an entirely different mechanism). This can make it hard to know which tests are included for automatic execution without deliberately making code changes to trigger them. You're probably already aware of this, but I figured I'd flag it just in case.
If I get some time, I'll see if I can create a sample solution. What I haven't done is pull the solution down into a new local repo and test. I think I'll do that first.
Appreciate the tip. You're right, I am aware of the exclude vs ignore option. FWIW: This project shows the expand icon w/ the tests, just like regular unit tests. And the other integration tests exclude the expand icon, just like a regular project (non test project). It's a bit quirky.
By chance are there any other locations I can look to see if something has been cached?
nCubed wrote:
By chance are there any other locations I can look to see if something has been cached?
It might be worth clearing out your NCrunch cache directory (_ncrunch_solution under your solution directory). Make sure you don't have the solution open when you do this.
There is only a directory named: .NCrunch_<solution.name> There is no folder for _ncrunch_<solution.name>
I did delete the directory (while VS was closed). Rebooted. Same issue.
Decided to just right-click on project in the tests window and ignore selected tests (only selected the project itself, no children). The tests that should be ignored with the filter are now ignored. But the odd thing is the manually ignored tests (project) does not show up in any of the ncrunch project/user files. It just works now. For now, I don't have much more time to dig into this further. If it pops up again, I'll just manually ignore as-needed.
nCubed wrote:There is only a directory named: .NCrunch_<solution.name> There is no folder for _ncrunch_<solution.name>
Apologies. There is a slight difference in our own dev builds where the name is difference (hence the confusion).
nCubed wrote:
I did delete the directory (while VS was closed). Rebooted. Same issue.
Decided to just right-click on project in the tests window and ignore selected tests (only selected the project itself, no children). The tests that should be ignored with the filter are now ignored. But the odd thing is the manually ignored tests (project) does not show up in any of the ncrunch project/user files. It just works now. For now, I don't have much more time to dig into this further. If it pops up again, I'll just manually ignore as-needed.
Sorry, I'll need a way to reproduce this issue so I can understand what is happening here. If you manage to whittle this down to a sample that can produce the problem, please do send it through the NCrunch contact form.