Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Custom Engine Mode RegEx Failing to Exclude Tests
nCubed
#1 Posted : Thursday, December 7, 2023 8:36:01 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/1/2012(UTC)
Posts: 22
Location: United States of America

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
VS 20222
NCrunch 4.18.0.3

I have several integration test projects. All of their namespaces/project follow the exact same pattern:
Abc.Xyz.IntegrationTesting.Zyx.Cba

Where the name always includes ".IntegrationTesting."

I've created a custom engine mode as follows
Code:

project name does not match regex: \.IntegrationTesting\.
and
is impacted


And have also tried w/ a 3rd condition so it reads same as above, but re-adds the first filter:
Code:

project name does not match regex: \.IntegrationTesting\.
and
is impacted
and
project name does not match regex: \.IntegrationTesting\.


For some reason, there is always one project that is NEVER ignored. The project show up as tests to run in the Tests window with all the tests. However, all of the other .IntegrationTest. projects are correctly ignored.

I've double and triple checked the project names, namespaces, folder names, etc... Everything in this one project is spelled correctly. I've also checked the .user file in the project and it doesn't reference anything (just some empty xml). And the solution project/user file doesn't reference this project either.

Any ideas on how I can get NCrunch to exclude this one rogue project?

FWIW: I've disabled NCrunch multiple times. Toggled engine modes. Recreated the engine mode. Still the same issue. This has been going on for a few months and I finally just got annoyed enough to post here :)
Remco
#2 Posted : Thursday, December 7, 2023 11:06:56 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,122

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Hi, thanks for sharing this issue and please accept my apologies for the frustration.

Is this a problem you're able to isolate with a sample project? It's not clear to me why this would be happening.

Something to be aware of is that when a custom engine mode excludes a test from automatic execution, it will still be shown as runnable inside the Tests Window (the 'ignore tests' option is an entirely different mechanism). This can make it hard to know which tests are included for automatic execution without deliberately making code changes to trigger them. You're probably already aware of this, but I figured I'd flag it just in case.

nCubed
#3 Posted : Thursday, December 7, 2023 11:19:01 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/1/2012(UTC)
Posts: 22
Location: United States of America

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
If I get some time, I'll see if I can create a sample solution. What I haven't done is pull the solution down into a new local repo and test. I think I'll do that first.

Appreciate the tip. You're right, I am aware of the exclude vs ignore option. FWIW: This project shows the expand icon w/ the tests, just like regular unit tests. And the other integration tests exclude the expand icon, just like a regular project (non test project). It's a bit quirky.

By chance are there any other locations I can look to see if something has been cached?
Remco
#4 Posted : Friday, December 8, 2023 12:59:43 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,122

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
nCubed;16967 wrote:

By chance are there any other locations I can look to see if something has been cached?


It might be worth clearing out your NCrunch cache directory (_ncrunch_solution under your solution directory). Make sure you don't have the solution open when you do this.
nCubed
#5 Posted : Monday, December 11, 2023 11:44:27 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/1/2012(UTC)
Posts: 22
Location: United States of America

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
There is only a directory named: .NCrunch_<solution.name> There is no folder for _ncrunch_<solution.name>

I did delete the directory (while VS was closed). Rebooted. Same issue.

Decided to just right-click on project in the tests window and ignore selected tests (only selected the project itself, no children). The tests that should be ignored with the filter are now ignored. But the odd thing is the manually ignored tests (project) does not show up in any of the ncrunch project/user files. It just works now. For now, I don't have much more time to dig into this further. If it pops up again, I'll just manually ignore as-needed.
Remco
#6 Posted : Tuesday, December 12, 2023 2:49:05 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,122

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
nCubed;16972 wrote:
There is only a directory named: .NCrunch_<solution.name> There is no folder for _ncrunch_<solution.name>


Apologies. There is a slight difference in our own dev builds where the name is difference (hence the confusion).

nCubed;16972 wrote:

I did delete the directory (while VS was closed). Rebooted. Same issue.

Decided to just right-click on project in the tests window and ignore selected tests (only selected the project itself, no children). The tests that should be ignored with the filter are now ignored. But the odd thing is the manually ignored tests (project) does not show up in any of the ncrunch project/user files. It just works now. For now, I don't have much more time to dig into this further. If it pops up again, I'll just manually ignore as-needed.


Sorry, I'll need a way to reproduce this issue so I can understand what is happening here. If you manage to whittle this down to a sample that can produce the problem, please do send it through the NCrunch contact form.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.043 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download