Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Nasty bug with Ignored tests
Options
Previous Topic Next Topic
Amir Kolsky
#1 Posted : Monday, April 5, 2021 9:34:13 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/5/2021(UTC)
Posts: 2
I have a bunch of acceptance tests that I marked as ignored (for now).
If I make any change to these tests the tests don't run, EXCEPT
when I changed:
Code:
namespace Engine.TestAcceptance

to
Code:
namespace Engine.Tests.Acceptance


Then the ignored test gets unignored automatically and is run
Back to top
 |  Edit by user
Remco
#2 Posted : Monday, April 5, 2021 10:57:47 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi, thanks for sharing this issue.

When ignoring tests, we identify those tests by their structural name in the codebase.

If the name of a test changes, the selector used to ignore the test is invalidated. The namespace forms a critical part of the test's name.

We have no other way to identify the test. There is no other reliable identifier that will persist across multiple versions of an assembly while it is being changed. Thus this is caused by a technical limitation that we cannot change or work around.

If you need to change namespaces of ignored tests, I recommend setting the engine to manual mode so that you have a chance to re-ignore the tests before the engine automatically re-runs them.
Back to top
 |  Edit by user
Amir Kolsky
#3 Posted : Tuesday, April 6, 2021 2:14:26 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/5/2021(UTC)
Posts: 2
Thanks for the quick reply Remco.

There may be an additional option here that can override the default behavior besides changing it.

What I usually do is have my tests split according to their frequency of execution (or at least how often I hope they run) and their purpose.

Lightweight unit tests are separate from longer running acceptance tests. This separation is enforced through different projects.

So, maybe a default behavior per project could be defined. So if I mark the entire project as disabled, any additional tests / test name / namespace changes within that project will be ignored...

That would definitely solve the problem.

Thanks!
Amir

Btw, i can't avoid asking for that elusive feature called 'check for updates :-)'
Back to top
Remco
#4 Posted : Tuesday, April 6, 2021 11:31:03 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
You could try placing the acceptance tests in their own category, then adjusting the tests to execute automatically setting on your engine modes so that NCrunch will never queue them for execution automatically.

You can also use the ignore this project completely setting to turn off NCrunch for your acceptance test project entirely.

NCrunch does have a built-in feature that checks for updates and notifies you when they are available (though it does not have a one-click install option).
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.037 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download