Daily Usage Issues

Nasty bug with Ignored tests

Started by Amir Kolsky on 2,081 views

I have a bunch of acceptance tests that I marked as ignored (for now).
If I make any change to these tests the tests don't run, EXCEPT
when I changed:
namespace Engine.TestAcceptance

to
namespace Engine.Tests.Acceptance


Then the ignored test gets unignored automatically and is run

Edited

Hi, thanks for sharing this issue.

When ignoring tests, we identify those tests by their structural name in the codebase.

If the name of a test changes, the selector used to ignore the test is invalidated. The namespace forms a critical part of the test's name.

We have no other way to identify the test. There is no other reliable identifier that will persist across multiple versions of an assembly while it is being changed. Thus this is caused by a technical limitation that we cannot change or work around.

If you need to change namespaces of ignored tests, I recommend setting the engine to manual mode so that you have a chance to re-ignore the tests before the engine automatically re-runs them.

Edited

Thanks for the quick reply Remco.

There may be an additional option here that can override the default behavior besides changing it.

What I usually do is have my tests split according to their frequency of execution (or at least how often I hope they run) and their purpose.

Lightweight unit tests are separate from longer running acceptance tests. This separation is enforced through different projects.

So, maybe a default behavior per project could be defined. So if I mark the entire project as disabled, any additional tests / test name / namespace changes within that project will be ignored...

That would definitely solve the problem.

Thanks!
Amir

Btw, i can't avoid asking for that elusive feature called 'check for updates :-)'
You could try placing the acceptance tests in their own category, then adjusting the tests to execute automatically setting on your engine modes so that NCrunch will never queue them for execution automatically.

You can also use the ignore this project completely setting to turn off NCrunch for your acceptance test project entirely.

NCrunch does have a built-in feature that checks for updates and notifies you when they are available (though it does not have a one-click install option).

Post a reply

Log in to reply.