MSpec Tags are ignored with custom build engine.

I'm having difficulties preventing NCrunch from running certain MSpec tests. Some tests I mark with a Tags attribute "[Tags("functional")]" or [Tags("justmock")]. Both of these I want NCrunch to ignore so I created a custom build engine with the attribute "Does not have category" <tag>. NCrunch doesn't seem to listen to this.

Am I missing something?

Thanks!

Did you activate the newly created engine mode via the NCrunch -> Set engine mode menu?

Yes, I did. Can't seem to get this working.

Hi!

The first thing to check is whether or not NCrunch has detected the categories that you've assigned to the tests. In the Tests Window, right click the column headers and select the 'Field Chooser'. Drag the category field into the Tests Window. This field should show any category that NCrunch has detected for the tests shown in the tree. If no categories are showing, this means that somehow NCrunch hasn't properly considered the Tag attribute. If the category is showing but the engine mode isn't considering it properly, then we'll need to take a look at how NCrunch is filtering the tests around the engine mode.

Thanks for the quick response. The Tests Window does correctly recognize the Tags attributes. For initial configuration purposes, my custom engine settings are very basic and only contain a single rule: "does not have category" -> "justmock"

My test has the category [Tags("justmock")]
I'm still seeing the test being run and fail in the Tests Window, even though it recognizes the category.

Thanks, this confirms that the problem is inside the test filtering and execution rather than the category recognition - with does help to narrow things down.

I've done some extra testing at my end and couldn't reproduce any obvious problems around category filtering with this engine mode condition. Although, I am wondering a bit about the arrangement of code within the tests that you're working with. MSpec has different behaviour from the other test runners in that it cannot separate individual specifications from their parent context. This means that if any specification within the context needs to run, then ALL the specifications within the context will be run together (NCrunch cannot filter them out).

Because MSpec's Tag attribute only works at class level, it's hard to see how this could be problem ... though MSpec does also have certain constructs such as behaviours which may change how this works, and I'm also conscious of complex situations that can be created using inheritance.

Are you able to reproduce the problem using a simple example that you can share? This would help enormously with troubleshooting.

I just thought I'd also make sure to check that you had your new engine mode selected using the top NCrunch menu. I know, it's obvious, but it's very easy to forget to do this (I have trouble with it myself).


Thanks!

Remco

Thanks for helping figure this out. The bug can be reproduced when you have nested classes. MSpec allows you to use inheritance. However, if instead you nest classes, then NCrunch's engine doesn't seem to recognize or obey the Categories of the subclass. Coming from previously using NSpec, you typically nest contexts rather than use inheritance.

Brief working example:

class A
{
}

[Tags("somecategory")]
class B : A
{
Establish ctx = () => {};
Because of = () => {};
It should_do_something = () => {};
}


Bug example:

class A
{
[Tags("somecategory")]
class B
{
Establish ctx = () => {};
Because of = () => {};
It should_do_something = () => {};
}
}

[size=9]Edit:
I'm not sure the above is correct anymore. I was able to get it to temporarily ignore it, but now the bug is back. Still trying to accurately reproduce this.

Thanks. I gave this one a spin but couldn't reproduce any problems. I'm wondering if perhaps we should take a closer look at your workflow. Do you switch engine modes very regularly, or queue tests manually at any time?