Test category not working with MSTest

I was glad to read that test categories are now supported under version 1.38b. However it looks like the TestCategory attribute used by MSTest is not working. When I add the Category column in NCrunch Tests window the value is empty even for tests that have the <TestCategory> attribute.

Are MSTest's categories going to be supported in a future version? I wish there will be also a way to exclude tests by category in NCrunch too.

Keep up with the good job. NCruch is a wonderful product!

David

Hi David,

Thanks for posting! Yes, NCrunch is supposed to support this attribute, but it looks like there is a problem here.

As a workaround, you should be able to use the TestProperty attribute instead, as such:

TestProperty("Category", "MyCategory")


Cheers,

Remco

Thanks Remco for the quick answer. Although not ideal, the workaround works just fine.

Is the "Exclude tests by category" feature going to be part of the next release of NCrunch?

David

It's already there :)

You can create a custom engine mode that will exclude tests from automatically execution by their category - https://www.ncrunch.net/documentation/concepts_engine-modes

Hi Remco,

I experimented this morning with the Custom Engine Mode to check if I could exclude some tests based on their category. I think I found a bug with how NCrunch manage test categories in VB.NET. Let met describe the problem.

Let's say I have a test class with two test methods. If I add the category attribute (using the provided workaround) to one of the method, both methods are shown to be within the "Category1" in the NCrunch Tests window

Public Class UnitTest1

<TestMethod()>
Public Sub TestMethod1()
' TODO: Add test logic here
End Sub

<TestProperty("Category", "Category1")>
<TestMethod()>
Public Sub TestMethod2()
End Sub

End Class


The official documentation from MSDN says the TestProperty attribute is supposed to "Establishes a test specific property on a method." It should not cause the property to be propagated to all the methods of the class. When using the <TestCategory> attribute the VS Tests window correctly shows that only TestMethod2 is categorized.

I'm I doing something wrong?

DB

Hi David,

Thanks for reporting this - it also looks like a bug. For some reason the category resolver is duplicating the categories between methods. This also seems to be the case when using the NCrunch.Framework.Category attribute, which was expected to be the workaround... My assumption at this stage is that this is a problem with NCrunch's MSTest provider.

Sorry, I'll try to arrange for a fix in the next release.


Cheers,

Remco

Hi Remco,

NUnit categories are being ignored as well. Trying to exclude tests with attribute [Category("Integration")] isn't working. Doesn't matter if the attribute is on the method or class.

Otherwise I'm loving NCrunch and looking forward to next release.

T.

Hi Tom,

Thanks for reporting this. Would you be able to share a code sample for the above? I can't reproduce this problem for NUnit.


Cheers,

Remco

Hi Remco,

Sorry seems I was wrong, they aren't infact being run but are just being listed as "Pending, Impacted" instead of being ignored completely. Not a major issue, just means the Tests window looks a bit cluttered.

Thanks,

T.