Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Tests with Category Not Showing Up in Tests Window
CoolBreeze
#1 Posted : Friday, July 30, 2021 7:54:11 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 7/11/2014(UTC)
Posts: 71
Location: United States of America

Was thanked: 8 time(s) in 8 post(s)
Hi,

I just added EntityFramework (EF) to my very large solution. I updated a few modules to access an Organizations database to use EF.

I wanted to test only those updated modules using EF.

I added <Category("EntityFramework")> to three tests (Tests not TestFixtures).

I saw the three tests in the Tests window and was able to run the tests.

Then, I added <Category("EntityFramework")> to a TestFixture
in a different code file from the three tests above.

I entered EntityFramework in the Search box. Only the initial three
tests where I placed the <Category("EntityFramework")> shows up
in the Tests window.

I moved the Category attribute to an individual test. Alas, the
test doesn't show up in the Tests window.

I have a second Category attribute with the name "TestRunImpacted"
on the TestFixture where I placed the Category("EntityFramework").

I wasn't sure if multiple different Category attributes are causing the problem.

I removed <Category("TestRunImpacted")> from the fourth code file.

However, the test with the fourth EntityFramework category doesn't
show up.

Btw: I have a vague memory I asked about this problem in the past.
I did look at my topics but nothing showed.

Ed
Remco
#2 Posted : Saturday, July 31, 2021 12:18:08 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,431

Thanks: 839 times
Was thanked: 1140 time(s) in 1064 post(s)
Hi Ed,

Thanks for posting.

Can you clarify which test framework you're using?

Do you see the same result if you use NCrunch.Framework.CategoryAttribute instead?
CoolBreeze
#3 Posted : Monday, August 2, 2021 5:33:01 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 7/11/2014(UTC)
Posts: 71
Location: United States of America

Was thanked: 8 time(s) in 8 post(s)
Hi Remco,

Oops, sorry to leave off the test framework info. Here it is:

Visual Studio 2019
4.9.08 (I just upgraded last week).

I tried both versions of the attribute:

<Category("EntityFramework")
<NUnit.Framework.Category("EntityFramework")>

Neither version works.

Additional info:

I don't know if this is related: But, for better or worse, the solution
contains 30 projects. Occasionally, I have problems building the solution.
I have to Clean Solution and then Rebuild Solution multiple times to clear up
the errors until I finally get a clean build.

Also, I see several:
Possible 'lost' project references
Assemblies are being referenced from the build output directory
NUnit tests have been detected with the same name

I don't know if the above are related to the tests with the <Category()> attribute
not showing up when I search for them.
Remco
#4 Posted : Tuesday, August 3, 2021 12:03:23 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,431

Thanks: 839 times
Was thanked: 1140 time(s) in 1064 post(s)
CoolBreeze;15568 wrote:

Oops, sorry to leave off the test framework info. Here it is:

Visual Studio 2019
4.9.08 (I just upgraded last week).

I tried both versions of the attribute:

<Category("EntityFramework")
<NUnit.Framework.Category("EntityFramework")>

Neither version works.


Thanks for the extra info. Can you confirm for me if you have the 'Category' column turned on in the Tests Window? This is the cleanest way to assess which categories a test belongs to.

Note that in the NCrunch metadata, we don't actually attach categories to test fixtures. When a test fixture has a category attached to it via attribute, this category is automatically placed on its child tests instead. Fixtures themselves are basically test containers and the critical metadata is always tied to the child tests.

If you're able to produce a standalone code sample where an attached category is not being recognised on a child test, would you be able to share this with me? If I can reproduce the problem locally I should be able to understand why this is happening.

CoolBreeze;15568 wrote:

Also, I see several:
Possible 'lost' project references
Assemblies are being referenced from the build output directory


Fixing the lost project references will likely fix your intermittently failing builds in the IDE. Incorrect build sequence is one of the symptoms of using assembly references instead of project references in the solution structure.

CoolBreeze;15568 wrote:

NUnit tests have been detected with the same name


This can cause some crazy downstream issues and it might be affecting the categories if the categorised tests are represented in the NUnit output more than once under the same name. I'd recommend fixing this as a starting point, as it's something you really don't want to have in a solution.
CoolBreeze
#5 Posted : Wednesday, August 4, 2021 2:36:36 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 7/11/2014(UTC)
Posts: 71
Location: United States of America

Was thanked: 8 time(s) in 8 post(s)
Hi,

I enabled the Categories column. I see the Categories column and I see the EntityFramework Category and other Categories I've assigned to other tests in this column.

re: If you're able to produce a standalone code sample where an attached category is not being recognised on a child test, would you be able to share this with me?

I have the same problem with this sample code. Although it probably doesn't matter, I created this sample code on my personal laptop and my copy of VS 2019 with NCrunch 4.9.08 (instead of my client's desktop computer).

Main code:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Library
{
public class Experiment_Categories
{
string m_Name;

public Experiment_Categories(String Name)
{
m_Name = Name;
}

public void Print()
{
Debug.WriteLine(m_Name);
}
}
}

Test code:

using Library;
using NUnit.Framework;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Tests
{
[Category("EntityFramework")]
class TEST_Experiment_Categories
{
[Test()]
public void Print()
{
Experiment_Categories Test_Obj = new Experiment_Categories("Test");

Test_Obj.Print();
}
}
}


In the meantime, I'll work on fixing the NCrunch warnings (Possible 'lost' project references...)

Thanks, Ed
Remco
#6 Posted : Thursday, August 5, 2021 12:52:15 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,431

Thanks: 839 times
Was thanked: 1140 time(s) in 1064 post(s)
Hi Ed,

Thanks for sharing the code.

I've created a small sample project and included the source provided. In this project, NCrunch seems to correctly detect the 'EntityFramework' category and this is visible in the Categories column for the child test as shown in the Tests Window. As per usual, the fixture itself has no category showing in the list.

I had to re-read your original post to fully understand the issue here - The problem isn't that the categories aren't there, but that the search box doesn't search for them.

Right now, test categories are not included in the path for the search box. Right now we search only on test name (including namespace/fixture) and project name.

Have you tried grouping the tests by Category instead? This should provide an alternative way to find tests by category.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.256 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download