Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

xunit v3 private test methods not discovered
Argamon
#1 Posted : Tuesday, April 29, 2025 9:26:42 PM(UTC)
Rank: Member

Groups: Registered
Joined: 12/4/2018(UTC)
Posts: 15
Location: Germany

Thanks: 1 times
Was thanked: 2 time(s) in 2 post(s)
I am currently evaluating an upgrade from xunit v2 to xunit v3.

So far everything looks good, but NCrunch does not discover the tests anymore.

The reason is that all our test classes are public, but our methods are private. Dont ask why, it is a coding convention where I work.

With v2 NCrunch discovers all tests, with v3 none are found.

It is no changed behavior in v3, cause when I do a "dotnet test" all tests work. When I make a method public NCrunch also works.

With more than 20k tests I dont want to look at every class.

I guess there is nothing else I can do besides waiting for a new version?
Remco
#2 Posted : Wednesday, April 30, 2025 1:56:35 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,312

Thanks: 990 times
Was thanked: 1329 time(s) in 1232 post(s)
Thanks for sharing this issue.

I've reproduced it as described. NCrunch is not detecting private tests under Xunit3.

I've prepared a build with a fix if you are interested in giving it a try:

NCrunch_Console_5.15.0.1.msi
NCrunch_Console_5.15.0.1.zip
NCrunch_GridNodeServer_5.15.0.1.msi
NCrunch_GridNodeServer_5.15.0.1.zip
NCrunch_LicenseServer_5.15.0.1.zip
NCrunch_Rider_5.15.0.1.7z
NCrunch_Rider_5.15.0.1.zip
NCrunch_VS2010_5.15.0.1.msi
NCrunch_VS2010_5.15.0.1.zip
NCrunch_VS2012_5.15.0.1.msi
NCrunch_VS2012_5.15.0.1.zip
NCrunch_VS2013_5.15.0.1.msi
NCrunch_VS2013_5.15.0.1.zip
NCrunch_VS2015_5.15.0.1.msi
NCrunch_VS2015_5.15.0.1.msi.7z
NCrunch_VS2015_5.15.0.1.zip
NCrunch_VS2017_5.15.0.1.msi
NCrunch_VS2017_5.15.0.1.msi.7z
NCrunch_VS2017_5.15.0.1.zip
NCrunch_VS2019_5.15.0.1.msi
NCrunch_VS2019_5.15.0.1.msi.7z
NCrunch_VS2019_5.15.0.1.zip
NCrunch_VS2022_5.15.0.1.msi
NCrunch_VS2022_5.15.0.1.msi.7z
NCrunch_VS2022_5.15.0.1.zip

I'll note that I don't generally recommend making test methods private. I suspect Xunit allows this just to simplify declaration syntax, but it's quite an unusual thing. Test methods are external entry points for code. Regardless, NCrunch is still supposed to work with them being private.
Argamon
#3 Posted : Wednesday, April 30, 2025 8:58:11 AM(UTC)
Rank: Member

Groups: Registered
Joined: 12/4/2018(UTC)
Posts: 15
Location: Germany

Thanks: 1 times
Was thanked: 2 time(s) in 2 post(s)
Wow, that was fast. I hope it wasnt more work than just some different BindingFlags. It works like charm. Thanks.

I know, I am not a big fan of this design, but it has been done like this for ages in the company I work for.

The problem is that there is running some kind of code analysis in the background enforcing rulesets for public methods. The rulesets for test methods vary a lot from these. So by making the methods private, the problem was solved.

We both now that the better solution would have been to make the analysis aware of tests and enforce a different ruleset for tests, but 8 years and 20k tests later this is hard to change now.
Remco
#4 Posted : Wednesday, April 30, 2025 10:03:38 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,312

Thanks: 990 times
Was thanked: 1329 time(s) in 1232 post(s)
Argamon;18095 wrote:
Wow, that was fast. I hope it wasnt more work than just some different BindingFlags. It works like charm. Thanks.


That was, quite literally, all it was. Nice to have something easy to fix for a change. I'm glad it works for you.

Argamon;18095 wrote:

We both now that the better solution would have been to make the analysis aware of tests and enforce a different ruleset for tests, but 8 years and 20k tests later this is hard to change now.


I share your familiarity with this problem. Certain things become hard to justify changing when projects start spanning into the decades.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.041 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download