Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
NCrunch reports Xunit tests failed when using custom [Fact] attribute and custom IXunitTestCase
Options
Previous Topic Next Topic
ClausAppel3Shape
#1 Posted : Tuesday, April 21, 2020 11:26:27 AM(UTC)
Rank: Member

Groups: Registered
Joined: 11/21/2019(UTC)
Posts: 10
Location: Denmark

Thanks: 2 times
Was thanked: 2 time(s) in 1 post(s)
In my C# project we use custom attributes as an alternative to Xunit's [Fact] and [Theory], and we use a a custom implementation of IXunitTestCase. This causes tests with these attributes to be marked as red by NCrunch with the message:
"This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures."

The reason we use these custom implementations has to do with synchronization contexts, but that's not what's breaking NCrunch. I've made a minimal failing example that shows the tests green in Visual Studio but red in NCrunch. It changes virtually no functionality; it just wraps existing functionality in a custom class.

Code: https://github.com/CAP-3...omAttribute.NCrunchTest

I'm using NCrunch 4.3.0.13, Visual Studio Professional 2019 version 16.4.6., C# 8.

Can anyone help me figure out whether this is fixable on my end or a bug in NCrunch? Thanks!
Back to top
Remco
#2 Posted : Tuesday, April 21, 2020 12:56:12 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for sharing this issue.

For us to be able to parse the Xunit test model and make sense of the test results that come from it, we need to make certain assumptions about how the model is structured and the types that are involved. Intrinsically, it is impossible for us to cater for all possible extension scenarios, particularly when custom attributes are implemented that override logic within Xunit.

The code that you've implemented effectively extends Xunit in a way that is building your own test framework. I appreciate that the scope of extension is probably fairly small when compared with Xunit as a whole, but there is simply no way we can feasibly support this kind of extension. Sorry, if you want to use NCrunch, you'll need to use the standard Xunit attributes and structure.
Back to top
ClausAppel3Shape
#3 Posted : Tuesday, April 21, 2020 1:00:56 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/21/2019(UTC)
Posts: 10
Location: Denmark

Thanks: 2 times
Was thanked: 2 time(s) in 1 post(s)
Remco;14649 wrote:
Hi, thanks for sharing this issue.

For us to be able to parse the Xunit test model and make sense of the test results that come from it, we need to make certain assumptions about how the model is structured and the types that are involved. Intrinsically, it is impossible for us to cater for all possible extension scenarios, particularly when custom attributes are implemented that override logic within Xunit.

The code that you've implemented effectively extends Xunit in a way that is building your own test framework. I appreciate that the scope of extension is probably fairly small when compared with Xunit as a whole, but there is simply no way we can feasibly support this kind of extension. Sorry, if you want to use NCrunch, you'll need to use the standard Xunit attributes and structure.

All right. Thanks for the reply. :(
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.032 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download