Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

NUnit tests marked with Platform attribute are treated as inconclusive when conditions not met
nshallred
#1 Posted : Thursday, May 7, 2026 4:48:14 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 5/5/2016(UTC)
Posts: 6
Location: United Kingdom

Thanks: 2 times
Was thanked: 1 time(s) in 1 post(s)
I'm running a project using the following versions

Visual Studio 2026 (18.5.2)
NCrunch (5.21.0.10)
Nunit (4.6.0)

I encountered an issue with a test marked with NUnit's PlatformAttribute. The intention was that these tests would not run when running under Windows.

It seems that NCrunch treats the result of tests so marked as Inconclusive rather than Ignore; consequently, with default settings, the test shows up as being in error (when it simply hasn't run when running on Windows).
I'm aware of the "Consider inconclusive tests as passing" setting but that seems a bit of a blunt instrument in this scenario (I'm perfectly happy with genuinely inconclusive tests showing as errors).

This seems like a potential issue to me; could you please advise?
Code:
namespace VS2022UnitTests
{
    [TestFixture]
    public sealed class NCrunchIssueTests
    {
        [Test]
        [Platform(Include = "Win", Reason = "Demonstrate test running on Windows")]
        public void TestRunsOnWindows()
        {
            Assert.Pass("This test does run on Windows");
        }

        [Test]
        [Platform(Include = "Linux", Reason = "Demonstrate test running on Linux")]
        public void TestRunsOnLinux()
        {
            Assert.Pass("This test does runs on Linux");
        }

        [Test]
        [Platform(Exclude = "Win", Reason = "Demonstrate test not running on Windows")]
        public void TestDoesNotRunOnWindows()
        {
            Assert.Pass("This test does not run on Windows");
        }

        [Test]
        [Platform(Exclude = "Linux", Reason = "Demonstrate test not running on Linux")]
        public void TestDoesNotRunOnLinux()
        {
            Assert.Pass("This test does not  runs on Linux");
        }

        [Test]
        public void TestIsInclusive()
        {
            Assert.Inconclusive("Demonstrate what happens with an inconclusive test.");
        }

        [Test]
        [Ignore("Show what happens with an ignored test")]
        public void TestIsIgnored()
        {
            Assert.Fail("This test should be ignored");
        }
    }


The Visual Studio Test runner also appears to have problems distinguishing (tests "TestDoesNotRunOnWindows", "TestsRunsOnLinux" and "TestIsInconclusive" are all marked with Not Run/Blue exclamation mark).

Resharper does better, marking the two excluded platform tests as Ignored. It seems to be able to distinguish from the genuine inconclusive test.
Remco
#2 Posted : Thursday, May 7, 2026 11:53:40 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,545

Thanks: 1028 times
Was thanked: 1379 time(s) in 1280 post(s)
Hi, thanks for sharing this issue.

At first glance, this looks like a simple open and shut case. The tests in question are marked to run on a different platform from the one being used, so they should just be ignored and not discovered.

It starts to get more complicated when getting under the hood.

Unfortunately, NUnit does not share the platform constraint with the runner. It processes it internally and does not provide metadata that allows a runner to separate it from other 'skip' states that can be surfaced during discovery. This means that the only way for NCrunch to know that the tests shouldn't be run on the current platform would be to break the framework's abstraction, go around the API and try to reconstruct NUnit's logic by physically reading the test out of the assembly structure. This is a terrible approach because it risks inconsistencies during test discovery, future breakages, and it's also really complex. So basically not worth the risk/damage of implementing.

My suggestion would be to avoid using PlatformAttribute and just stick with compiler conditionals instead. Put the linux based tests together in the file, and use a #if directive around them so that they are only built and run in the correct environment. Likewise with the windows based tests.
1 user thanked Remco for this useful post.
nshallred on 5/8/2026(UTC)
nshallred
#3 Posted : Friday, May 8, 2026 6:54:59 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 5/5/2016(UTC)
Posts: 6
Location: United Kingdom

Thanks: 2 times
Was thanked: 1 time(s) in 1 post(s)
Hi, thanks for the quick response.

I was afraid that it would be something like that. I appreciate that trying to work around the limitations of the Platform attribute is fraught with peril.
1 user thanked nshallred for this useful post.
Remco on 5/9/2026(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.031 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download