Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Rendering each test case for an XUnit Theory
Tim Long
#1 Posted : Tuesday, November 18, 2025 3:36:37 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/20/2016(UTC)
Posts: 27
Location: United Kingdom

Thanks: 2 times
Was thanked: 2 time(s) in 2 post(s)
I'm trying to use an XUnit theory, with data supplied by a DataClass, to expand a large number of scenarious (a few hundred). However, I only see one entry in the test output. I was expecting to see each "data row" listed as a test. This appears to work in Visual Studio's test runner, but not in NCrunch.

I created a simple minimal repro to illustrate:

Code:

using System.Collections;

namespace TestProject1
{
    public class Poco
    {
        public int    Id   { get; set; }
        public string Name { get; set; }

        /// <inheritdoc />
        public override string ToString() => $"{nameof(Id)}: {Id}, {nameof(Name)}: {Name}";
    }

    public class DataClass : IEnumerable<object[]>
    {
        /// <inheritdoc />
        public IEnumerator<object[]> GetEnumerator()
        {
            yield return new object[] { new Poco { Id = 1, Name = "First" } };
            yield return new object[] { new Poco { Id = 2, Name = "Second" } };
        }

        /// <inheritdoc />
        IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
    }

    public class UnitTest1
    {
        [Theory]
        [ClassData(typeof(DataClass))]
        public void Test1(Poco row) => Assert.NotNull(row);
    }
}


In Visual Studio's test runner, this "discovers" as a single test, but int the Test Detail pane, I see an entry for each "row" with its individual pass/fail result.

Quote:

 TestProject1.UnitTest1.Test1
 Source: UnitTest1.cs line 31

Test has multiple result outcomes
 1 Passed
 1 Failed

Results

2)   TestProject1.UnitTest1.Test1(row: Id: 2, Name: Second) 
Duration: 2 ms

Message: 
Assert.Equal() Failure: Values differ
Expected: 2
Actual: 1

Stack Trace: 
UnitTest1.Test1(Poco row) line 31
InvokeStub_UnitTest1.Test1(Object, Span`1)
MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

1)   TestProject1.UnitTest1.Test1(row: Id: 1, Name: First) 
Duration: 8 ms


Is there any way to get a similar result in NCrunch?
Remco
#2 Posted : Tuesday, November 18, 2025 11:12:51 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,420

Thanks: 1004 times
Was thanked: 1351 time(s) in 1254 post(s)
Hi, thanks for posting.

NCrunch doesn't support granular reporting of theories that use ClassData, because this requires the runner to deconstruct the xunit test metadata in detail (which is something we try very hard not to do, as it increases the fragility of the integration).

If reporting these tests in more detail is required, it's better to look to less dynamic methods of doing so (such as InlineData or explicit metheds).
ndeslandes
#3 Posted : Wednesday, November 19, 2025 12:37:22 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/12/2014(UTC)
Posts: 11
Location: United Kingdom

Thanks: 2 times
Was thanked: 3 time(s) in 2 post(s)
Out of curiosity, how come this works as expected, producing one test report per theory use case?

Code:

public class IntDataClass : IEnumerable<object[]>
{
    /// <inheritdoc />
    public IEnumerator<object[]> GetEnumerator()
    {
        yield return new object[] { 1, 22 };
        yield return new object[] { 42, 0 };
    }

    /// <inheritdoc />
    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}

public class IntDataUnitTests
{
    [Theory]
    [ClassData(typeof(IntDataClass))]
    public void IntTest(int x, int y) => Assert.NotEqual(42, x + y);
}
Remco
#4 Posted : Wednesday, November 19, 2025 4:51:10 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,420

Thanks: 1004 times
Was thanked: 1351 time(s) in 1254 post(s)
ndeslandes;18471 wrote:
Out of curiosity, how come this works as expected, producing one test report per theory use case?


This is because the types involved are primitives, so they are treated differently by xunit.

User types are somewhat problematic because they are harder to reliably transfer between domains/processes, so they are more constrained by the test discovery/execution lifecycle.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.031 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download