Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Rendering each test case for an XUnit Theory
Tim Long
#1 Posted : Tuesday, November 18, 2025 3:36:37 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/20/2016(UTC)
Posts: 27
Location: United Kingdom

Thanks: 2 times
Was thanked: 2 time(s) in 2 post(s)
I'm trying to use an XUnit theory, with data supplied by a DataClass, to expand a large number of scenarious (a few hundred). However, I only see one entry in the test output. I was expecting to see each "data row" listed as a test. This appears to work in Visual Studio's test runner, but not in NCrunch.

I created a simple minimal repro to illustrate:

Code:

using System.Collections;

namespace TestProject1
{
    public class Poco
    {
        public int    Id   { get; set; }
        public string Name { get; set; }

        /// <inheritdoc />
        public override string ToString() => $"{nameof(Id)}: {Id}, {nameof(Name)}: {Name}";
    }

    public class DataClass : IEnumerable<object[]>
    {
        /// <inheritdoc />
        public IEnumerator<object[]> GetEnumerator()
        {
            yield return new object[] { new Poco { Id = 1, Name = "First" } };
            yield return new object[] { new Poco { Id = 2, Name = "Second" } };
        }

        /// <inheritdoc />
        IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
    }

    public class UnitTest1
    {
        [Theory]
        [ClassData(typeof(DataClass))]
        public void Test1(Poco row) => Assert.NotNull(row);
    }
}


In Visual Studio's test runner, this "discovers" as a single test, but int the Test Detail pane, I see an entry for each "row" with its individual pass/fail result.

Quote:

 TestProject1.UnitTest1.Test1
 Source: UnitTest1.cs line 31

Test has multiple result outcomes
 1 Passed
 1 Failed

Results

2)   TestProject1.UnitTest1.Test1(row: Id: 2, Name: Second) 
Duration: 2 ms

Message: 
Assert.Equal() Failure: Values differ
Expected: 2
Actual: 1

Stack Trace: 
UnitTest1.Test1(Poco row) line 31
InvokeStub_UnitTest1.Test1(Object, Span`1)
MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

1)   TestProject1.UnitTest1.Test1(row: Id: 1, Name: First) 
Duration: 8 ms


Is there any way to get a similar result in NCrunch?
Remco
#2 Posted : Tuesday, November 18, 2025 11:12:51 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,444

Thanks: 1011 times
Was thanked: 1357 time(s) in 1260 post(s)
Hi, thanks for posting.

NCrunch doesn't support granular reporting of theories that use ClassData, because this requires the runner to deconstruct the xunit test metadata in detail (which is something we try very hard not to do, as it increases the fragility of the integration).

If reporting these tests in more detail is required, it's better to look to less dynamic methods of doing so (such as InlineData or explicit metheds).
ndeslandes
#3 Posted : Wednesday, November 19, 2025 12:37:22 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/12/2014(UTC)
Posts: 22
Location: United Kingdom

Thanks: 3 times
Was thanked: 3 time(s) in 2 post(s)
Out of curiosity, how come this works as expected, producing one test report per theory use case?

Code:

public class IntDataClass : IEnumerable<object[]>
{
    /// <inheritdoc />
    public IEnumerator<object[]> GetEnumerator()
    {
        yield return new object[] { 1, 22 };
        yield return new object[] { 42, 0 };
    }

    /// <inheritdoc />
    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}

public class IntDataUnitTests
{
    [Theory]
    [ClassData(typeof(IntDataClass))]
    public void IntTest(int x, int y) => Assert.NotEqual(42, x + y);
}
Remco
#4 Posted : Wednesday, November 19, 2025 4:51:10 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,444

Thanks: 1011 times
Was thanked: 1357 time(s) in 1260 post(s)
ndeslandes;18471 wrote:
Out of curiosity, how come this works as expected, producing one test report per theory use case?


This is because the types involved are primitives, so they are treated differently by xunit.

User types are somewhat problematic because they are harder to reliably transfer between domains/processes, so they are more constrained by the test discovery/execution lifecycle.
Remco
#5 Posted : Friday, November 28, 2025 1:48:40 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,444

Thanks: 1011 times
Was thanked: 1357 time(s) in 1260 post(s)
I apologise in providing some misleading advice above. Another user has managed to get user types enumerating with classdata by implementing IXunitSerializable - https://forum.ncrunch.net/yaf_postst3640_XUnit--Memberdata.aspx.

It seems that if serialization logic is implemented on the class, Xunit will enumerate the tests during discovery and it's possible for them to be shown separately in the NCrunch results.

My mistake was due to my lack of detailed understanding of how this is handled inside Xunit between versions (it's not actually implemented in NCrunch, this one is 'under the hood').

I urge caution when using user types in this way. It's difficult for the runner to consistently handle/report any issues that appear when transferring complex types. If an exception occurs during serialization, it might be difficult to pin it down. Anyway, I hope this helps.
1 user thanked Remco for this useful post.
ndeslandes on 11/28/2025(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.038 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download