Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Error not picked up
Teejwex
#1 Posted : Friday, May 22, 2026 5:30:52 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 5/22/2026(UTC)
Posts: 4

Hi,

I am passing some complex objects to an xunit test. I see in other posts that NCrunch will collapse these test and show a single result.

The problem that i have though is that single result was showing a pass. When i used VS2022 test runner it correctly showed a fail for one of the tests.

In my case it will return the result of the last test and hide any earlier failures - not ideal. The RDI shows that the method evaluation and the expected result were different values and shows the full set of calls made to the test method.

Is there a way to resolve this? I tried the suggestion of using IXunitSerializable from one of the other posts but the tests took way too long to run and all showed a fail then.


Code:
    [Theory]
    [ClassData(typeof(GetChosenSummerSelection))]
    internal void MixedLeagueClass_ShouldBeBasedOnSummerLeagueSelectionAndNomination(
        TestCase data)
    {
        var result = CallEvaluate(
            data.MixedLeagueRank,
            data.SummerSelectedDivisions,
            data.SummerNominatedRank);
        result.Should().Be(data.Expected);
    }

...

    internal class GetChosenSummerSelection : TheoryData<TestCase>
    {
        public GetChosenSummerSelection()
        {
            Add(new TestCase(10, "", DR.Class1, [CreateSelected(DR.Class2, 1)], null, true));
            Add(new TestCase(20, "", DR.Class1, [CreateSelected(DR.Class3, 1)], null, true));
        }
    }


This shows the results

[img]null[/img]test runner


Thanks
Remco
#2 Posted : Saturday, May 23, 2026 12:10:11 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,559

Thanks: 1033 times
Was thanked: 1387 time(s) in 1287 post(s)
Hi, thanks for sharing this.

A test that hides a failing result is definitely not ok. Can you share the declaration of your TestCase type? I think what's happening here is that the two tests are being internally identified as the same test, because the runner has no way to tell them apart (TestCase.ToString() returns the same value). So NCrunch is simply taking the last result surfaced from the framework and isn't acknowledging that there are actually two tests here.
Teejwex
#3 Posted : Saturday, May 23, 2026 8:27:16 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 5/22/2026(UTC)
Posts: 4

Here's a stripped back version of the test and hopefully all of the code needed to replicate it. Test case 10 should return false, i have changed it to true here to highlight the issue.
Test case number 10 should fail here but the result of test case 20 is all that I see.

Code:
public class EvaluateTests
{
    [Theory]
    [ClassData(typeof(GetChosenSummerSelection))]
    internal void MixedLeagueClass_ShouldBeBasedOnSummerLeagueSelectionAndNomination(
        TestCase data)
    {
        var result = CallEvaluate(
            data.MixedLeagueRank,
            data.SummerSelectedDivisions,
            data.SummerNominatedRank);
        result.Should().Be(data.Expected);
    }
 
    private bool CallEvaluate(
        DltcDivisionRank mixedDivision,
        List<SummerSelectedDivision> summerSelectedDivisions,
        DltcDivisionRank? nominatedSummerDivision)
    {
        var winterLeagueRule = new MixedLeagueSummerSelectionRule();

        return winterLeagueRule.Evaluate(
            mixedDivision, 
            summerSelectedDivisions,
            nominatedSummerDivision
        );
    }

    internal class GetChosenSummerSelection : TheoryData<TestCase>
    {
        public GetChosenSummerSelection()
        {
            Add(new TestCase(10, DR.Class1, [CreateSelected(DR.Class2, 1)], null, true));
            Add(new TestCase(20, DR.Class1, [CreateSelected(DR.Class3, 1)], null, true));

        }

        private static SummerSelectedDivision CreateSelected(DR rank, int timesSelected)
            => new() { Rank = rank, TimesSelected = timesSelected };
    }

    internal record TestCase(
        int TestNumber,
        DR MixedLeagueRank,
        List<SummerSelectedDivision> SummerSelectedDivisions,
        DR? SummerNominatedRank,
        bool Expected)
    {
        public override string ToString() => $"{TestNumber}";
    }
}



Code:
internal enum DltcDivisionRank
{
    Premier = 1,
    Class1 = 2,
    Class2 = 3,
    Class3 = 4,
    Class4 = 5,
    Class5 = 6,
    Class6 = 7,
    Class7 = 8,
    Class8 = 9,
    Class9 = 10,
}


Code:
using DR = DltcDivisionRank;

internal class MixedLeagueSummerSelectionRule
{
    public bool Evaluate(
        DltcDivisionRank mixedDivision,
        List<SummerSelectedDivision> selectedDivisions,
        DltcDivisionRank? nominatedSummerDivision)
    {
        return (mixedDivision, RecognisedSummerDivision()) switch
        {
            var (m, s) when m == DR.Class1 && s < DR.Class3 => false,
            var (m, s) when m == DR.Class2 && s < DR.Class5 => false,
            var (m, s) when m == DR.Class3 && s < DR.Class6 => false,
            var (m, s) when m == DR.Class4 && s < DR.Class7 => false,
            _ => true,
        };

        DR? RecognisedSummerDivision()
        {
            int totalPlayed = 0;
            DR? selectedDivisionRank = null;

            foreach (var selectedDivision in selectedDivisions.OrderBy(sd => sd.Rank))
            {
                totalPlayed += selectedDivision.TimesSelected;
                if (totalPlayed > 1)
                {
                    selectedDivisionRank = selectedDivision.Rank;
                    break;
                }
            }

            if (SelectedRankIsHigher())
                return selectedDivisionRank;

            return nominatedSummerDivision
                ?? selectedDivisionRank
                ?? selectedDivisions.FirstOrDefault()?.Rank;

            bool SelectedRankIsHigher()
            {
                if (nominatedSummerDivision == null)
                    return false;
                if (selectedDivisionRank == null)
                    return false;

                return selectedDivisionRank < nominatedSummerDivision;
            }
        }
    }
}


Code:
internal class SummerSelectedDivision
{
    public DltcDivisionRank Rank { get; set; }
    public int TimesSelected { get; set; }
}


Teejwex
#4 Posted : Saturday, May 23, 2026 8:28:48 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 5/22/2026(UTC)
Posts: 4

Not sure that was clear. The real expected result for Test Case 10 is false. The code above should fail on test case 10 because i set it to true here to show the issue.
Remco
#5 Posted : Monday, May 25, 2026 6:27:18 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,559

Thanks: 1033 times
Was thanked: 1387 time(s) in 1287 post(s)
Sorry for the delayed response on this. I'm taking a look at the problem and will get back to you soon.
Remco
#6 Posted : Tuesday, May 26, 2026 7:35:46 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,559

Thanks: 1033 times
Was thanked: 1387 time(s) in 1287 post(s)
Teejwex
#7 Posted : Tuesday, May 26, 2026 9:46:31 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 5/22/2026(UTC)
Posts: 4

Thanks for looking into it quickly. Under a bit of pressure now but I'll definitely give it a test again in the next couple of days.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.043 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download