Daily Usage Issues

Seperate parameterized tests

Started by TURNSKIN on 5,414 views

Hi
I'm using AutoData to parameterize my XUnit test,
e.g.

 [Theory]
        [DefaultAutoData(-4, "0")]
        [DefaultAutoData(0, "0")]
        [DefaultAutoData(1862, "5+")]
        [DefaultAutoData(5000, "5+")]
        public async Task ExportsEndOfLifeColumn(
            double endOfLifeDaysFromNow, string expected,
            [Frozen] EquipmentListModel model,
            [Frozen] IFeatureProfile featureProfile,
            DeviceRendition item,
            IReportRenderer<EquipmentListModel, DeviceRendition> renderer,
            ExportEquipmentListBuilder sut)
        {
.......
         }


NCrunch shows 1 test name [h]ExportsEndOfLifeColumn[/h], while Reshaprer test runner shows 4 test in this case.
NCrunch shows 1 test and puts 4 test output.
Screenshots:
External image
vs
External image

With Resharper I can easy see what test case fails, while with NCrunch I need to "smoke" very long console log.

Edited

Hi, thanks for sharing this issue.

Although it might look wrong from a glance, this is actually by design and it is caused by a technical limitation.

Your test has user-type parameters, which cannot uniquely identify themselves in the parameter line. For example, the DeviceRendition class may be a very complex type with no override on ToString, so there is no way for xunit to safely use it to construct an identifier for any test cases that use it.

Uniquely identifying individual test cases is extremely important for NCrunch, as NCrunch needs to be able to direct a test framework to precisely which tests must be executed within a given batch. NCrunch uses separate steps for discovery and execution, so where tests cannot be correlated between these steps, there is no way to safely execute them or account for their results between cycles of the engine. Unique identification is less important for non-concurrent test runners that don't perform batching and have less strict requirements on correlating test results.

Xunit detects this situation and then puts itself into a simplified mode, in which it will 'roll up' the test cases into their parent test. This is abstracted away from NCrunch. All NCrunch sees is the parent test.

If you want to split this test into individual test cases, you'll need to redesign it to remove all user-types from the test parameter line.

THanks for your reply.
Option could be to make console log windows more advanced and group output there.

Post a reply

Log in to reply.