Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Seperate parameterized tests
TURNSKIN
#1 Posted : Saturday, September 17, 2016 11:56:46 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/31/2015(UTC)
Posts: 18
Location: Lithuania

Thanks: 6 times
Was thanked: 2 time(s) in 2 post(s)
Hi
I'm using AutoData to parameterize my XUnit test,
e.g.
Code:

 [Theory]
        [DefaultAutoData(-4, "0")]
        [DefaultAutoData(0, "0")]
        [DefaultAutoData(1862, "5+")]
        [DefaultAutoData(5000, "5+")]
        public async Task ExportsEndOfLifeColumn(
            double endOfLifeDaysFromNow, string expected,
            [Frozen] EquipmentListModel model,
            [Frozen] IFeatureProfile featureProfile,
            DeviceRendition item,
            IReportRenderer<EquipmentListModel, DeviceRendition> renderer,
            ExportEquipmentListBuilder sut)
        {
.......
         }


NCrunch shows 1 test name ExportsEndOfLifeColumn, while Reshaprer test runner shows 4 test in this case.
NCrunch shows 1 test and puts 4 test output.
Screenshots:

vs


With Resharper I can easy see what test case fails, while with NCrunch I need to "smoke" very long console log.
Remco
#2 Posted : Sunday, September 18, 2016 12:05:47 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1287 time(s) in 1194 post(s)
Hi, thanks for sharing this issue.

Although it might look wrong from a glance, this is actually by design and it is caused by a technical limitation.

Your test has user-type parameters, which cannot uniquely identify themselves in the parameter line. For example, the DeviceRendition class may be a very complex type with no override on ToString, so there is no way for xunit to safely use it to construct an identifier for any test cases that use it.

Uniquely identifying individual test cases is extremely important for NCrunch, as NCrunch needs to be able to direct a test framework to precisely which tests must be executed within a given batch. NCrunch uses separate steps for discovery and execution, so where tests cannot be correlated between these steps, there is no way to safely execute them or account for their results between cycles of the engine. Unique identification is less important for non-concurrent test runners that don't perform batching and have less strict requirements on correlating test results.

Xunit detects this situation and then puts itself into a simplified mode, in which it will 'roll up' the test cases into their parent test. This is abstracted away from NCrunch. All NCrunch sees is the parent test.

If you want to split this test into individual test cases, you'll need to redesign it to remove all user-types from the test parameter line.

1 user thanked Remco for this useful post.
TURNSKIN on 9/27/2016(UTC)
TURNSKIN
#3 Posted : Thursday, September 22, 2016 8:58:36 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/31/2015(UTC)
Posts: 18
Location: Lithuania

Thanks: 6 times
Was thanked: 2 time(s) in 2 post(s)
THanks for your reply.
Option could be to make console log windows more advanced and group output there.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.029 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download