Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Tests with TestCaseAttribute won't be marked as failing when number of arguments are wrong
Options
Previous Topic Next Topic
AnNCrunchUser
#1 Posted : Monday, October 6, 2014 11:12:48 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 10/6/2014(UTC)
Posts: 2
Location: Germany
If you take the following test:

Code:
    [TestFixture]
    public class TestCaseSourceTests
    {
        [TestCase(@"Invalid number of arguments")]
        [TestCase(@"Valid argument", 42)]
        public void TestCaseSourceTests_ProvideWrongNumberOfArguments_LetTestFail(string someText, int someInteger)
        {
        }
    }


The test will be marked with a green bubble despite the fact that one test case will be skipped.

On the other side, providing the wrong type will be marked:

Code:
    [TestFixture]
    public class TestCaseSourceTests
    {
        [TestCase(@"Invalid type of arguments", Int64.MaxValue)]
        [TestCase(@"Valid argument", 42)]
        public void TestCaseSourceTests_ProvideWrongTypeOfArguments_LetTestFail(string someText, int someInteger)
        {
        }
    }
Back to top
Remco
#2 Posted : Monday, October 6, 2014 10:49:47 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for posting!

This is behaviour as designed. NUnit itself is rejecting the invalid test case and NCrunch is never made aware of it. This means that there is no way to represent the failure, as there is no test to associate it with. The markers therefore report green for the testcase that passes without problems.

Unfortunately NCrunch is rather limited in that it can only work with the information provided by test frameworks such as NUnit. In many cases, the frameworks themselves won't report structural issues with the tests - they will simply ignore the test. In other cases, the framework presents error information that is difficult or impossible for NCrunch to represent in its UI. To be safe, always follow Red->Green->Refactor to ensure new tests you have written are working correctly before you implement the code they target.
Back to top
AnNCrunchUser
#3 Posted : Tuesday, October 7, 2014 6:30:29 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 10/6/2014(UTC)
Posts: 2
Location: Germany
Hi Remco,

okay, but i just found this issue cause Jenkins was able to report this test case as skipped.

So the interesting question would be, what makes Jenkins different to detect those faulty tests. So I'm (and probably you too) are not able to answer this question now. But maybe some Jenkins specialist out there knows and if i meet one I'll ask him and come back to give you the answer.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.031 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download