Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
NUnit Tests with TestCaseSource are failing
Options
Previous Topic Next Topic
danielp37
#1 Posted : Wednesday, June 11, 2014 4:55:19 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/11/2014(UTC)
Posts: 1
Location: United States of America
I am running NCrunch 2.7.0.5 and NUnit 2.6.3.13283. It seems like most of our tests that use the TestCaseSource attribute are failing with the message: The parent fixture of this test experienced a failure during test execution.

When I run the tests in debug, I'm not seeing any exceptions being thrown and the tests just don't get run.

I tried duplicating this in an isolated project and haven't been able to get it to not work.

I turned off the dynamic analysis for NUnit and still had failures except that now it just told me that the tests were inconclusive.

The test run just fine in resharper and in the nunit-gui runner.

Any ideas?
Back to top
Remco
#2 Posted : Wednesday, June 11, 2014 5:35:55 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,980

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for sharing this issue.

Let's see if we can get a more descriptive error.

In your global NCrunch configuration, set the 'Log Verbosity' to 'Detailed', then try running one of the failing tests.

Open up the processing queue, then find the test task responsible for the test run you just executed.

Take a look at the trace output at the bottom of the processing queue when you have this task selected. Do you see anything interesting? Exception information perhaps?

Something else worth checking is to make sure you're not experiencing a failure on the analysis step. TestCaseSource executes code during the 'Analysis Task' for the respective project. If this code fails, you can receive an analysis error. This will usually be visible against the project itself (in much the same way as a build error).
Back to top
 |  Edit by user
Marco Eckstein
#3 Posted : Wednesday, September 3, 2014 1:04:12 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/3/2014(UTC)
Posts: 2
I think I have a similar problem. When the method TestCaseSource is referring to throws an exception, the test seems to be completely invisible to NCrunch, no matter whether in debug or release mode.
I am using NCrunch for Visual Studio 2010 2.7.0.5, with UseDynamicAnalysis for NUnit.

Code:
using System;
using NUnit.Framework;

namespace ConsoleApplication1
{
    [TestFixture]
    public class TestNCrunchIsBroken
    {
        private static TestCaseData[] Working()
        {
            return new [] { new TestCaseData("") };
        }

        private static TestCaseData[] Failing()
        {
            throw new Exception();
        }

        [Test]
        [TestCaseSource("Working")]
        public void WorkingTest(string x)
        {
        }

        // This test does not show up in NCrunch View, but fails in ReSharper.
        [Test]
        [TestCaseSource("Failing")]
        public void FailingTest(string x)
        {
        }
    }
}
Back to top
 |  Edit by user
Remco
#4 Posted : Wednesday, September 3, 2014 11:54:51 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,980

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi,

This problem is related to the time when the exception is thrown.

NCrunch's test discovery occurs during a separate analysis step where no context exists to record data for individual tests. Use of the TestCaseSource attribute allows you to run code inside this discovery step, but there is no data that NCrunch can use to correlate the TestCaseSource failures/execution with existing tests - as the tests haven't been discovered yet.

If you wish to see error information for the TestCaseSourceAttribute, you need to set the 'Framework utilisation type for NUnit' to 'StaticAnalysis'. This will in some ways bring NCrunch's handling of the NUnit framework closer to how Resharper does it, using pre-execution analysis to discover tests. Note that changing this setting will make a considerable difference in the behaviour of the framework around TestCaseSource.


Cheers,

Remco
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.037 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download