Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

AfterTestRun Method from SpecFlow is never Called if Tests run with NCrunch
Daniel Zimmermann
#1 Posted : Tuesday, February 2, 2016 7:36:03 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/2/2016(UTC)
Posts: 1
Location: Switzerland

Thanks: 1 times
I discovered that NCrunch never runs our AfterTestRun method.
If i run the tests with ReSharper TestRunner, the method gets called.
Is there a configuration that can fix this problem?

I made a sample project to reproduce the problem, unfortunately I cannot upload it with this post.
But it should be easy to reproduce:
1. Create a new class library project in VS2013
2. Install nuget package nunit and specflow
3. Add a new class with the following source code:

Code:


using System;
using System.Diagnostics;
using NUnit.Framework;
using TechTalk.SpecFlow;

namespace SpecFlowTest
{
    [Binding]
    public static class TestContext
    {
        [BeforeScenario]
        public static void BeforeScenario()
        {
            Debug.WriteLine("Before scenario");
        }

        [AfterScenario]
        public static void AfterScenario()
        {
            Debug.WriteLine("After scenario");
        }

        [BeforeTestRun]
        public static void SetUpFeature()
        {
            Debug.WriteLine("Before test run");
        }

        [AfterTestRun]
        public static void AfterTestRun()
        {
            Debug.WriteLine("After test run");
        }
    }
}



4. Create a test to execute
5. Configure ncrunch
6. Set a breakpoint in the AfterTestRun method
7. Debug the test with ncrunch and with resharper to see the differene

Edit:
NCrunch Version: 2.16.0.13

I hope I am not the only one that has this problem.

Regards

Daniel
Remco
#2 Posted : Tuesday, February 2, 2016 12:18:58 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi Daniel,

Thanks for sharing this issue. I've managed to reproduce it by following your instructions.

The problem is being caused by NUnit being unable to load the SpecFlow NUnit plugin which is used to hook the shut-down of the test environment. NCrunch doesn't officially support NUnit plugins, as the way in which these plugins work can make it impossible to warrant that the test environment will behave the way that it should. I grant that SpecFlow itself is a bit of a special case considering it is a test framework itself, but it looks like work will be required to allow such a hook to work. At this stage, I am not certain whether implementing support for this feature is feasible.

I recommend steering away from using such a hook-point to execute code after your tests have run. I'm not sure what you are using this hook for. If you are using it to clean up resources at the end of a test run, I recommend considering a different design.

Even when this feature is working correctly, there is no guarantee that the cleanup code at the end of a test run will always be called. For example, the process may experience a stack overflow, out of memory exception, or any other unexpected problem that destabilises it and causes it to terminate unexpectedly. This could be as simple as a debug session that is interrupted. A good design for cleaning up resources created by frequently executed tests is often to ensure that tests clean up any stale resources at the beginning of their execution. In this way, you can be sure the test will always start with a clean slate.
1 user thanked Remco for this useful post.
Daniel Zimmermann on 2/2/2016(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.032 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download