Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Getting NCrunch to load NUnit Addins
xerxesb
#1 Posted : Wednesday, November 7, 2012 10:49:15 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 11/7/2012(UTC)
Posts: 2
Location: Australia

Hi there,

We've built an NUnit addin to assist us in writing our tests (it's a thin bdd-style framework). When I try to run our tests under NCrunch, the tests fail because our addin isn't being loaded. I've tried adding the assemblies as AdditionalFiles and even copied the assemblies into the NCrunch installation directory, but to no avail.

The way we configure R# is by copying the addin assemblies to the "addins" directory of the R# installation and change R# settings to "always load addins". Is there a similar mechanism in NCrunch? If not, how can I ensure the addin will be picked up correctly by NUnit?

Thanks,
Xerxes
Remco
#2 Posted : Thursday, November 8, 2012 3:48:57 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi Xerxes -

Right now NCrunch doesn't support NUnit addins, as they require additional steps to be taken during the initialisation of the test environment in order to function.

Can you share any details about the BDD framework you're using? Perhaps I can suggest an alternative approach.


Cheers,

Remco
xerxesb
#3 Posted : Thursday, November 8, 2012 9:59:27 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 11/7/2012(UTC)
Posts: 2
Location: Australia

The framework was written a number of years ago (before most of the current ones out there). An (basic) example of its use:

Code:
public class CalculatorSpec : SpecFor<BasicCalculator>
{
        [Test]
        public void WhenAdding()
        {
            // subject is created by the base [SetUp].
            specify(() => subject.Add(1, 2) == 3);  // example 1
            specify(() => Assert.That(subject.Add(-1, -1), Is.EqualTo(-2)));  // example 2
            specify(() => subject is IArithmeticEngine); // example 3
        }
}


The framework will queue all of the expressions in the specify blocks during execution of the test, and invoke them upon teardown. This gives us all spec failures for a single test, rather than traditional frameworks which will stop after the first assertion. Additionally specify() has two overloads:
1. specify(Action) which takes in the expression and expects the expression to Assert
2. specify(Func<bool>) which allows us to write literal expressions to better express intent (examples 1&3) rather than *needing* assertion frameworks (example 2).


We could look at moving to another framework, but our project is reasonably large (4000+ tests) so it's not our first choice :)
Remco
#4 Posted : Thursday, November 8, 2012 9:08:06 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Understood. If you have a history of tests being written this way, I can see how the lack of plugin support would be a problem for you.

I could suggest a few things that may help with porting these tests over to a structure that doesn't require the plugin, though I have a feeling that you would have considered this very carefully already, and considering the size of your test suite I would guess that there could be quite a bit of work involved.

In theory, it should be possible to create a base class for each test fixture that contains the specify method, which adds specify actions to a list of actions that are then invoked during a [TearDown] method on the base class inside an exception handler.

You can then spit the test results out into the console or trace output, and return a test result based on whether or not any exceptions were thrown during the test.

I'll make a note to look into adding support for NUnit plugins in a future version of NCrunch. At the moment I'm not too sure about the effort involved in this, so it may be either a minor release feature or something that is rolled into V2. Sorry, I realise this doesn't help you much with your immediate situation, but it's the best I can offer.


Cheers,

Remco
Dribs
#5 Posted : Thursday, September 26, 2013 11:41:45 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/26/2013(UTC)
Posts: 3
Location: Australia

Hi Remco,

Was any progress made on getting NUnit plugins to work with NCrunch? I'm trying to pull AutoFixture.Nunit2 into our solution however NCrunch fails to build the test project when I do.
Remco
#6 Posted : Thursday, September 26, 2013 11:46:47 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi -

I'm afraid that the official line on this is that NUnit plugins are still not supported by NCrunch. There IS code within NCrunch that will attempt to load them in a basic scenario, but because the plugins themselves are so configurable, there is no way to warrant that NCrunch will work reliably. The cost of maintaining custom integration with every NUnit plugin that exists is simply too high, so I'm afraid that if the plugin doesn't immediately work with NCrunch, then this is unlikely to change in future.
Dribs
#7 Posted : Thursday, September 26, 2013 11:49:34 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/26/2013(UTC)
Posts: 3
Location: Australia

Thanks Remco,

where does NCrunch try to load the plugins from?
gertjvr
#8 Posted : Friday, September 27, 2013 12:00:50 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/26/2013(UTC)
Posts: 2
Location: Australia

I created the AutoFixture.NUnit2 in a few days and spent last month arguing about how nunit addins works, and people miss information about them.

All the nunit addin implementations needs is that there is a addins folder same level as nunit.core.dll that is the default location where nunit looks for addins, nunit addins have been supported for a long period (years!) and isn't as unpredictable as you belief.

A few the major test runners support them to this day.

R#
TestDriven.net

Supporting them shouldn't be difficult, nunit.core.dll has all the support already in there...
Remco
#9 Posted : Friday, September 27, 2013 12:01:42 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
In theory, NUnit should try to load from C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Remco Software\NCrunch for Visual Studio 2012\bin\addins - although admittedly I have never personally seen this work ... it may be worth a try.

The only situation I have observed NUnit successfully loading an AddIn under NCrunch is where the AddIn code was actually written inside the test assembly (i.e. a class within the test assembly implemented IAddin).
gertjvr
#10 Posted : Friday, September 27, 2013 12:05:06 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/26/2013(UTC)
Posts: 2
Location: Australia

From my experience the locations should be C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\Remco Software\NCrunch for Visual Studio 2012\addins because the base folder contains the nunit.core.dll.
Dribs
#11 Posted : Friday, September 27, 2013 12:12:48 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/26/2013(UTC)
Posts: 3
Location: Australia

No luck with either folders :(

An error occurred while analysing this project after it was built: System.NullReferenceException: Object reference not set to an instance of an object.
at NUnit.Core.NUnitFramework.IsAddinAvailable(String name)
at NUnit.Core.NUnitFramework.ApplyCommonAttributes(Attribute[] attributes, Test test)
at NUnit.Core.Builders.TestAssemblyBuilder.BuildTestAssembly(Assembly assembly, String assemblyName, IList fixtures, Boolean autoSuites)
at NUnit.Core.Builders.TestAssemblyBuilder.Build(String assemblyName, Boolean autoSuites)
at NUnit.Core.Builders.TestAssemblyBuilder.Build(String assemblyName, String testName, Boolean autoSuites)
at NUnit.Core.TestSuiteBuilder.Build(TestPackage package)
at nCrunch.TestExecution.Frameworks.NUnit.NUnitDynamicTestFinder.FindFrameworkTestsUsingRuntimeInvoke(TestPackage package, ILogger logger, List`1 tests)
at nCrunch.TestExecution.Frameworks.NUnit.NUnitTestFramework.FindFrameworkTestsInAssembly(ReflectedAssembly assembly, String assemblyFilePath, String[] referencedAssemblyFilePaths)
at nCrunch.TestExecution.TestFinder.FindTestsForFrameworks(ReflectedAssembly assembly, String assemblyFilePath, String[] referencedAssemblyFilePaths, TestFrameworkDescription[] frameworks)
at nCrunch.TestExecution.RemoteTaskRunner.AnalyseAssembly(TestFrameworkDescription[] applicableFrameworks)
Remco
#12 Posted : Wednesday, October 2, 2013 11:19:23 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
For anyone interested in getting AutoFixture.NUnit2 working with NCrunch, Gert Jansen (gertjvr) has published a way to do this by inheriting the addin into the test project.

Thanks Gert!
kjetilk
#13 Posted : Friday, April 10, 2015 8:38:37 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 11/2/2012(UTC)
Posts: 6
Location: Norway

Was thanked: 1 time(s) in 1 post(s)
Hi!

Any news on running NUnit addins with NCrunch or is Gert Jansen's receipe still the only way to do this?

I'm trying to get Concordion (http://concordion.org) tests to run in NCrunch, but this requires the Concordion NUnit-plugin.

sincerely,

Kjetil Klaussen
Remco
#14 Posted : Friday, April 10, 2015 9:22:06 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.109 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download