Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
NUnit 2 vs NUnit 3
Options
Previous Topic Next Topic
dariusdamalakas
#1 Posted : Tuesday, October 3, 2017 2:11:08 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/10/2013(UTC)
Posts: 42
Location: Lithuania

Thanks: 4 times
Was thanked: 5 time(s) in 5 post(s)
We still happen to use NUnit v2. I've noticed as of recent versions that NCrunch does not correctly split TestCases into separate tests. Below you'll see the output from a single test, event though this is 2 test cases.

With NUnit 3 (3.8.1) this works fine. Is this a regression issue, or NUnit 2 no longer supported? Happy with either answer, NUnit2 is very very old, so i don't expect this will be supported for ever (although this will be a big nuisance until we migrate to unit 3).




NCrunch: This test was executed on server '(local)'

NCrunchTestCases.ClientStoreTest.When_("a","c"):

String lengths are both 1. Strings differ at index 0.
Expected: "c"
But was: "a"
-----------^

at NUnit.Framework.Assert.That(Object actual, IResolveConstraint expression, String message, Object[] args)
at NUnit.Framework.Assert.That(Object actual, IResolveConstraint expression)
at NCrunchTestCases.ClientStoreTest.When_(String actual, String expected) in C:\work\Projects\NCrunchTestCases\NCrunchTestCases\Class1.cs:line 18


NCrunchTestCases.ClientStoreTest.When_("a","b"):

String lengths are both 1. Strings differ at index 0.
Expected: "b"
But was: "a"
-----------^

at NUnit.Framework.Assert.That(Object actual, IResolveConstraint expression, String message, Object[] args)
at NUnit.Framework.Assert.That(Object actual, IResolveConstraint expression)
at NCrunchTestCases.ClientStoreTest.When_(String actual, String expected) in C:\work\Projects\NCrunchTestCases\NCrunchTestCases\Class1.cs:line 18


One or more child tests had errors
Back to top
Remco
#2 Posted : Tuesday, October 3, 2017 11:45:48 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi, thanks for sharing this problem.

NUnit2 is still supported and probably always will be. I am actually more certain of the NUnit2 integration than I am with NUnit3, because it's stood the test of time and this framework version isn't being updated anymore.

Are you able to share this test code? I'm not aware of any problems in TestCase generation under NUnit2 (aside from the usual unique name constraints), so I'll need to be able to build a test case to reproduce the problem before I can understand it.
Back to top
dariusdamalakas
#3 Posted : Wednesday, October 4, 2017 10:26:39 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/10/2013(UTC)
Posts: 42
Location: Lithuania

Thanks: 4 times
Was thanked: 5 time(s) in 5 post(s)
A picture to illustrate this:
[img]https://ibb.co/foF7tw[/img]


Repo to reproduce this. VS 2017 with 3.10.0.20 NCrunch.

https://github.com/dariu...malakas/NCrunchTestCases
Back to top
dariusdamalakas
#4 Posted : Wednesday, October 4, 2017 10:27:56 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/10/2013(UTC)
Posts: 42
Location: Lithuania

Thanks: 4 times
Was thanked: 5 time(s) in 5 post(s)
Also, tried running tests using console runner, but that failed horribly. Looks unrelated. Thought just to run console tool to see if it behaves dfferently.


ncrunch NCrunchTestCases.sln

NCrunch Console Tool v3.11.0.9
Copyright ¸ 2010-2017 Remco Software Ltd
Usage of this tool is permitted only under the terms described in License.rtf



This product is licensed to Darius Damalakas (support/maintenance expires 15-Sep-2017)

[11:24:57.5097-?-1] Initialising UI Services
[11:24:57.5877-?-1] NCrunch Services Initialised
[11:24:57.6607-?-1] Publishing Event: [EngineModeSwitchedInUIEvent:Run all tests automatically [Global]]
[11:24:57.6627-?-1] Event [EngineModeSwitchedInUIEvent:Run all tests automatically [Global]] is being published on thread CoreThread to subscriber: ClientSettingsBlock.
[11:24:57.6717-?-1] Publishing Event: [EngineModesChangedEvent]
[11:24:57.6777-Core-4] Event [EngineModeSwitchedInUIEvent:Run all tests automatically [Global]] is being processed on Core thread with subscriber: ClientSettingsBlock.
[11:24:58.4409-Core-7] Loading NCrunch modules
[11:24:58.4509-Core-7] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\Modules\Gallio\nCrunch.Module.Gallio.crunchmodule
[11:24:58.465-Core-7] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\Modules\MSpec\nCrunch.Module.MSpec.crunchmodule
[11:24:58.4709-Core-7] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\Modules\MSTest\nCrunch.Module.MSTest.crunchmodule
[11:24:58.4759-Core-7] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\Modules\NUnit\nCrunch.Module.NUnit.crunchmodule
[11:24:58.483-Core-7] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\Modules\NUnit3\nCrunch.Module.NUnit3.crunchmodule
[11:24:58.4939-Core-7] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\Modules\XUnit2\nCrunch.Module.XUnit2.crunchmodule
[11:25:01.2669-Core-7] Framework description manager has loaded data for 6 test frameworks
[11:25:01.537-InitialisationQueueTask-14] Auto restoring NuGet packages for 'C:\work\projects\NCrunchTestCases\NCrunchTestCasesV3\NCrunchTestCasesV3.csproj'
[11:25:01.537-InitialisationQueueTask-4] Auto restoring NuGet packages for 'C:\work\projects\NCrunchTestCases\NCrunchTestCasesV2\NCrunchTestCasesV2.csproj'
[11:25:02.8183-InitialisationQueueTask-14] Loading project: C:\work\projects\NCrunchTestCases\NCrunchTestCasesV3\NCrunchTestCasesV3.csproj
[11:25:02.8183-InitialisationQueueTask-4] Loading project: C:\work\projects\NCrunchTestCases\NCrunchTestCasesV2\NCrunchTestCasesV2.csproj
[11:25:07.2896-InitialisationQueueTask-4] ERROR (Load): Error occurred during load of component at C:\work\projects\NCrunchTestCases\NCrunchTestCasesV2\NCrunchTestCasesV2.csproj: System.InvalidCastException: Specified cast is not valid.
at nCrunch.Module.NUnit.NUnitTestFrameworkUtilisationTypeIdentifier.RegisterFrameworkUtilisationTypes(IDictionary`2 utilisationTypesByFramework, ComponentUniqueName component, SettingsEngine settingsEngine)
at nCrunch.Core.TestManagement.TestFrameworkDescriptionManager.(TestFrameworkDescription[] , SnapshotComponent )
at nCrunch.Core.ComponentLoader.TestFrameworkResolutionHelper.GetTestFrameworksSelectedByComponent(SnapshotComponent component)
at nCrunch.Client.ComponentLoader.SnapshotComponentLoader.CreateComponentFromXml(FilePath projectFilePath, ParsedBuildXml projectXml, FilePath solutionFilePath, String[] additionalFilesToIncludeAtSolutionLevel, Boolean isLoadedFromFile, VisualStudioVersion vsVersion, ComponentUniqueName componentName, TaskSettings componentTaskSettings, Exception parseException, String targetFramework)

[11:25:07.2896-InitialisationQueueTask-14] ERROR (Load): Error occurred during load of component at C:\work\projects\NCrunchTestCases\NCrunchTestCasesV3\NCrunchTestCasesV3.csproj: System.InvalidCastException: Specified cast is not valid.
at nCrunch.Module.NUnit.NUnitTestFrameworkUtilisationTypeIdentifier.RegisterFrameworkUtilisationTypes(IDictionary`2 utilisationTypesByFramework, ComponentUniqueName component, SettingsEngine settingsEngine)
at nCrunch.Core.TestManagement.TestFrameworkDescriptionManager.(TestFrameworkDescription[] , SnapshotComponent )
at nCrunch.Core.ComponentLoader.TestFrameworkResolutionHelper.GetTestFrameworksSelectedByComponent(SnapshotComponent component)
at nCrunch.Client.ComponentLoader.SnapshotComponentLoader.CreateComponentFromXml(FilePath projectFilePath, ParsedBuildXml projectXml, FilePath solutionFilePath, String[] additionalFilesToIncludeAtSolutionLevel, Boolean isLoadedFromFile, VisualStudioVersion vsVersion, ComponentUniqueName componentName, TaskSettings componentTaskSettings, Exception parseException, String targetFramework)

[11:25:07.3591-Core-7] All projects have been loaded
[11:25:09.1685-?-1] Reporting engine execution results
[11:25:09.4326-?-1] Shutting down engine
[11:25:11.064-?-1] Returning result: BuildFailure
Back to top
Remco
#5 Posted : Wednesday, October 4, 2017 11:20:14 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Thanks for the code sample. This problem is appearing because you have your NUnit v2 framework utilisation type set to 'StaticAnalysis' which although is faster, it doesn't have good support for many of NUnit's more niche features, such as TestCases.

If you want to use TestCases, I recommend setting your framework utilisation type to 'DynamicAnalysis'. In this way, NCrunch will interrogate NUnit itself for the tests rather than trying to infer them using IL.

Edit: Looking at the console tool issue, I think you may actually have a completely invalid framework utilisation type declared in your config file. I'm not sure how this has happened, but changing the utilisation type in the UI should solve both problems.
Back to top
 |  Edit by user
1 user thanked Remco for this useful post.
dariusdamalakas on 10/4/2017(UTC)
dariusdamalakas
#6 Posted : Wednesday, October 4, 2017 12:18:12 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/10/2013(UTC)
Posts: 42
Location: Lithuania

Thanks: 4 times
Was thanked: 5 time(s) in 5 post(s)
Indeed it did!


Console runner runs now too.

[13:17:10.1044-LocalTestExecutionTask-25] Calling into task runner to run 1 tests:
NCrunchTestCasesV2.TestNUnitV2.When_("a","b")
[13:17:10.5579-Core-33] Successfully completed execution of 1 tests on local machine
[13:17:10.6734-LocalTestExecutionTask-34] Calling into task runner to run 1 tests:
NCrunchTestCasesV3.TestNUnitV3.When_("a","b")
[13:17:11.0999-Core-21] Successfully completed execution of 1 tests on local machine
[13:17:11.1239-LocalTestExecutionTask-23] Calling into task runner to run 1 tests:
NCrunchTestCasesV3.TestNUnitV3.When_("a","a")
[13:17:11.1464-Core-24] Successfully completed execution of 1 tests on local machine
[13:17:11.1659-LocalTestExecutionTask-22] Calling into task runner to run 1 tests:
NCrunchTestCasesV2.TestNUnitV2.When_("a","a")
[13:17:11.1749-Core-17] Successfully completed execution of 1 tests on local machine
[13:17:13.2331-?-1] Reporting engine execution results
[13:17:13.5646-?-1] Shutting down engine

Thanks for prompt response
Back to top
1 user thanked dariusdamalakas for this useful post.
Remco on 10/5/2017(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.056 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download