Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

V3 support for [DataRow] in MSTest V2 is not working
billgr
#1 Posted : Thursday, February 9, 2017 7:49:26 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 12/4/2016(UTC)
Posts: 1
Location: United States of America

With the release of V3 back in November, we got support for the new MSTest. From the release notes: NCrunch now also has support for the new version of MSTest, including its assembly-level test categories and row tests.

But row tests aren't working for me. Do they only work in VS 2017? Do they only work with .Net Core?

In VS 2015, NCrunch 3.3.0.6 is only running the first two test methods in the repro-case below. The second two test cases that have [DataRow] attributes aren't getting called and show up with white NCrunch "not-run" dots on the left side. MSTest v2 does run them and the Test Explorer shows a separate result for each [DataRow].

using System;
using ClassLibrary1;
using FluentAssertions;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace UnitTestProject1
{
[TestClass]
public class NumericExtensionsTests
{
[TestMethod]
public void IsInRange_ValueIsWithinRange()
{
((int)5).EnsureIsInRange(5, 10).Should().BeTrue();
((int)8).EnsureIsInRange(5, 10).Should().BeTrue();
((int)10).EnsureIsInRange(5, 10).Should().BeTrue();
}

[TestMethod]
public void IsInRange_ValueIsNotWithinRange()
{
Action action = () => ((int)4).EnsureIsInRange(5, 10);
action.ShouldThrow<ArgumentOutOfRangeException>()
.Which.ParamName.Should().Be("value");
action = () => ((int)11).EnsureIsInRange(5, 10);
action.ShouldThrow<ArgumentOutOfRangeException>()
.Which.ParamName.Should().Be("value");
}

[TestMethod]
[DataRow(5, 5, 10)]
[DataRow(8, 5, 10)]
[DataRow(10, 5, 10)]
public void IsInRange_ValueIsWithinRangeViaDataRow(int testValue, int min, int max)
{
testValue.EnsureIsInRange(min, max).Should().BeTrue();
}

[TestMethod]
[DataRow(4, 5, 10)]
[DataRow(11, 5, 10)]
public void IsInRange_ValueIsNotWithinRangeViaDataRow(int testValue, int min, int max)
{
Action action = () => testValue.EnsureIsInRange(min, max);
action.ShouldThrow<ArgumentOutOfRangeException>()
.Which.ParamName.Should().Be("value");
}
}
}
Remco
#2 Posted : Thursday, February 9, 2017 10:10:43 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for sharing this issue.

From my own experiments with the DataRowAttributes, it appeared to me that these would only work if you specified a [DataTestMethod] attribute instead of [TestMethod].

After comparing your code between VSTest and NCrunch, it looks like MSTest can actually use either of these. I'm now not sure why DataTestMethod actually exists. I've noted this down to be adjusted in NCrunch.

For now, if you're using the DataRowAttribute, make sure you use DataTestMethod instead of TestMethod.
steelerc
#3 Posted : Friday, April 28, 2017 12:35:27 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/28/2017(UTC)
Posts: 2
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
I am seeing the same behavior running with DataTestMethod and DataRow tests in VS2017. I have the following decoration
[DataTestMethod]
[DataRow(true, DisplayName = "Internal User")]
[DataRow(false, DisplayName = "External User")]
public void GetTransactionGetsReadOnlyTransactionWhenCommandReferredAndNotReadOnly(bool isInternalUser)...

and the error I'm seeing is

ENGINE - [08:23:54.9859-Core-31] Event [ErrorEvent:[Error:
Context: (global)
Category: Internal
Message: System.Exception: Unknown parameter data type: System.Boolean
at nCrunch.Module.MSTest.Integration.MSTestResolvedTest.convertStringToType(String stringValue, Type type)
at nCrunch.Module.MSTest.Integration.MSTestResolvedTest.DecodeAndResolve(ReflectedAssembly assemblyContainingTest, String encodedTest)
at nCrunch.Module.MSTest.Integration.MSTestTest..ctor(MSTestNames names, ReflectedAssembly assembly, ReflectedType fixtureType, PendingTest pendingTest, MSTestInterceptingContext testContext, Boolean considerInconclusiveTestsAsPassing)
at nCrunch.Module.MSTest.Integration.MSTestFixture.<>c__DisplayClass2_0.<.ctor>b__0(PendingTest t)
at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at nCrunch.Module.MSTest.Integration.MSTestFixture..ctor(MSTestNames names, ReflectedAssembly assembly, TestName fixtureTest, IList`1 tests, MSTestInterceptingContext testContext, Boolean considerInconclusiveTestsAsPassing)
at nCrunch.Module.MSTest.Integration.MSTestRunner.runTestsWithinAssembly(IList`1 testsToRun, TestTaskOutput testOutput)
at nCrunch.Module.MSTest.Integration.MSTestRunner.<>c__DisplayClass7_1.<RunTests>b__1()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackUnreliableActivity(String name, Action activity)
at nCrunch.Common.PerformanceTracking.PerfTracker.TryTrackUnreliableActivity(String name, Action activity)
at nCrunch.Module.MSTest.Integration.MSTestRunner.RunTests(IList`1 testsToRun, TestTaskOutput testOutput)
at nCrunch.Module.MSTest.Integration.MSTestFrameworkRuntimeEnvironment.<>c__DisplayClass11_0.<RunTests>b__3()
at nCrunch.TestExecution.TestExecutionMonitor.PerformMonitoredTestExecution(Action testExecutionAction)
at nCrunch.Module.MSTest.Integration.MSTestFrameworkRuntimeEnvironment.RunTests(TestTaskOutput output, TestExecutionMapSet testMapSet, TestExecutionParameters parameters, IIpcMessageProcessor messageProcessor)
at nCrunch.TestExecution.TestRunnerThread.()
Details:
]] is being processed on Core thread with subscriber: ErrorHandler.
ENGINE - [08:23:54.9859-LocalTestExecutionTask-34] Task processing complete for [LocalTestExecutionTask: [SnapshotComponent: QuotingService.Tests, 3, 6064464], 1 test(s), MSTest, BeingProcessed, (local), cf6d3d00-7fee-446a-b8b6-8644160bb68f], processing time: 00:00:00
Remco
#4 Posted : Friday, April 28, 2017 1:17:01 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi,

Thanks for sharing this problem.

This looks to be a problem related to the boolean parameter. Whenever I create a DataRow test using a boolean, I see the same exception.

I'll see if I can get a fix included for this in the 3.7 release out next week. Thanks again for bringing it to my attention.
steelerc
#5 Posted : Friday, April 28, 2017 2:20:18 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/28/2017(UTC)
Posts: 2
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
Thanks for the quick response. Unfortunately it seems to be more than just booleans, hope this helps clarify a simple underlying issue:

[DataRow(LineOfBusiness.MmaCpl, UpdateAttributeType.TransportationPLPriorConditionsDate, null, "NullDate", "TransportationPLPriorConditionsDate", DisplayName = "MMACPL /TransportationPLPriorConditionsDate/ Null")]
public void TestUpdateField_PassingValue_UpdatesCorrectObject(LineOfBusiness lineOfBusiness, UpdateAttributeType updateAttributeType, object value, string testType, string testObject)


ENGINE - Process 10076: 10:16:33.3136 [7] - System.Exception: Unknown parameter data type: System.Object
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestResolvedTest.convertStringToType(String stringValue, Type type)
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestResolvedTest.DecodeAndResolve(ReflectedAssembly assemblyContainingTest, String encodedTest)
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestTest..ctor(MSTestNames names, ReflectedAssembly assembly, ReflectedType fixtureType, PendingTest pendingTest, MSTestInterceptingContext testContext, Boolean considerInconclusiveTestsAsPassing)
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestFixture.<>c__DisplayClass2_0.<.ctor>b__0(PendingTest t)
ENGINE - Process 10076: at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
ENGINE - Process 10076: at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestFixture..ctor(MSTestNames names, ReflectedAssembly assembly, TestName fixtureTest, IList`1 tests, MSTestInterceptingContext testContext, Boolean considerInconclusiveTestsAsPassing)
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestRunner.runTestsWithinAssembly(IList`1 testsToRun, TestTaskOutput testOutput)
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestRunner.<>c__DisplayClass7_1.<RunTests>b__1()
ENGINE - Process 10076: at nCrunch.Common.PerformanceTracking.PerfTracker.TrackUnreliableActivity(String name, Action activity)
ENGINE - Process 10076: at nCrunch.Common.PerformanceTracking.PerfTracker.TryTrackUnreliableActivity(String name, Action activity)
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestRunner.RunTests(IList`1 testsToRun, TestTaskOutput testOutput)
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestFrameworkRuntimeEnvironment.<>c__DisplayClass11_0.<RunTests>b__3()
ENGINE - Process 10076: at nCrunch.TestExecution.TestExecutionMonitor.PerformMonitoredTestExecution(Action testExecutionAction)
ENGINE - Process 10076: at nCrunch.Module.MSTest.Integration.MSTestFrameworkRuntimeEnvironment.RunTests(TestTaskOutput output, TestExecutionMapSet testMapSet, TestExecutionParameters parameters, IIpcMessageProcessor messageProcessor)
ENGINE - Process 10076: at nCrunch.TestExecution.TestRunnerThread.()
ENGINE - Process 10076:
ENGINE - [10:16:33.3406-LocalTestExecutionTask-28] ERROR (Internal): System.Exception: Unknown parameter data type: System.Object
at nCrunch.Module.MSTest.Integration.MSTestResolvedTest.convertStringToType(String stringValue, Type type)
at nCrunch.Module.MSTest.Integration.MSTestResolvedTest.DecodeAndResolve(ReflectedAssembly assemblyContainingTest, String encodedTest)
at nCrunch.Module.MSTest.Integration.MSTestTest..ctor(MSTestNames names, ReflectedAssembly assembly, ReflectedType fixtureType, PendingTest pendingTest, MSTestInterceptingContext testContext, Boolean considerInconclusiveTestsAsPassing)
at nCrunch.Module.MSTest.Integration.MSTestFixture.<>c__DisplayClass2_0.<.ctor>b__0(PendingTest t)
at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at nCrunch.Module.MSTest.Integration.MSTestFixture..ctor(MSTestNames names, ReflectedAssembly assembly, TestName fixtureTest, IList`1 tests, MSTestInterceptingContext testContext, Boolean considerInconclusiveTestsAsPassing)
at nCrunch.Module.MSTest.Integration.MSTestRunner.runTestsWithinAssembly(IList`1 testsToRun, TestTaskOutput testOutput)
at nCrunch.Module.MSTest.Integration.MSTestRunner.<>c__DisplayClass7_1.<RunTests>b__1()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackUnreliableActivity(String name, Action activity)
at nCrunch.Common.PerformanceTracking.PerfTracker.TryTrackUnreliableActivity(String name, Action activity)
at nCrunch.Module.MSTest.Integration.MSTestRunner.RunTests(IList`1 testsToRun, TestTaskOutput testOutput)
at nCrunch.Module.MSTest.Integration.MSTestFrameworkRuntimeEnvironment.<>c__DisplayClass11_0.<RunTests>b__3()
at nCrunch.TestExecution.TestExecutionMonitor.PerformMonitoredTestExecution(Action testExecutionAction)
at nCrunch.Module.MSTest.Integration.MSTestFrameworkRuntimeEnvironment.RunTests(TestTaskOutput output, TestExecutionMapSet testMapSet, TestExecutionParameters parameters, IIpcMessageProcessor messageProcessor)
at nCrunch.TestExecution.TestRunnerThread.()
1 user thanked steelerc for this useful post.
Remco on 4/28/2017(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.073 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download