Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

DataRow issue with MSTest 3
jschreuder
#1 Posted : Wednesday, December 21, 2022 1:03:37 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 10/5/2015(UTC)
Posts: 42
Location: Australia

Thanks: 14 times
Was thanked: 23 time(s) in 11 post(s)
Tried updating to the latest MSTest 3 package and ran into a build issue.

With [DataRow] tests that use params for a list of values passed into a test, eg.

Code:

[DataRow("A")]
[DataRow("A", "B")]
[DataRow("A", "B", "C")]
[DataRow("A", "B", "C", "D")]
[DataRow("A", "B", "C", "D", "E")]
[DataRow("A", "B", "C", "D", "E", "F")]
[DataTestMethod]
public void DataRowTest(params string[] letters)
{
     Assert.AreEqual(letters, letters);
}


NCrunch can no longer build the project. Error is as follows:
Quote:

NCrunch has encountered an internal error: System.Exception: System.InvalidCastException: Unable to cast object of type 'System.String' to type 'System.Object[]'.
at nCrunch.Module.MSTest.Integration.MSTestCilResolvedParameters..ctor(CilCustomAttribute dataRowAttribute)
at nCrunch.Module.MSTest.Integration.MSTestCilDiscoverer.createTestCases(CilMethodDefinition method, CilTypeDefinition fixture)
at nCrunch.Module.MSTest.Integration.MSTestCilDiscoverer.<FindFrameworkTestsInAssembly>b__12_0()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Module.MSTest.Integration.MSTestCilDiscoverer.FindFrameworkTestsInAssembly(ReflectedAssembly assembly, FilePath assemblyFilePath, IList`1 referencedAssemblyFilePaths, ComponentUniqueName testComponentUniqueName, PlatformType platformType, DynamicProxy[] dynamicProxies)
at nCrunch.TestExecution.TestFinder..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.TestExecution.TestFinder..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.TestExecution.TestFinder.FindTestsForFrameworks(ReflectedAssembly assembly, FilePath assemblyFilePath, IList`1 referencedAssemblyFilePaths, DescribedTestFrameworkDiscoverer[] describedDiscoverers, ComponentUniqueName testComponentUniqueName, PlatformType platformType, DynamicProxy[] dynamicProxies)
at nCrunch.Compiler.StaticManipulation.BuiltCilAssembly.DiscoverTests(TestFrameworkDescription[] applicableTestFrameworks, ComponentUniqueName testComponentUniqueName)
at nCrunch.Compiler.CilProcessingTasks.CilTestDiscoveryTask.ProcessTask(IBuiltAssembly builtAssembly, ComponentInstrumentationParameters instrumentationParameters, BuildOutput output)
at nCrunch.Compiler.CilProcessingTasks.ConcurrentCilTaskProcessor.()
at nCrunch.Compiler.CilProcessingTasks.ConcurrentCilTaskProcessor.ProcessTasks(Int32 backgroundThreadsAllowed)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentInstrumentationParameters , BuildOutput , IBackgroundTaskProcessor )
at nCrunch.Compiler.RemoteBuildRunner.PerformPostProcessingOfBuiltAssembly(ComponentInstrumentationParameters instrumentationParameters, BuildOutput output, IBackgroundTaskProcessor backgroundTaskProcessor)
at nCrunch.Core.BuildManagement.BuildEnvironment..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.BuildManagement.BuildEnvironment.Build(SnapshotComponent snapshotComponentToBuild, IList`1 referencedComponents, GridClientId gridClientId, IList`1 customEnvironmentVariables, IPlatformBuildExtender extender, Guid taskId, GridAddress clientAddress, Boolean extractCoverageReportStructure)


The same code builds and works fine in MSTest 2.2. My best guess is this is something related to this breaking change.
https://github.com/microsoft/testfx/pull/1332

Sidenote - there is a variant that NCrunch will build with MSTest 3, but the test cannot run under NCrunch (though it runs via VS test runner).

Code:

[DataRow(new string[] { "A" })]
[DataRow(new string[] { "A", "B" })]
[DataRow(new string[] { "A", "B", "C" })]
[DataRow(new string[] { "A", "B", "C", "D" })]
[DataRow(new string[] { "A", "B", "C", "D", "E" })]
[DataRow(new string[] { "A", "B", "C", "D", "E", "F" })]
[DataTestMethod]
public void DataRowTestArray(string[] letters)
{
    Assert.AreEqual(letters, letters);
}


This results in a failing test with message:
Quote:
Unable to transfer an Array from a statically analysed domain into a dynamic runtime domain. Please change your 'Framework utilisation type for MSTest' setting to 'UseDynamicAnalysis' to run this test.


Here's a repro solution with both of the above cases:
https://1drv.ms/u/s!ArNW31rplqplm0WV52MqdOpMOrxA?e=BLQkmw
Remco
#2 Posted : Wednesday, December 21, 2022 1:19:55 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for sharing this issue. I believe your analysis is correct; they've changed the constructor signatures for this attribute and it's broken our static test discovery.

The workaround for this problem is to set your Framework utilisation type for MSTest to 'UseDynamicAnalysis'. This will force NCrunch to use runtime reflection to discover the tests (as opposed to statically analysing the assembly), which will allow the project to build and the tests to be discovered.

I'll include a proper fix for this problem in the next release of NCrunch.
1 user thanked Remco for this useful post.
jschreuder on 12/21/2022(UTC)
jschreuder
#3 Posted : Monday, May 29, 2023 12:54:03 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 10/5/2015(UTC)
Posts: 42
Location: Australia

Thanks: 14 times
Was thanked: 23 time(s) in 11 post(s)
Looks like another change here broke your test discovery (v3.0.3)

Quote:
Unable to transfer an Array from a statically analysed domain into a dynamic runtime domain. Please change your 'Framework utilisation type for MSTest' setting to 'UseDynamicAnalysis' to run this test.


v3.0.2 has been working fine since your last fix, guessing it's this change based on the description
https://github.com/Microsoft/testfx/pull/1646
Remco
#4 Posted : Monday, May 29, 2023 8:27:11 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
jschreuder;16665 wrote:
Looks like another change here broke your test discovery (v3.0.3)


Thanks for the heads up on this. I'll check it out. Dynamic analysis should hopefully be a workaround for the time being.
1 user thanked Remco for this useful post.
jschreuder on 5/30/2023(UTC)
nCubed
#5 Posted : Monday, July 3, 2023 6:26:41 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/1/2012(UTC)
Posts: 22
Location: United States of America

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
Remco;16666 wrote:
Thanks for the heads up on this. I'll check it out. Dynamic analysis should hopefully be a workaround for the time being.

Issue persists with tests using DataRow attributes, even when updating the test project to use Dynamic Analysis.

Win 10 x64
VS 2022 x64 v16.6.4
NCrunch v4.17.0.7
Microsoft.NET.Test.Sdk v17.6.3
MSTest.TestAdapter" v3.0.4
MSTest.TestFramework v3.0.4

Edit: Converting everything from DataRow attributes to DynamicData and reverting back to UseStaticAnalysis resolves the issue. Would be nice to use DataAttributes when there's only a couple of rows of data.
Remco
#6 Posted : Thursday, July 6, 2023 4:27:00 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
nCubed
#7 Posted : Thursday, July 6, 2023 1:28:51 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/1/2012(UTC)
Posts: 22
Location: United States of America

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
Remco;16725 wrote:
Would you be interested in trying the build below to see if this solves the problem for you?

NCrunch_Console_4.18.0.1.msi


Installed the 4.18.0.1 MSI - same issue.

Quote:
Unable to transfer an Array from a statically analysed domain into a dynamic runtime domain. Please change your 'Framework utilisation type for MSTest' setting to 'UseDynamicAnalysis' to run this test.


After updating to UseDynamicAnalysis, and rebuilding w/ NCruch, no change as before. The test is about as basic as it gets:

Code:

[DataRow(true)]
[DataRow(false)]
[TestMethod]
public void SomeTest(bool val)
{
// code
}
Remco
#8 Posted : Friday, July 7, 2023 6:17:48 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
1 user thanked Remco for this useful post.
jschreuder on 7/9/2023(UTC)
nCubed
#9 Posted : Friday, July 7, 2023 1:08:47 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/1/2012(UTC)
Posts: 22
Location: United States of America

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
Remco;16737 wrote:
Sorry for the trouble. Could you try the build below and let me know how this goes?

NCrunch_VS2022_4.18.0.2.msi


Looks like I installed the console installer last time. Sorry! Just tried w/ the VS 2022 installer and DataRow works again. Thanks!
1 user thanked nCubed for this useful post.
Remco on 7/7/2023(UTC)
jschreuder
#10 Posted : Sunday, July 9, 2023 4:59:55 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 10/5/2015(UTC)
Posts: 42
Location: Australia

Thanks: 14 times
Was thanked: 23 time(s) in 11 post(s)
Also working for me with 4.18.0.2 and MSTest 3.0.4 without needing to switch to dynamic runtime domain. Thanks Remco!
1 user thanked jschreuder for this useful post.
Remco on 7/9/2023(UTC)
gnewtzie
#11 Posted : Thursday, November 16, 2023 3:06:32 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/10/2018(UTC)
Posts: 1
Location: United States of America

This seems to be broke on MSTest 3.1.1 again.
Remco
#12 Posted : Thursday, November 16, 2023 10:45:05 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
gnewtzie;16932 wrote:
This seems to be broke on MSTest 3.1.1 again.


Thanks for sharing this.

Can you share a code sample of the breakage? I can't seem to get this to fail using the simple example above.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.116 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download