Array as datarow attribute parameter

Hello,
There is problem in datarow attribute for unit test method. NCrunch can not substitute array parameter into test method.

For example:

[TestMethod]
[DataRow(new string[] { "0" }, false)]
[DataRow(new string[] { "0", "0", "0" }, false)]
[DataRow(new string[] { "0.0001" }, true)]
[DataRow(new string[] { "0", "5" }, true)]
[DataRow(new string[] { "0", "0.0001" }, true)]
[DataRow(new string[] { "0.0001", "0.0001" }, true)]
public void Test1_ExpectSuccess(string[] rateValues, bool expectedResult)
{
...
}

throw an exemption:
Type mismatch on test parameter 1 (System.String[] != System.Object[])

Best regards,
Konstantin

Hi Konstantin,

Do you experience this problem if you set your 'Framework utilisation type for MSTest' to 'DynamicAnalysis'?

DynamicAnalysis provides better support for edge cases like this one.

Thanks a lot, Remco. It works.

This appears to still be "broken" with the default installation. Any chance you could change this behavior to succeed with the Static setting? (So I don't have to tell 19 other developers to change a setting, for every test assembly.) Thanks!

Hi, thanks for posting.

I'm sorry to say that right now that due to feasibility constraints we have no plans to change how this is implemented. Transferring complex types from an unloaded static/CIL domain into a runtime domain is very tricky business and is full of nasty edge cases, many of which are impossible to handle.

In the current version of the software, when you try to run a test of this structure under StaticAnalysis, you'll receive the following error message:

"Unable to transfer an Array from a statically analysed domain into a dynamic runtime domain. Please change your 'Framework utilisation type for MSTest' setting to 'UseDynamicAnalysis' to run this test."

The correct solution from your side is to change the configuration setting. Using NCrunch's configuration system, you can do this at shared solution level so that all developers on the team will automatically have the setting applied for them when they next update from your VCS.

DynamicAnalysis gives much better support for complex test metadata than StaticAnalysis, because we can discover the tests using the same structure as they will be executed. The current implementation is to enable StaticAnalysis by default (for performance reasons), then provide errors and warnings to encourage moving to DynamicAnalysis for codebases that make use of the edge cases that we can't properly handle using static analysis.

Thank you for the detailed response. Didn't know about the shared solution configuration. Will pursue.

I still have a problem with this, even with the DynamicAnalysis setting.

Code example to reproduce the error:
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Tests
{
[TestClass]
public class UnitTest1
{
[DataTestMethod]
[DataRow(1234)]
public void Test(params int[] n)
{
Assert.AreEqual(n[0], 1234);
}
}
}

Works fine in VS and when running 'dotnet test' from command line.
Error message from NCrunch:
"Type mismatch on test parameter 1 (System.Int32[] != System.Int32)"

Software and framework versions:
NCrunch 3.31.0.3
.net core 3.0.100
VS2019 16.3.4
Microsoft.NET.Test.Sdk 16.3.0
MSTest.TestAdapter 2.0.0
MSTest.TestFramework 2.0.0

Thanks for sharing this. I'll take a look and will see if we can get this fixed.

MrAgitato wrote:I still have a problem with this, even with the DynamicAnalysis setting.

Code example to reproduce the error:
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Tests
{
[TestClass]
public class UnitTest1
{
[DataTestMethod]
[DataRow(1234)]
public void Test(params int[] n)
{
Assert.AreEqual(n[0], 1234);
}
}
}

Works fine in VS and when running 'dotnet test' from command line.
Error message from NCrunch:
"Type mismatch on test parameter 1 (System.Int32[] != System.Int32)"

Software and framework versions:
NCrunch 3.31.0.3
.net core 3.0.100
VS2019 16.3.4
Microsoft.NET.Test.Sdk 16.3.0
MSTest.TestAdapter 2.0.0
MSTest.TestFramework 2.0.0

Hi,

I've had a look at this. When I run the sample you provided with dotnet test and the VS test explorer, they fail in the same way that NCrunch does.

The Test Explorer throws: System.ArgumentException: Object of type 'System.Int32' cannot be converted to type 'System.Int32[]'.
dotnet test throws the same exception. To me that makes sense since the parameter in the DataRow is an int.

Yes, but the test accepts params int[], which should accept an int as in the code above (this can easily be verified by calling the method from code instead of using attributes).
I discovered that the version of the test adapter affects this behavoir.

Using the following project file you get the failing test behavior (on command line and in VS test runner) that you have observed:

<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>

<IsPackable>false</IsPackable>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />
<PackageReference Include="MSTest.TestAdapter" Version="1.4.0" />
<PackageReference Include="MSTest.TestFramework" Version="1.4.0" />
</ItemGroup>

</Project>

----------------------------------------------------------------------------------------------------


However, upgrading the mstest references makes the test run on the command line and in VS test runner but not with NCrunch:
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>

<IsPackable>false</IsPackable>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />
<PackageReference Include="MSTest.TestAdapter" Version="2.0.0" />
<PackageReference Include="MSTest.TestFramework" Version="2.0.0" />
</ItemGroup>

</Project>

Thanks for pointing that out! I'll have another look :)

Sorry it took a while to figure this out. This is fixed in the build below. Would you mind giving it a try?

NCrunch_Console_4.2.0.4.msi
NCrunch_Console_4.2.0.4.zip
NCrunch_GridNodeServer_4.2.0.4.msi
NCrunch_GridNodeServer_4.2.0.4.zip
NCrunch_LicenseServer_4.2.0.4.zip
NCrunch_VS2008_4.2.0.4.msi
NCrunch_VS2010_4.2.0.4.msi
NCrunch_VS2010_4.2.0.4.zip
NCrunch_VS2012_4.2.0.4.msi
NCrunch_VS2012_4.2.0.4.zip
NCrunch_VS2013_4.2.0.4.msi
NCrunch_VS2013_4.2.0.4.zip
NCrunch_VS2015_4.2.0.4.msi
NCrunch_VS2015_4.2.0.4.msi.7z
NCrunch_VS2015_4.2.0.4.zip
NCrunch_VS2017_4.2.0.4.msi
NCrunch_VS2017_4.2.0.4.msi.7z
NCrunch_VS2017_4.2.0.4.zip
NCrunch_VS2019_4.2.0.4.msi
NCrunch_VS2019_4.2.0.4.msi.7z
NCrunch_VS2019_4.2.0.4.zip

Works like a charm, thanks!