Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

MSTest test assemblies using DynamicDataAttribute cannot be built/analyzed
baokr
#1 Posted : Wednesday, June 28, 2023 2:02:46 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/28/2023(UTC)
Posts: 3
Location: Germany

We use MSTest for our unit tests and are unable to use NCrunch with some test assemblies, which use the DynamicDataAttribute to generate multiple test cases like this:

Code:

[TestClass]
public class ExampleTests
{
    public static IEnumerable<object[]> MyTestData
    {
        get
        {
            yield return new object[] { 1 , "one"};
            yield return new object[] { 2 , "two"};
            yield return new object[] { 3 , "three"};
        }
    }

    [DataTestMethod]
    [DynamicData(nameof(MyTestData))]
    public void ExampleTest(int number, string word)
    {
        // ...
    }
}


However, test assemblies using the DynamicDataAttribute can't be built/analyzed with the following error:

Code:

An error occurred while analysing this project after it was built: System.InvalidOperationException: Sequence contains no matching element
   at System.Linq.ThrowHelper.ThrowNoMatchException()
   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
   at nCrunch.Module.MSTest.Integration.MSTestDynamicDiscoverer.extractDynamicDataSourceFromMethodAttribute(ReflectedType fixtureType, ReflectedAttribute reflectedAttribute)
   at nCrunch.Module.MSTest.Integration.MSTestDynamicDiscoverer.<>c__DisplayClass5_0.<FindFrameworkTestsInAssembly>b__1()
   at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
   at nCrunch.Common.PerformanceTracking.PerfTracker.TryTrackActivity(String name, Action activity)
   at nCrunch.Module.MSTest.Integration.MSTestDynamicDiscoverer.FindFrameworkTestsInAssembly(ReflectedAssembly assembly, FilePath assemblyFilePath, IList`1 referencedAssemblyFilePaths, ComponentUniqueName testComponentUniqueName, PlatformType platformType, DynamicProxy[] dynamicProxies)
   at nCrunch.TestExecution.TestFinder.<>c__DisplayClass0_2.<FindTestsForFrameworks>b__1()
   at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
   at nCrunch.TestExecution.TestFinder.<>c__DisplayClass0_0.<FindTestsForFrameworks>b__0()
   at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
   at nCrunch.TestExecution.TestFinder.FindTestsForFrameworks(ReflectedAssembly assembly, FilePath assemblyFilePath, IList`1 referencedAssemblyFilePaths, DescribedTestFrameworkDiscoverer[] describedDiscoverers, ComponentUniqueName testComponentUniqueName, PlatformType platformType, DynamicProxy[] dynamicProxies)
   at nCrunch.TestExecution.RemoteTaskRunner.AnalyseAssembly(DescribedTestFrameworkDiscoverer[] applicableFrameworks, ComponentUniqueName testComponentUniqueName, PerfTracker perfTracker, TaskLogId taskLogId)


I suspect is due to a recent change in the MSTest framework. They changed the namings of the internal fields (they added underscores, so dynamicDataSourceName became _dynamicDataSourceName etc.), but NCrunch appears to still be looking for the old naming. (for the changes see https://github.com/micro...1a7c2eab643c3fbb9532b9, src/TestFramework/MSTest.Core/Attributes/DataSource/DynamicDataAttribute.cs)

Downgrading the packages MSTest.TestFramework and MSTest.TestAdapter to version 2.3.0-preview-20220810-02 seems to confirm this, as tests with DynamicTestAttribute work again. That build is from 8/10/2022, so before the naming change on 8/30/2022. Upgrading the packages to version 3.0.0-preview-20221110-04, released on 11/11/2022 breaks the NCrunch build again, just like the most recent version.

We're using NCrunch 4.17.0.7 with Visual Studio 2022 17.6.
Remco
#2 Posted : Thursday, June 29, 2023 4:21:48 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,015

Thanks: 937 times
Was thanked: 1262 time(s) in 1174 post(s)
Hi, thanks for sharing this issue.

I believe your assessment is correct. They changed the internal field names and this has broken our integration.

Hopefully, this should be simple to fix. I've noted this down to be addressed. For the time being, I recommend using an earlier version of MSTest until we can get this fixed.
baokr
#4 Posted : Thursday, June 29, 2023 8:46:59 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/28/2023(UTC)
Posts: 3
Location: Germany

Hi,

Thanks for the quick response. I'm looking forward to the fix!

Best regards
UppSol
#3 Posted : Tuesday, July 11, 2023 11:15:25 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/21/2019(UTC)
Posts: 39
Location: Austria

Thanks: 30 times
Was thanked: 7 time(s) in 7 post(s)
Remco;16712 wrote:
Hi, thanks for sharing this issue.

I believe your assessment is correct. They changed the internal field names and this has broken our integration.

Hopefully, this should be simple to fix. I've noted this down to be addressed. For the time being, I recommend using an earlier version of MSTest until we can get this fixed.


@Remco I think we have a quite similar problem (I just submited a bug report via VS/NCrunch), but we are using MSTest.TestAdapter 2.2.10, it was working with NCrunch 4.16.04, unfortunately I can't find the bug I've created via VS/NCrunch to relate it to this issue. --> never mind NCrunch_VS2022_4.18.0.2.msi fixes the issue.

br
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.039 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download