MSTest test assemblies using DynamicDataAttribute cannot be built/analyzed

We use MSTest for our unit tests and are unable to use NCrunch with some test assemblies, which use the DynamicDataAttribute to generate multiple test cases like this:

[TestClass]
public class ExampleTests
{
    public static IEnumerable<object[]> MyTestData
    {
        get
        {
            yield return new object[] { 1 , "one"};
            yield return new object[] { 2 , "two"};
            yield return new object[] { 3 , "three"};
        }
    }

    [DataTestMethod]
    [DynamicData(nameof(MyTestData))]
    public void ExampleTest(int number, string word)
    {
        // ...
    }
}

However, test assemblies using the DynamicDataAttribute can't be built/analyzed with the following error:

An error occurred while analysing this project after it was built: System.InvalidOperationException: Sequence contains no matching element
   at System.Linq.ThrowHelper.ThrowNoMatchException()
   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source, Func`2 predicate)
   at nCrunch.Module.MSTest.Integration.MSTestDynamicDiscoverer.extractDynamicDataSourceFromMethodAttribute(ReflectedType fixtureType, ReflectedAttribute reflectedAttribute)
   at nCrunch.Module.MSTest.Integration.MSTestDynamicDiscoverer.<>c__DisplayClass5_0.<FindFrameworkTestsInAssembly>b__1()
   at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
   at nCrunch.Common.PerformanceTracking.PerfTracker.TryTrackActivity(String name, Action activity)
   at nCrunch.Module.MSTest.Integration.MSTestDynamicDiscoverer.FindFrameworkTestsInAssembly(ReflectedAssembly assembly, FilePath assemblyFilePath, IList`1 referencedAssemblyFilePaths, ComponentUniqueName testComponentUniqueName, PlatformType platformType, DynamicProxy[] dynamicProxies)
   at nCrunch.TestExecution.TestFinder.<>c__DisplayClass0_2.<FindTestsForFrameworks>b__1()
   at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
   at nCrunch.TestExecution.TestFinder.<>c__DisplayClass0_0.<FindTestsForFrameworks>b__0()
   at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
   at nCrunch.TestExecution.TestFinder.FindTestsForFrameworks(ReflectedAssembly assembly, FilePath assemblyFilePath, IList`1 referencedAssemblyFilePaths, DescribedTestFrameworkDiscoverer[] describedDiscoverers, ComponentUniqueName testComponentUniqueName, PlatformType platformType, DynamicProxy[] dynamicProxies)
   at nCrunch.TestExecution.RemoteTaskRunner.AnalyseAssembly(DescribedTestFrameworkDiscoverer[] applicableFrameworks, ComponentUniqueName testComponentUniqueName, PerfTracker perfTracker, TaskLogId taskLogId)

I suspect is due to a recent change in the MSTest framework. They changed the namings of the internal fields (they added underscores, so dynamicDataSourceName became _dynamicDataSourceName etc.), but NCrunch appears to still be looking for the old naming. (for the changes see https://github.com/microsoft/testfx/commit/aba70698c87c16c29e1a7c2eab643c3fbb9532b9, src/TestFramework/MSTest.Core/Attributes/DataSource/DynamicDataAttribute.cs)

Downgrading the packages MSTest.TestFramework and MSTest.TestAdapter to version 2.3.0-preview-20220810-02 seems to confirm this, as tests with DynamicTestAttribute work again. That build is from 8/10/2022, so before the naming change on 8/30/2022. Upgrading the packages to version 3.0.0-preview-20221110-04, released on 11/11/2022 breaks the NCrunch build again, just like the most recent version.

We're using NCrunch 4.17.0.7 with Visual Studio 2022 17.6.

Hi, thanks for sharing this issue.

I believe your assessment is correct. They changed the internal field names and this has broken our integration.

Hopefully, this should be simple to fix. I've noted this down to be addressed. For the time being, I recommend using an earlier version of MSTest until we can get this fixed.

Hi,

Thanks for the quick response. I'm looking forward to the fix!

Best regards

Remco wrote:Hi, thanks for sharing this issue.

I believe your assessment is correct. They changed the internal field names and this has broken our integration.

Hopefully, this should be simple to fix. I've noted this down to be addressed. For the time being, I recommend using an earlier version of MSTest until we can get this fixed.

@Remco I think we have a quite similar problem (I just submited a bug report via VS/NCrunch), but we are using MSTest.TestAdapter 2.2.10, it was working with NCrunch 4.16.04, unfortunately I can't find the bug I've created via VS/NCrunch to relate it to this issue. --> never mind NCrunch_VS2022_4.18.0.2.msi fixes the issue.

br