I am using NCrunch 3.7.0.7 and have run into a problem where running a single test takes a long time because all TestCaseSources are executed before running the test. This happens even when NCrunch is idle, which indicates to me that it has already built the solution and enumerated all the tests.
Steps to reproduce:
1. Create a .NET Framework class library with a NuGet reference to NUnit and put the following code into the Class1.cs file that is created.
Code:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Runtime.CompilerServices;
using NUnit.Framework;
[TestFixture]
public class TheTestFixture
{
[TestCaseSource(nameof(SomeTestsToRun))]
public void RunMe()
{
}
static IEnumerable<TestCaseData> SomeTestsToRun()
{
yield return new TestCaseData().SetName("A test");
CommonStuff.AppendProcessId();
}
}
[TestFixture]
public class AnotherUnrelatedTestFixture
{
[TestCaseSource(nameof(SomeTestsNotToRun))]
public void DoNotRunMe()
{
}
static IEnumerable<TestCaseData> SomeTestsNotToRun()
{
yield return new TestCaseData().SetName("A test");
CommonStuff.AppendProcessId();
}
}
public static class CommonStuff
{
public static void AppendProcessId([CallerMemberName]string caller = null)
{
var filePath = Path.Combine(Path.GetTempPath(), "TestRunFiles.txt");
File.AppendAllText(filePath, $"{Process.GetCurrentProcess().Id}: {caller}{Environment.NewLine}");
}
}
2. Enable NCrunch for the project. Both tests should pass (because there are no assertions). Make sure that NCrunch is idle.
3. Run the following PowerShell command:
Code:
gc $env:temp\TestRunFiles.txt -Wait -Tail 0
4. Run the `RunMe` test.
Expected behaviour: Nothing will appear in the PowerShell window because the tests have already been enumerated by NCrunch and there's no need to do it again.
Actual behaviour:Both test case sources are re-run, so something like the following appears in the PowerShell window:
Code:
12345: SomeTestsToRun
12345: SomeTestsNotToRun
