Hi Henrry,

The warning given by NCrunch here gives an accurate description of the problem. When test cases use a user-defined type, there is no way for NCrunch to tell the tests apart, so you end up with a uniqueness problem. The default NUnit runner doesn't encounter a problem here because in its situation each of the tests is held in memory over the duration of the session, so they are uniquely identifiable via their memory address. Because NCrunch retains the test outside the test process itself, it can't use memory addresses for identification.

The solution is to implement .ToString() on your user defined type so that each of the test cases has a unique and descriptive name.

I'd suggest redesigning the test to include the test cases and individual tests within a single [Test]. The user defined type doesn't seem stable enough to be used as a parameter for a test case. Parameters for test cases must be uniquely identifiable and consistent, otherwise NCrunch can't identify the tests or correlate results for them.

Unfortunately, the problem is caused by a technical limitation. Basically, it's caused by the way that test frameworks work. If NCrunch were only ever to do a single unparalleled run through your test project and never tried to persist any test results or code coverage beyond a single test run, then it might be possible to find a way to make such code work. But this would mean removing all the features that make NCrunch worthwhile, so I don't see this ever being supported by NCrunch or any similar test runner.

I expect that test frameworks will eventually move away from allowing code like this to exist. Unique identification of test cases is becoming a big deal right now.

I've just updated the documentation to paint a clearer picture of why this is a problem for NCrunch. It's an interesting coincidence that you posted this support request while I was writing up the documentation page. I guess it shows just how many people are confused by why this is an issue for NCrunch.

I appreciate that. I go through great pain to try to support some crazy things with NCrunch, not because I believe in them, but just to try and make life easier for everyone. I've tried very hard to find solutions to this problem. Either the solutions don't exist, or they are far beyond my ability to conceive or implement :(

Interesting...I have another test that is doing something very similar and it is working fine

[Test]
[TestCaseSource(typeof(JobResolutionStatusTestsSource), nameof(JobResolutionStatusTestsSource.StepForward))]
public ResolutionStatus JobResolutionStatusStepForward(Job job, IUserThresholdService userThresholdService, IDateThresholdService dateThresholdService)
{
...

[TestFixture]
public class JobResolutionStatusStepForwardTests
{
[Test]
[TestCaseSource(typeof(JobResolutionStatusTestsSource), nameof(JobResolutionStatusTestsSource.StepForward))]
[Category("JobResolutionStatus StepForward")]
public ResolutionStatus JobResolutionStatusStepForward(Job job, IUserThresholdService userThresholdService, IDateThresholdService dateThresholdService)
{
var sut = new JobResolutionStatus(userThresholdService, dateThresholdService);

return sut.StepForward(job);
}
}

class JobResolutionStatusTestsSource
{
public static IEnumerable StepForward
{
get
{
yield return new TestCaseData(Imported(), CreateUserThresholdService(true), DateThresholdService(DateTime.Now))
.Returns(ResolutionStatus.DriverCompleted)
.SetDescription("Job should move to DriverCompleted");

//more testcasedata
}
}

private static Job Imported()
{
return JobFactory.New
.With(p => p.ResolutionStatus = ResolutionStatus.Imported)
.Build();
}
}