I'm having two issues, which may be related, so I'll report them together:
Issue 1: Tests fail in background mode, pass in debug/TestDriven.net.
Issue 2: When stepping through a failed test and it passes, icons are left in funny state.

The original issue I'm investigating is that NCrunch is reporting a unit tests as failing, but if I run the test via TestDriven.Net, or if I use NCrunch to debug into the test, it passes. If I "Run/Prioritize covering tests", it fails. The test is pretty straightforward, looking something like this:

[Test]
public void AllSettingsOff()
{
var args = new[] { "" };
var mySettingsObject = new MySettings(args, Keys.Shift);
Assert.IsFalse(mySettingsObject.BoolProperty1);
Assert.IsFalse(mySettingsObject.BoolProperty2);
Assert.IsFalse(mySettingsObject.BoolProperty3);
Assert.IsFalse(mySettingsObject.BoolProperty4);
Assert.IsFalse(mySettingsObject.BoolProperty5); //This is the one failing

Assert.AreEqual(mySettingsObject.ReferenceProperty, null);
Assert.AreEqual(mySettingsObject.StringProperty, null);
}



Test was showing as failing (red dots up until a failing assert call, where a red X was shown, followed by black dots for each subsequent line.


I used the "Break into first covering test at this line" option to debug to the line right before the exception. At that point, I stepped through each line (using F10) until the test completed. That time through, it passed. However, the NCrunch dots updated into a funny state. The red dots turned green, the red X turned into a *green X* and the block dots stayed black (instead of turning green).



I'm running NCrunch v1.37.0.46b, Visual Studio 2010 SP1, and using Mbunit v3.3.

I do have a few assemblies in the solution where "Copy Referenced Assemblies to Workspace" is turned on, but not for the assembly containing this test or for the class being tested.

Hi Remco,

I just ran into this issue today, and after discovering that the test failed only after other tests had run in the same process, I followed the following path to figure out what was going on.

1. Wrap the offending line in a try/catch.
2. Inside the catch handler, throw again.
3. Place a breakpoint on the re-throwing line.
4. Use another unit testing tool to run the problem set of tests in one process, under the debugger.
5. Fix the problem after seeing the proper diagnostic information.

Hopefully, you could implement another menu choice similar to "Break into the first failing covering test at this line". Instead it would be "Break into the first failure at this line." Presumably, you're doing some work in NCrunch to dynamically modify code in order to deal handle testing while the developer is typing. So, you could quickly modify the code by wrapping the selected line in an exception, then run the normal set of tests in the same process and only break when you hit the exception. It may not be quite this simple, since the debugger would be showing code that the user didn't write -- but NCrunch is amazing enough that I'm sure you could pull off something useful here. :o)

Thanks,
John