Test runner process crashes

I have some integration tests where I create a View, which creates auto-injected ViewModels, etc. Depending on the system load, these tests occasionally run into the test timeout.

However, when this happens, in most cases the test runner process itself (nCrunch.TaskRunner40.x86.exe) crashes. I've attached a debugger to a crashing process, and it is due to an unhandled ThreadAbortException.

Is this fixable? Do you need more information?

Hi Daniel,

Thanks for sharing this issue.

This is most likely occurring because NCrunch enforces a default test timeout of 60 seconds whenever no existing timeout has been specified. This is because timeouts are very important when running tests continuously, as otherwise the engine can hang. See http://www.ncrunch.net/documentation/concepts_test-timeouts for more details on how NCrunch uses timeouts.

As the test likely is running longer than 60 seconds, I'd recommend labeling it with a sensible Timeout with a much higher value .. something that will always be more than it needs to finish a normal run.

Hi Remco,

I understand the concept of test timeouts and it is very useful. It is just that when a timeout occurs, it shouldn't crash the entire process.

No - you're absolutely right. It shouldn't. Sorry, I misunderstood.

This isn't a known problem, so I might need some more information to try and figure out what is happening. NCrunch should only execute ThreadAbortExceptions inside try/catch blocks.

Which test framework are you using? Is there any chance you can get me a code sample that reproduces the issue?

I am using NUnit. I created a sample app:
SkyDrive

You'll find a StackTrace.txt with a stack trace of a crash. It was relatively difficult to reproduce, since it apparently is very timing-sensitive. I set the timeout (via the project configuration) to be somewhere where it just doesn't complete. Then I kept on changing the values and/or re-executing the test until it crashed.

Hi Daniel -

Thanks for the sample solution. The first time I opened it, the task runner immediately crashed. After some tinkering, it stopped crashing and NCrunch/NUnit started handling the thread abort normally, reporting a sensible error. I since haven't been able to get the problem to occur again, but placing this observation together with the information you've given me, I think I can draw a conclusion about why it is happening.

ThreadAbortException is not known for its reliability. Generally, it is considered 'evil' to make use of it, as there is no telling what kind of code is being executed at the point of the abort. This could be code that is managed or unmanaged, critical or otherwise. In the vast majority of cases, using this exception will result in the correct handling where either NCrunch or NUnit can step in to prevent the task runner from being torn down. In this specific case, it looks as though the code being executed (Wpf Window Show) sometimes steps into situations where the abort cannot be properly handled and the process is immediately torn down with an irritating error box.

The only alternative approach I can think of to stop a test from executing is to arbitrarily terminate the task runner during execution. This will prevent the error from showing, but it is a major step backward in that it is impossible to retrieve data from a terminated process, and any user error handling within the test code will not have a chance to fire. There have been some remote plans to introduce this approach to control overrunning tests, but only when the existing Thread.Abort method fails and the process refuses to return to a useful state. In your error situation, there is no way to detect the unclean failure of Thread.Abort before the crash error box is shown.

So I'm sorry, but I'm afraid this doesn't look like a problem I can fix. We are limited here by the constraints of the operating system and the unknowns in the code being executed. All I can suggest is to try and make sure your timeouts are set to sensible values so that the tests can execute normally.

Too bad. Thanks for trying!