Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Test runner process crashes
Daniel Rose
#1 Posted : Thursday, March 6, 2014 3:11:20 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/7/2011(UTC)
Posts: 29
Location: Aachen, Germany

Thanks: 6 times
Was thanked: 5 time(s) in 4 post(s)
I have some integration tests where I create a View, which creates auto-injected ViewModels, etc. Depending on the system load, these tests occasionally run into the test timeout.

However, when this happens, in most cases the test runner process itself (nCrunch.TaskRunner40.x86.exe) crashes. I've attached a debugger to a crashing process, and it is due to an unhandled ThreadAbortException.

Is this fixable? Do you need more information?
Remco
#2 Posted : Thursday, March 6, 2014 10:10:33 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,986

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi Daniel,

Thanks for sharing this issue.

This is most likely occurring because NCrunch enforces a default test timeout of 60 seconds whenever no existing timeout has been specified. This is because timeouts are very important when running tests continuously, as otherwise the engine can hang. See http://www.ncrunch.net/documentation/concepts_test-timeouts for more details on how NCrunch uses timeouts.

As the test likely is running longer than 60 seconds, I'd recommend labeling it with a sensible Timeout with a much higher value .. something that will always be more than it needs to finish a normal run.
Daniel Rose
#3 Posted : Friday, March 7, 2014 7:12:03 AM(UTC)
Rank: Member

Groups: Registered
Joined: 10/7/2011(UTC)
Posts: 29
Location: Aachen, Germany

Thanks: 6 times
Was thanked: 5 time(s) in 4 post(s)
Hi Remco,

I understand the concept of test timeouts and it is very useful. It is just that when a timeout occurs, it shouldn't crash the entire process.
Remco
#4 Posted : Friday, March 7, 2014 7:17:01 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,986

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
No - you're absolutely right. It shouldn't. Sorry, I misunderstood.

This isn't a known problem, so I might need some more information to try and figure out what is happening. NCrunch should only execute ThreadAbortExceptions inside try/catch blocks.

Which test framework are you using? Is there any chance you can get me a code sample that reproduces the issue?
Daniel Rose
#5 Posted : Friday, March 7, 2014 8:44:12 AM(UTC)
Rank: Member

Groups: Registered
Joined: 10/7/2011(UTC)
Posts: 29
Location: Aachen, Germany

Thanks: 6 times
Was thanked: 5 time(s) in 4 post(s)
I am using NUnit. I created a sample app:
SkyDrive

You'll find a StackTrace.txt with a stack trace of a crash. It was relatively difficult to reproduce, since it apparently is very timing-sensitive. I set the timeout (via the project configuration) to be somewhere where it just doesn't complete. Then I kept on changing the values and/or re-executing the test until it crashed.
Remco
#6 Posted : Friday, March 7, 2014 11:16:58 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,986

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi Daniel -

Thanks for the sample solution. The first time I opened it, the task runner immediately crashed. After some tinkering, it stopped crashing and NCrunch/NUnit started handling the thread abort normally, reporting a sensible error. I since haven't been able to get the problem to occur again, but placing this observation together with the information you've given me, I think I can draw a conclusion about why it is happening.

ThreadAbortException is not known for its reliability. Generally, it is considered 'evil' to make use of it, as there is no telling what kind of code is being executed at the point of the abort. This could be code that is managed or unmanaged, critical or otherwise. In the vast majority of cases, using this exception will result in the correct handling where either NCrunch or NUnit can step in to prevent the task runner from being torn down. In this specific case, it looks as though the code being executed (Wpf Window Show) sometimes steps into situations where the abort cannot be properly handled and the process is immediately torn down with an irritating error box.

The only alternative approach I can think of to stop a test from executing is to arbitrarily terminate the task runner during execution. This will prevent the error from showing, but it is a major step backward in that it is impossible to retrieve data from a terminated process, and any user error handling within the test code will not have a chance to fire. There have been some remote plans to introduce this approach to control overrunning tests, but only when the existing Thread.Abort method fails and the process refuses to return to a useful state. In your error situation, there is no way to detect the unclean failure of Thread.Abort before the crash error box is shown.

So I'm sorry, but I'm afraid this doesn't look like a problem I can fix. We are limited here by the constraints of the operating system and the unknowns in the code being executed. All I can suggest is to try and make sure your timeouts are set to sensible values so that the tests can execute normally.
1 user thanked Remco for this useful post.
Daniel Rose on 3/7/2014(UTC)
Daniel Rose
#7 Posted : Friday, March 7, 2014 12:20:06 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/7/2011(UTC)
Posts: 29
Location: Aachen, Germany

Thanks: 6 times
Was thanked: 5 time(s) in 4 post(s)
Too bad. Thanks for trying!
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.042 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download