Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Griod node server crash when restarting all grid nodes
GreenMoose
#1 Posted : Friday, January 30, 2015 9:23:09 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 507

Thanks: 145 times
Was thanked: 66 time(s) in 64 post(s)
Just upgraded local grid node server / vstudio2013 to NCRunch NCrunch_VS2013_2.13.0.1 along with 1 external grid node server (other 3 grid node server remain on 2.10).

Running tests went very slow (the 3 grid nodes not upgraded should already have the "spin up time" done since I did not upgrade those) but I chose to restart all grid nodes via vstudio, then my local grid node service crashed with below stacktrace and had to be started manually. (InvalidOperationException).

Code:

>	System.ServiceProcess.dll!System.ServiceProcess.ServiceController.GetServiceHandle(int desiredAccess)	Unknown
 	System.ServiceProcess.dll!System.ServiceProcess.ServiceController.Start(string[] args)	Unknown
 	nCrunch.GridNode.Recycler.exe!nCrunch.GridNode.Recycler.Program.Main(string[] args)	Unknown


2 other grid nodes (one upgraded, one not) crashed as well according to event viewer (had to start them manually).

Can easily repro it by setting engine mode to run all tests, when nodes are processing I switch back to "Run all tests manually" but before engines settles I restart every grid node one by one.

The Azure nodes never crashes though (I miss a version info column in the "Server Information" for the grid nodes).
Remco
#2 Posted : Monday, February 2, 2015 4:21:40 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Thanks, I'm looking into this and will get back to you soon. Sorry about the wait.
Remco
#3 Posted : Tuesday, February 3, 2015 2:56:22 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Ok - I've managed to reproduce this issue. It looks like the recycler is trying to restart the node too early after termination. Interesting that this problem didn't appear earlier. I'll see about arranging a fixed build. Thanks for reporting this issue.
Remco
#4 Posted : Wednesday, February 4, 2015 5:16:48 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.030 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download