Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Grid Node Storage Data Limit
VNicholson
#1 Posted : Wednesday, March 30, 2016 8:52:34 PM(UTC)
Rank: Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 25
Location: New Zealand

Thanks: 7 times
Was thanked: 3 time(s) in 3 post(s)
I have a Grid Node server with the 'Solution storage data limit (in MB)' setting set to 5000. On the Grid Node server itself, the workspace base path currently has 12 GB of data in it.

How is the storage data limit applied on a Grid Node server? I understand it can't exactly cut-off at the configured limit but exceeding the limit by twice seems excessive.
Remco
#2 Posted : Wednesday, March 30, 2016 10:30:53 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi, thanks for posting.

The solution/snapshot storage on the grid node is a long term storage system designed to store raw solutions between sessions on the grid node. Its size needs to be carefully controlled by the grid node to prevent it from growing forever.

The workspace storage is transient storage that survives only for the duration of a session. As soon as a grid client disconnects from the server, the workspace storage for the client's session will be removed.

So basically, these are two very different concepts. The solution storage data limit doesn't apply to the workspace storage. It has been requested that a workspace storage limit be introduced, but such a system would be self-defeating as the engine would only be able to police such a limit by slowing itself down, which seems completely counter to the idea of actually setting up the node.
VNicholson
#3 Posted : Wednesday, March 30, 2016 11:57:57 PM(UTC)
Rank: Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 25
Location: New Zealand

Thanks: 7 times
Was thanked: 3 time(s) in 3 post(s)
Okay, thanks for the explanation. In my case I only notice the problem every now and again so I assumed the clean-up wasn't happening and files were building up over time (in my case the Snapshots folder takes up all the space). It seems more likely I'm just doing something in Visual Studio that is causing a spike in the transient storage e.g. I had three solutions open this morning all of which would have been hitting the Grid Node Server.
Remco
#4 Posted : Thursday, March 31, 2016 12:03:42 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
VNicholson;8508 wrote:
Okay, thanks for the explanation. In my case I only notice the problem every now and again so I assumed the clean-up wasn't happening and files were building up over time (in my case the Snapshots folder takes up all the space). It seems more likely I'm just doing something in Visual Studio that is causing a spike in the transient storage e.g. I had three solutions open this morning all of which would have been hitting the Grid Node Server.


NCrunch is quite militant in keeping the snapshot storage directory under the specified limit. If there is a new solution being uploaded to the grid node and there are no solutions to clear out (i.e. they're all in use), the server will actually kick up an error rather than go over the limit.

If you have your workspace base path set under the same place as the snapshot storage though, then the disk space usage of this area will likely be considerably higher than the limit you've specified - especially if you're working with many large solutions with a high level of concurrency.

There are exceptional situations where the grid node can go over the snapshot storage limit - such as if you have a test that reaches back into the foreground solution and stores data there. The grid node will then detect this data when it's next restarted and should adjust the snapshot storage accordingly.
eriove
#5 Posted : Wednesday, June 21, 2017 7:09:17 AM(UTC)
Rank: Member

Groups: Registered
Joined: 3/22/2016(UTC)
Posts: 21
Location: Sweden

Thanks: 14 times
Was thanked: 2 time(s) in 2 post(s)
Remco;8509 wrote:


NCrunch is quite militant in keeping the snapshot storage directory under the specified limit. If there is a new solution being uploaded to the grid node and there are no solutions to clear out (i.e. they're all in use), the server will actually kick up an error rather than go over the limit.



I'm not sure if I misunderstand the discussion above. On our grid nodes the "Solution storage data limit (in MB)" is set to 10GB but the Snapshot directory is 66GB (which is the size it ended up with when the disk was full). Am I setting the wrong limit or is there a bug. Here's a screen shot of my settings.

Remco
#6 Posted : Wednesday, June 21, 2017 9:17:41 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
eriove;10672 wrote:

I'm not sure if I misunderstand the discussion above. On our grid nodes the "Solution storage data limit (in MB)" is set to 10GB but the Snapshot directory is 66GB (which is the size it ended up with when the disk was full). Am I setting the wrong limit or is there a bug. Here's a screen shot of my settings.



This doesn't look right to me. Do you have any code that might try to store or create files under your solution directory? NCrunch won't try to stop anything else from adding files to Snapshots and it doesn't have a facility to track files that are added here outside of its normal grid synchronisation process.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.068 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download