Grid Node Storage Data Limit

I have a Grid Node server with the 'Solution storage data limit (in MB)' setting set to 5000. On the Grid Node server itself, the workspace base path currently has 12 GB of data in it.

How is the storage data limit applied on a Grid Node server? I understand it can't exactly cut-off at the configured limit but exceeding the limit by twice seems excessive.

Hi, thanks for posting.

The solution/snapshot storage on the grid node is a long term storage system designed to store raw solutions between sessions on the grid node. Its size needs to be carefully controlled by the grid node to prevent it from growing forever.

The workspace storage is transient storage that survives only for the duration of a session. As soon as a grid client disconnects from the server, the workspace storage for the client's session will be removed.

So basically, these are two very different concepts. The solution storage data limit doesn't apply to the workspace storage. It has been requested that a workspace storage limit be introduced, but such a system would be self-defeating as the engine would only be able to police such a limit by slowing itself down, which seems completely counter to the idea of actually setting up the node.

Okay, thanks for the explanation. In my case I only notice the problem every now and again so I assumed the clean-up wasn't happening and files were building up over time (in my case the Snapshots folder takes up all the space). It seems more likely I'm just doing something in Visual Studio that is causing a spike in the transient storage e.g. I had three solutions open this morning all of which would have been hitting the Grid Node Server.

VNicholson wrote:Okay, thanks for the explanation. In my case I only notice the problem every now and again so I assumed the clean-up wasn't happening and files were building up over time (in my case the Snapshots folder takes up all the space). It seems more likely I'm just doing something in Visual Studio that is causing a spike in the transient storage e.g. I had three solutions open this morning all of which would have been hitting the Grid Node Server.

NCrunch is quite militant in keeping the snapshot storage directory under the specified limit. If there is a new solution being uploaded to the grid node and there are no solutions to clear out (i.e. they're all in use), the server will actually kick up an error rather than go over the limit.

If you have your workspace base path set under the same place as the snapshot storage though, then the disk space usage of this area will likely be considerably higher than the limit you've specified - especially if you're working with many large solutions with a high level of concurrency.

There are exceptional situations where the grid node can go over the snapshot storage limit - such as if you have a test that reaches back into the foreground solution and stores data there. The grid node will then detect this data when it's next restarted and should adjust the snapshot storage accordingly.

Remco wrote:

NCrunch is quite militant in keeping the snapshot storage directory under the specified limit. If there is a new solution being uploaded to the grid node and there are no solutions to clear out (i.e. they're all in use), the server will actually kick up an error rather than go over the limit.

I'm not sure if I misunderstand the discussion above. On our grid nodes the "Solution storage data limit (in MB)" is set to 10GB but the Snapshot directory is 66GB (which is the size it ended up with when the disk was full). Am I setting the wrong limit or is there a bug. Here's a screen shot of my settings.

eriove wrote:
I'm not sure if I misunderstand the discussion above. On our grid nodes the "Solution storage data limit (in MB)" is set to 10GB but the Snapshot directory is 66GB (which is the size it ended up with when the disk was full). Am I setting the wrong limit or is there a bug. Here's a screen shot of my settings.

This doesn't look right to me. Do you have any code that might try to store or create files under your solution directory? NCrunch won't try to stop anything else from adding files to Snapshots and it doesn't have a facility to track files that are added here outside of its normal grid synchronisation process.