General Feedback

duplicate source code transmission to node servers

Started by Grendil on 6,090 views

We're looking into setting up an NCrunch node farm, and we're wondering about network traffic load. Most developers will be working on the same large code base. Only about 10% of that code will be mutating at any one time. Does NCrunch do anything clever to avoid transmitting the same versioned code file coming from 5 different developer machines?
The first sync may take a few minutes (we use NCrunch over residential internet connections on a daily basis) but there is some logic in there to re-use identical files (so they only need to be transferred once)
Grendil wrote:We're looking into setting up an NCrunch node farm, and we're wondering about network traffic load. Most developers will be working on the same large code base. Only about 10% of that code will be mutating at any one time. Does NCrunch do anything clever to avoid transmitting the same versioned code file coming from 5 different developer machines?


Sorry for the late reply here. I'm not sure how this question slipped my net.

NCrunch does have logic to avoid re-copying files that it doesn't need to. It has an internal hash cache of all files in the snapshot storage directory. When a file is requested for a snapshot that matches an existing file in the storage area, it is simply copied across. Snapshots are also stored in an intact form on each node so that they can be quickly picked up and re-used. If everything is operating as designed, any reconnection and resync with a node should be quick and painless.

Post a reply

Log in to reply.