Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

duplicate source code transmission to node servers
Grendil
#1 Posted : Wednesday, July 26, 2017 9:32:46 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2017(UTC)
Posts: 48
Location: United States of America

Thanks: 19 times
Was thanked: 10 time(s) in 9 post(s)
We're looking into setting up an NCrunch node farm, and we're wondering about network traffic load. Most developers will be working on the same large code base. Only about 10% of that code will be mutating at any one time. Does NCrunch do anything clever to avoid transmitting the same versioned code file coming from 5 different developer machines?
MatthewSteeples
#2 Posted : Friday, August 18, 2017 11:30:45 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 10/28/2014(UTC)
Posts: 49
Location: United Kingdom

Thanks: 4 times
Was thanked: 4 time(s) in 4 post(s)
The first sync may take a few minutes (we use NCrunch over residential internet connections on a daily basis) but there is some logic in there to re-use identical files (so they only need to be transferred once)
1 user thanked MatthewSteeples for this useful post.
Grendil on 8/18/2017(UTC)
Remco
#3 Posted : Friday, August 18, 2017 11:56:39 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,238

Thanks: 691 times
Was thanked: 844 time(s) in 804 post(s)
Grendil;10856 wrote:
We're looking into setting up an NCrunch node farm, and we're wondering about network traffic load. Most developers will be working on the same large code base. Only about 10% of that code will be mutating at any one time. Does NCrunch do anything clever to avoid transmitting the same versioned code file coming from 5 different developer machines?


Sorry for the late reply here. I'm not sure how this question slipped my net.

NCrunch does have logic to avoid re-copying files that it doesn't need to. It has an internal hash cache of all files in the snapshot storage directory. When a file is requested for a snapshot that matches an existing file in the storage area, it is simply copied across. Snapshots are also stored in an intact form on each node so that they can be quickly picked up and re-used. If everything is operating as designed, any reconnection and resync with a node should be quick and painless.
1 user thanked Remco for this useful post.
Grendil on 8/18/2017(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.024 seconds.