Welcome Guest! To enable all features please Login or Register.



Understanding Scale needs for Distributed Processing
#1 Posted : Monday, April 29, 2019 4:00:04 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/29/2019(UTC)
Posts: 1
Location: United States of America

We are evaluating the cost/benefit of a Distributed Processing cluster and are interested if there are best practices regarding the number of nodes per developer.

I realize this is going to be variable based on solution and tests, but am just needing to understand if normal is 1 server per 2 users or 15 servers per 2 users.

Thanks in advance!
#2 Posted : Monday, April 29, 2019 9:36:17 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,140

Thanks: 795 times
Was thanked: 1050 time(s) in 1000 post(s)
Hi Doug,

Thanks for posting! The documentation in this area gives some limited information you might find useful around grid performance. Generally, however, like most things in the software world, the answer is 'it depends'.

Because NCrunch needs to duplicate build tasks over the nodes, you will in most cases be better off having bigger, chunkier grid nodes as opposed to having a larger number of smaller nodes. Unless your setup calls for a large number of nodes (i.e. if you have resource concurrency issues of some kind), you will want all grid nodes to be shared by all users so that the resources are used as much as possible.

The number of grid nodes needed will vary heavily depending on the nature of your tests and the size of your solution.

The best approach you can take here would be to experiment. Start with maybe one big node, then evaluate whether your team is spending much time waiting for tests to run. If so, add another node, and keep adding them until you're certain that adding more won't make any difference in productivity. If you're leaning on a cloud based system, it should be easy to just keep adding nodes until it's not worth it. If you're working on physical hardware, maybe stage your procurement so you have time to assess things properly. Our own experience suggests that workstation CPUs seem to be a better choice for grid nodes than server ones (less reliable, but faster and more cost efficient).

Given how cheap hardware is compared to developer time, it's probably better to have more power than you need rather than the other way around. Do consider though the time that needs to be taken to maintain grid nodes. Every VS/platform update installed on a workstation must also be loaded onto every active NCrunch grid node.
Users browsing this topic
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.027 seconds.