Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Extremely slow file transferring on distributing processing
Options
Previous Topic Next Topic
avishnyakov
#1 Posted : Tuesday, April 26, 2016 2:40:23 PM(UTC)
Rank: Member

Groups: Registered
Joined: 7/12/2015(UTC)
Posts: 27
Location: Australia

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
We constantly have extremely slow filr transferring with the latest NCrunch version while using distributed processing.

It takes an hour or so to get several grid node in sync. NCrunch just stuck at some number, like 11 / 957 or 20/957. Does not say anything, constantly looses connection, starts over and stuck. No tests can be run at all. Azure A4 sized VMs are used underneath, not sure what might be the reason for all that thing.

[img](- BROKEN LINK -)[/img]
Back to top
Remco
#2 Posted : Tuesday, April 26, 2016 11:56:59 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this issue.

Do you have any particularly large files included in this solution? The fact that it is stuck on one particular file suggests that there may be a very big one that needs to be uploaded.

At the moment, the upload resume functionality works per-file, so if the connection is cut or lost while uploading a big file, the upload for this file must resume from scratch the next time a connection is established.
Back to top
avishnyakov
#3 Posted : Wednesday, April 27, 2016 12:11:07 AM(UTC)
Rank: Member

Groups: Registered
Joined: 7/12/2015(UTC)
Posts: 27
Location: Australia

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
Correct, we have some 3rd part assemblies altogether up to ~100M. Next, that seems to be related to local intranet provider and unusual, random port NCrunch uses. Switching to mobile internet helped, lol.

Donno, may be zipping stuff in chunks over back to back file upload will do better?
Also, noticed that NCruch maxes out the internet connection, like literally killing it while transferring and as a consequence - you really CANNOT sync 10-15 grid nodes, you HAVE to turn them off and manually sync one-by-one. Of all 10-15 nodes are up, NCruch might max out the connection and nothing gets pushed at all ending up with endless re-connections from scratch.

All that makes the whole experience, well, pretty crazy.
Back to top
Remco
#4 Posted : Wednesday, April 27, 2016 12:25:54 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
What is the largest file you have in your solution? If you have any snapshots that are fully uploaded to your grid server, can you check the size of these snapshots and see if they contain any particularly large files? A 10MB assembly file shouldn't be an issue, but a 100MB file might be a problem if you have a slow connection or your ISP is throttling you. Considering many ISPs have systems to try and block heavy traffic on standard ports (to prevent torrents etc), changing the NCrunch port to something more standard might be worth trying. You can easily adjust the listening port in the grid node server configuration.

Note that if a grid node server already contains a required file in any of its stored snapshots, it won't request the file to be uploaded again. This means that you can manually copy the snapshot up to the grid node (using ZIP and FTP or whatever) and the node will use files from this snapshot to build any additional snapshots it requires.
Back to top
avishnyakov
#5 Posted : Wednesday, April 27, 2016 12:34:00 AM(UTC)
Rank: Member

Groups: Registered
Joined: 7/12/2015(UTC)
Posts: 27
Location: Australia

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
No, ~100M on total. All assemblies are about 1-15M, nothing really big.
Thanks! I'll give a try with standard ports setting up NCrunch on 80 or something, will see.

Again, with mobile connection all went quite alright. Seems to be the first sync only.
Back to top
Remco
#6 Posted : Wednesday, April 27, 2016 1:06:24 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
avishnyakov;8678 wrote:

Again, with mobile connection all went quite alright. Seems to be the first sync only.


If the data transfer seems very slow over a normal (wired) connection, but is fast over the mobile connection, then the network is definitely a suspect :)

I'm not sure what kind of setup you're using with VMs, but if you are creating new VMs from a base image regularly, it might be worth storing the larger and less frequently changed parts of your solution inside the VM image. In this way, most of the files will already be on the node prior to the first sync starting up.
Back to top
avishnyakov
#7 Posted : Wednesday, April 27, 2016 1:07:58 AM(UTC)
Rank: Member

Groups: Registered
Joined: 7/12/2015(UTC)
Posts: 27
Location: Australia

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
Thanks! Guess, sorted. Will update with more insights once we have something new. Thanks for the help, again.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.084 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download