Experiences/Advice about setting up an Azure VM as grid node

Hi,

I'm thinking about setting up a grid node because my solution has a lot of projects and tests and despite my machine being pretty well-equipped, this is starting to hurt a bit, especially on git-operations that merge incoming changes.

But as I am working from home and got only the one machine, I cannot create a node in my LAN.

So I was wondering whether it would be worth it to use an Azure VM.
Does anyone have experience with this?

* Does it make sense at all, given the network distance? (I do have a fairly good connection.)
* what VM size do you use?
* what cost do you typically accrue for that VM per month, assuming that you auto-hibernate it nightly?
* considering that the whole code-base is netstandard2.0 and net5 (should be upgraded to net6 within January, if that makes any difference) and for example the CI already runs on linux, could I use a linux VM? Can I install and run the grid-tools on linux? I'm asking because the linux machines are cheaper.
* I assume I'd have to create some sort of firewall rule in Azure to ensure that only my machine can talk to the VM - but every once in a while I get another IP assigned by my provider. Any ideas how to avoid having to update that rule manually again and again?
* any other caveats or good tips you can share?


Thank you,
MR

Hi MR,

Thanks for posting. Personally, my experience with the Azure platform itself is quite limited and not very up to date, so I'm going to limit my advice to the software side in the hope that someone else may be able to give some insight into their experiences in hosting NCrunch on Azure (I know that a number of people posting in this forum do this).

NCrunch's distributed processing is selective with the amount of data it pushes up the wire. Your solution will be stored on the remote node, which means that it must be uploaded over your network connection, but once it's up there it usually won't need to be sent again unless it sees significant changes (i.e. VCS branch switch). The protocol is designed to be responsive on connections with high latency and limited bandwidth. Having your grid node in an Azure data centre should work well in terms of network, especially if you have a site close to you geographically.

The cost is going to depend heavily on the type of VM you allocate. Generally speaking, you're best going for a chunky, high CPU node with just enough memory to fit the number of task runners you have planned (you'll need to measure this for your solution as it depends on the behaviour of your code). Note that later versions of .NET and MSBuild will require more memory.

There is presently no option to run the NCrunch grid node on linux. I think there's a high chance that we will eventually need to support it, but optimistically we are years away from this.

I strongly recommend against opening a TCP port in your Azure firewall for an NCrunch connection. The grid protocol isn't robust enough to give strong protection when routing through the internet. Ideally, set up a VPN connection to your cloud based instance and route through that instead. This will also give better security for your other connections (i.e. remote administration etc).

Be sure to allocate a VM with adequate disk space. It can sometimes be painful to increase it and the MS tool stack grows in size every year. If you have a large number of task processors, NCrunch can also eat quite a bit of disk space with its workspaces.