I agree that it's probably worth investing your performance issues in combination with the other issues you're experiencing in your testing environment.
It's also important though to establish where the loss in performance is originating from, as this may have an impact on how you work with this environment.
I'm aware of several things that can cause the problem you've described:
1. Excessive file I/O that slows down any synchronous file system activities while you work in Visual Studio (for example, saving a change to a source file might result in a temporary lock up if you have a disk write queue several seconds long). Lots of build won't help here.
2. Insufficient CPU resources on CPU Cores that have been assigned to Visual Studio (see
http://www.ncrunch.net/documentation/reference_global-configuration_cpu-cores-assigned-to-ncrunch-or-vs). This can happen if an application outside NCrunch's control (i.e. MS Word in your case) overloads the cores being used by VS
3. Excessive network activity when working with distributed processing and many remote grid nodes - This is largely theoretical, and a threshold under which it can cause problems has not yet been established.
4. Excessive memory traffic causing heavy garbage collector compaction inside the IDE - This tends to happen if you have many VS packages installed on a machine that is struggling to keep up (watch out for VCS plugins - the default MS Git provider is not good here)
5. Not enough memory available, resulting in heavy paging
You may be able to narrow these down very quickly through the use of perfmon and watching your system resources in Task Manager. If you can figure out where the resources are falling short, I may be able to help with further advice on how to rectify.