Hi, thanks for posting and for providing the configuration file. I'll try to answer your questions directly.
1. Disabling RDI for your console run is strongly recommended, as the console tool doesn't have a way to access/present RDI data in a useful way, so this tends to just result in I/O churn that will slow down the run. You can turn off RDI for the console run by overriding the config setting using a command-line parameter passed to NCrunch.exe, as such:
NCrunch.exe mysolution.sln -EnableRDI=False
2. Most likely this is due to your I/O being a bottleneck with RDI enabled. You have a relatively high number of processing threads for just two machines, so without splitting the RDI storage between multiple drives, your SSD is likely struggling to keep up. I have also noticed that there seems to be a limit to the amount of data that can be processed through memory mapped files on Windows (which RDI is reliant upon), which seems to set a ceiling on the number of threads that can work with RDI enabled even on machines of very high spec. I can't tell you exactly where this will kick in for you as it's highly dependent on the nature of your tests and how aggressive your RDI collection settings are. Regardless, I expect that turning off RDI for your console run will result in improvement here.
If you still see timeouts after turning off RDI, it may be worth checking to see if you have any tests that are overstepping the engine's resource expectations. For example, if you have a large test that hammers some code with 20 threads (i.e. to check for race conditions), NCrunch will normally run this expecting it to use only a single thread. When combined with the other tests in the pipeline, this can cause resource consumption to spike in a way that might cause timeouts. If you have tests of this nature make sure to mark them with
UsesThreadsAttribute.
3. Your ideal
Max Number of Processing Threads will likely be higher for your CI run if you have RDI disabled. Note that there presently isn't a way to have your distributed server run with a different number of processing threads for RDI work vs other tasks (as it's complicated to do this with machine used like a shared resource).
If you don't care to use impact detection for your CI run, and you have no interest in any of the coverage related exports, you can turn off instrumentation by specifying '-InstrumentOutputAssembly false' on the command line for NCrunch.exe.
When instrumentation is off, you can also turn on compiler optimizations with '-CustomBuildProperties Optimize=true'.
Ensure your log verbosity is set to low, as logging a lot of data (even to a console window) can slow things down, so '-LogVerbosity Low'.
Here is the command line we use for our own CI build: NCrunch.exe nCrunch.sln /o NCrunchResults /c "C:\NCrunch Console Tool\globalconfig.crunch.v3.xml" /VS 2022 /E "Run all tests automatically" -InstrumentOutputAssembly false -LogVerbosity Low -CustomBuildProperties Optimize=true /IgnoredFilteredTestsInRunFailureReporting
Turning off instrumentation also implicitly disables RDI (which requires it).
4. Use the engine modes. The /E parameter for NCrunch.exe allows you to specify the engine mode used for execution. You can set up engine modes with filters that will only target specific groups of tests for execution (use the 'TestsToExecuteAutomatically' setting). NCrunch.exe will only execute tests that are marked for automatic execution.
It IS actually possible to use the impacted only engine mode, so it's possible to have a CI build that runs impacted tests only. The engine achieves this by using the same differential methods as the VS plugin. However, I would suggest using caution with this approach given that impact detection is intrinsically imperfect and your tests will still need to be run end-to-end before pushing to production. This also won't work if you disable instrumentation.
