Howto disable analysis task

"NCrunch Processing Queue" almost permanently shows a Task called "Analysis". "NCrunch Timeline Export to html" confirms, that most of the time one CPU core is busy doing analysis.

My understanding of the task "analysis" is, that NCrunch runs the roslyn analyzers that come with nuget packages like "Roslynator.Analyzers" or "Microsoft.CodeAnalysis.FxCopAnalyzers".

If that understading is correct, Visual Studio is doing the exact same thing and NCrunch does not need to do it.

Question: How can I make NCrunche not doing "analyisis", but focus on fast build and test execution?

(I have a rather old computer and about 5 such analyser nuget packages installed in a solution with 35 projects.)

Hi, thanks for posting.

The analysis task is a test discovery action. It involves creating a runtime environment (i.e. launching your code using the CLR and loading the assembly), then asking the test framework for details about the tests contained in the assembly. There is no relationship between this task and roslyn analysers (it was actually introduced years before Roslyn even existed in any public form).

The performance of this task is directly dependent on the performance of the testing framework you are using. Test frameworks such as NUnit and XUnit can be quite complex during test discovery as they build an in-memory model of your tests using reflection. If you want NCrunch to be able to execute your tests, there is no way to skip this task.

Later in the queue NCrunch will use the same runtime environment to execute your tests. This means that if there were a way to prevent this task from being run, the work being done would simply move to the execution tasks instead.

Ok, so disabling the analysis does not make any sense. :|o
That also explains, why the tests start running only after the analysis task has completed.

I actually use xUnit Facts and Theories.
I assume there is no way to accelerate the analysis task other than using a more powerfull computer or distributed processing. Is that correct?

samuelhess1 wrote:
I assume there is no way to accelerate the analysis task other than using a more powerfull computer or distributed processing. Is that correct?

Distributed Processing will be of very limited help here, because we presently don't distribute the analysis task in the same way as the test tasks. Right now the analysis task will run on each node (we have plans to change this eventually).

Upgrading your hardware will usually help.

It may also be possible to redesign your test suite to reduce the cost of test discovery. I'm not really an expect on the behaviour of the internals of Xunit, but I would expect that more complex constructions such as Theories will probably require more CPU to discover by the framework. Reducing your overall number of tests will improve discovery time. If the option is available, you could also try different test frameworks to see which ones will perform better for your solution. MSTest usually gives the best performance under NCrunch because this is run under optimised emulation (rather than being directly integrated).

Thank you for your tipp!

I've restructured one big Theory with ~150 InlineDataAttributes to a normal FactAttribute test. This reduced the required time of the Analysis task to less than a second.

However, the existence of Theories are the main reason I am using xUnit instead of MSTest - so I'm going to reconsider that decision :|

I've just learned, that MSTest has the same concept as xUnit Theories:

[DataRow(0, 0)]
[DataRow(1, 1)]
[DataRow(2, 1)]
[DataRow(80, 23416728348467685)]
[DataTestMethod]
public void MyTest(int number, int result)
{
// do something
}

I've also observed, that NCrunch does not have any "Analysis Task" when using MSTest. This explains why I've never seen it during the last 8 Years of NCrunch usage, but now suddenly (since I'am playing around with xUnit) it showed up :)

Before using xUnit, I consulted https://www.ncrunch.net/support/frameworks. Maybe it is a good idea to add a hint on ncrunch runtime perforamance for the single test frameworks. This might help others no not learn it the hard way as I diD.

samuelhess1 wrote:
Before using xUnit, I consulted https://www.ncrunch.net/support/frameworks. Maybe it is a good idea to add a hint on ncrunch runtime perforamance for the single test frameworks. This might help others no not learn it the hard way as I diD.

I'd like to do this, and actually have been planning for it for a while ... though it's sadly not as simple as making general comments about performance between the frameworks. Some frameworks do better at some things than others, and it usually depends largely on how the code is structured and which features of the frameworks are being used. It is also a bit of a tricky area politically, as we try to stay on good terms with the developers of these frameworks (most of which are volunteers), so publishing comparisons of the performance of their work would seem like picking sides.

Right now there is a very large piece of development work pushing its way up through the NCrunch engine to resolve issues with instrumentation performance. Following this, there'll probably be a renewed focus on the performance of test discovery and execution (which will be the new bottleneck), so I suppose many things are likely to change.