We're moving from using separate NUnit and MSpec testing steps in TeamCity to using the NCrunch Console Tool, mainly because the MSpec runner does not support running tests in parallel so we get a significant performance increase even taking into account NCrunch doing its own build of the entire project.

First of all it seems like the instructions for setting up the Console Tool with TeamCity are no longer accurate for TeamCity v2017.1.3.



I started getting test results showing in TeamCity even without the add artifact paths step (above), and adding the artifact path had no effect (beside the files showing up under artifacts). I also see the tests showing up while the Console Tool is still running, so it looks like TeamCity is somehow automatically detecting the tests are they are running.

I have found two somewhat irritating issues with this automatic test detection though. The first issue is that it lists an extra test for each spec class resulting in a lot of extraneous tests listed. Here an example from the last run:


Using the MSpec runner this shows up as 4 tests, but here (and in AllResults.html) we get the extra "AccountDtoBuilderSpecs.when_getting_quarter_contract_name.*".

The second issue is that the Duration listed for each test is completely wrong. If you look at my example above every single test has a listed duration of only a couple milliseconds even though I know from previous runs that this test class takes around half a second to run.

I suspect this may have something to do with the automatic test detection not realizing the tests are being run in parallel.

One thing I experimented with was going to Build Features => Add => XML report processing and selecting a report type of NUnit and adding TestResultsInNUnitFormat.xml.

This resolved both issues:


The problem with this is that I haven't found a way to disable the automatic test detection so now I get both the problematic ones and the good ones. This seems to be the expected behavior based on the "Please make sure that tests are not detected automatically before using this feature." warning on this build feature in TeamCity.

Is there a way to either fix the automatic test detection so it reports the expected number of tests and the correct duration or to disable it so I can use the XML report processing feature? I'd prefer to use the automatic test detection since it shows incremental results as the tests are being run but either would be an improvement.

I'm happy to file some/all of this as an issue with TeamCity if appropriate but it's not clear to me whether this is an issue with the Console Tool or with TeamCity.

Thanks for all your hard work on NCrunch.

Thanks for your response and your honesty about when this can be addressed. It works well enough for our purposes for now. It would be great if you could take a look at improving it when things settle down.