Hot to avoid ConsoleTool error in TC: NCrunch result: TestsNotRun

[NCrunchConsoleToolv3.17.0.2]

Is there a way I can prevent NCrunch to tell TeamCity it's an error if some tests were not run?

[img=https://i.imgur.com/wMOKDM5.png]TestsNotRun[/img]

(I cannot see it being mentioned in the docs)


(The process itself seems to exit with exit code 0, but maybe it never exits with exit code 5 in newer NCrunch version?):

[12:43:51] :	 [Step 4/4] [GridMessageSender-18] Resetting ping
[12:44:00]E:	 [Step 4/4] NCrunch result: TestsNotRun
[12:43:55]i:	 [Step 4/4] ##teamcity[buildProblem description='NCrunch result: TestsNotRun' timestamp='2018-06-08T12:43:55.204+0200']
...
[12:44:17] :	 [Step 4/4] [Core-27] Cleaning up workspace: D:\...\NcWc\9408\26
[12:44:19] :	 [Step 4/4] [?-1] Returning result: OK
[12:44:19] :	 [Step 4/4] Exit code: #0
[12:44:19] :	 [Step 4/4] Process exited with code 0

Thanks.

There isn't any built-in way of doing this, other than making sure all the tests get run (perhaps through the 'ignored tests' configuration).

I've noticed an oversight here in that tests marked as explicit will never be run by the console tool, and therefore will cause this result.

This can actually be a valid error case if someone has a filter that goes wrong or there are important grid nodes offline.

I would recommend wrapping the call to the console tool in a batch file or powershell script that can intercept the result and return something else to TeamCity.

Remco wrote:There isn't any built-in way of doing this, other than making sure all the tests get run (perhaps through the 'ignored tests' configuration)
...
This can actually be a valid error case if someone has a filter that goes wrong or there are important grid nodes offline.

In my case it is also valid since I have both explicit tests, HTTP integration tests (filtered via category), long running tests (category), performance tests (category) etc. which I don't want to be run as part of my "build on commit" CI config.
I also have a bunch of explicitly ignored tests we simply don't want to spend time on fixing right now.

Remco wrote:
I would recommend wrapping the call to the console tool in a batch file or powershell script that can intercept the result and return something else to TeamCity.

But it isn't the exit code that is the problem (which is #0), but it is the error reported by NCRunch as a TC service message.

GreenMoose wrote:
But it isn't the exit code that is the problem (which is #0), but it is the error reported by NCRunch as a TC service message.

Sorry, I didn't realise that was the case. I don't think this is what we should be doing here. We'll take a look at it and will see what we can do.

Just noticed there was a new command line switch /TeamCityDisableTestNotRunFailureReporting which worked out great!

GreenMoose wrote:Just noticed there was a new command line switch /TeamCityDisableTestNotRunFailureReporting which worked out great!

Yes! Sorry I'd forgotten that we'd added this one.