Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Hot to avoid ConsoleTool error in TC: NCrunch result: TestsNotRun
Options
Previous Topic Next Topic
GreenMoose
#1 Posted : Friday, June 8, 2018 11:17:49 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 507

Thanks: 145 times
Was thanked: 66 time(s) in 64 post(s)
[NCrunchConsoleToolv3.17.0.2]

Is there a way I can prevent NCrunch to tell TeamCity it's an error if some tests were not run?

TestsNotRun

(I cannot see it being mentioned in the docs)


(The process itself seems to exit with exit code 0, but maybe it never exits with exit code 5 in newer NCrunch version?):
Code:

[12:43:51] :     [Step 4/4] [GridMessageSender-18] Resetting ping
[12:44:00]E:     [Step 4/4] NCrunch result: TestsNotRun
[12:43:55]i:     [Step 4/4] ##teamcity[buildProblem description='NCrunch result: TestsNotRun' timestamp='2018-06-08T12:43:55.204+0200']
...
[12:44:17] :     [Step 4/4] [Core-27] Cleaning up workspace: D:\...\NcWc\9408\26
[12:44:19] :     [Step 4/4] [?-1] Returning result: OK
[12:44:19] :     [Step 4/4] Exit code: #0
[12:44:19] :     [Step 4/4] Process exited with code 0




Thanks.
Back to top
Remco
#2 Posted : Friday, June 8, 2018 11:39:22 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
There isn't any built-in way of doing this, other than making sure all the tests get run (perhaps through the 'ignored tests' configuration).

I've noticed an oversight here in that tests marked as explicit will never be run by the console tool, and therefore will cause this result.

This can actually be a valid error case if someone has a filter that goes wrong or there are important grid nodes offline.

I would recommend wrapping the call to the console tool in a batch file or powershell script that can intercept the result and return something else to TeamCity.
Back to top
GreenMoose
#3 Posted : Friday, June 8, 2018 11:51:33 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 507

Thanks: 145 times
Was thanked: 66 time(s) in 64 post(s)
Remco;12306 wrote:
There isn't any built-in way of doing this, other than making sure all the tests get run (perhaps through the 'ignored tests' configuration)
...
This can actually be a valid error case if someone has a filter that goes wrong or there are important grid nodes offline.


In my case it is also valid since I have both explicit tests, HTTP integration tests (filtered via category), long running tests (category), performance tests (category) etc. which I don't want to be run as part of my "build on commit" CI config.
I also have a bunch of explicitly ignored tests we simply don't want to spend time on fixing right now.

Remco;12306 wrote:

I would recommend wrapping the call to the console tool in a batch file or powershell script that can intercept the result and return something else to TeamCity.


But it isn't the exit code that is the problem (which is #0), but it is the error reported by NCRunch as a TC service message.
Back to top
Remco
#4 Posted : Friday, June 8, 2018 12:53:32 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
GreenMoose;12308 wrote:

But it isn't the exit code that is the problem (which is #0), but it is the error reported by NCRunch as a TC service message.


Sorry, I didn't realise that was the case. I don't think this is what we should be doing here. We'll take a look at it and will see what we can do.
Back to top
1 user thanked Remco for this useful post.
GreenMoose on 6/11/2018(UTC)
GreenMoose
#5 Posted : Wednesday, June 13, 2018 4:27:25 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 507

Thanks: 145 times
Was thanked: 66 time(s) in 64 post(s)
Just noticed there was a new command line switch /TeamCityDisableTestNotRunFailureReporting which worked out great!
Back to top
Remco
#6 Posted : Wednesday, June 13, 2018 7:32:27 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
GreenMoose;12324 wrote:
Just noticed there was a new command line switch /TeamCityDisableTestNotRunFailureReporting which worked out great!


Yes! Sorry I'd forgotten that we'd added this one.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.037 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download