Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Hot to avoid ConsoleTool error in TC: NCrunch result: TestsNotRun
GreenMoose
#1 Posted : Friday, June 8, 2018 11:17:49 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 390

Thanks: 86 times
Was thanked: 40 time(s) in 39 post(s)
[NCrunchConsoleToolv3.17.0.2]

Is there a way I can prevent NCrunch to tell TeamCity it's an error if some tests were not run?

TestsNotRun

(I cannot see it being mentioned in the docs)


(The process itself seems to exit with exit code 0, but maybe it never exits with exit code 5 in newer NCrunch version?):
Code:

[12:43:51] :     [Step 4/4] [GridMessageSender-18] Resetting ping
[12:44:00]E:     [Step 4/4] NCrunch result: TestsNotRun
[12:43:55]i:     [Step 4/4] ##teamcity[buildProblem description='NCrunch result: TestsNotRun' timestamp='2018-06-08T12:43:55.204+0200']
...
[12:44:17] :     [Step 4/4] [Core-27] Cleaning up workspace: D:\...\NcWc\9408\26
[12:44:19] :     [Step 4/4] [?-1] Returning result: OK
[12:44:19] :     [Step 4/4] Exit code: #0
[12:44:19] :     [Step 4/4] Process exited with code 0




Thanks.

Remco
#2 Posted : Friday, June 8, 2018 11:39:22 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,325

Thanks: 700 times
Was thanked: 866 time(s) in 824 post(s)
There isn't any built-in way of doing this, other than making sure all the tests get run (perhaps through the 'ignored tests' configuration).

I've noticed an oversight here in that tests marked as explicit will never be run by the console tool, and therefore will cause this result.

This can actually be a valid error case if someone has a filter that goes wrong or there are important grid nodes offline.

I would recommend wrapping the call to the console tool in a batch file or powershell script that can intercept the result and return something else to TeamCity.
GreenMoose
#3 Posted : Friday, June 8, 2018 11:51:33 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 390

Thanks: 86 times
Was thanked: 40 time(s) in 39 post(s)
Remco;12306 wrote:
There isn't any built-in way of doing this, other than making sure all the tests get run (perhaps through the 'ignored tests' configuration)
...
This can actually be a valid error case if someone has a filter that goes wrong or there are important grid nodes offline.


In my case it is also valid since I have both explicit tests, HTTP integration tests (filtered via category), long running tests (category), performance tests (category) etc. which I don't want to be run as part of my "build on commit" CI config.
I also have a bunch of explicitly ignored tests we simply don't want to spend time on fixing right now.

Remco;12306 wrote:

I would recommend wrapping the call to the console tool in a batch file or powershell script that can intercept the result and return something else to TeamCity.


But it isn't the exit code that is the problem (which is #0), but it is the error reported by NCRunch as a TC service message.
Remco
#4 Posted : Friday, June 8, 2018 12:53:32 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,325

Thanks: 700 times
Was thanked: 866 time(s) in 824 post(s)
GreenMoose;12308 wrote:

But it isn't the exit code that is the problem (which is #0), but it is the error reported by NCRunch as a TC service message.


Sorry, I didn't realise that was the case. I don't think this is what we should be doing here. We'll take a look at it and will see what we can do.
1 user thanked Remco for this useful post.
GreenMoose on 6/11/2018(UTC)
GreenMoose
#5 Posted : Wednesday, June 13, 2018 4:27:25 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 390

Thanks: 86 times
Was thanked: 40 time(s) in 39 post(s)
Just noticed there was a new command line switch /TeamCityDisableTestNotRunFailureReporting which worked out great!
Remco
#6 Posted : Wednesday, June 13, 2018 7:32:27 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,325

Thanks: 700 times
Was thanked: 866 time(s) in 824 post(s)
GreenMoose;12324 wrote:
Just noticed there was a new command line switch /TeamCityDisableTestNotRunFailureReporting which worked out great!


Yes! Sorry I'd forgotten that we'd added this one.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.034 seconds.