Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Console Tool: How to run only certain projects?
Regenhardt
#1 Posted : Friday, April 5, 2019 6:43:09 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/5/2019(UTC)
Posts: 6
Location: Germany

Thanks: 3 times
Was thanked: 1 time(s) in 1 post(s)
Hello,

we have test projects with *.Unit.Tests and those with *.Integration.Tests, and want to run only unit tests in some cases. (No integration tests for standard feature branch pushes).
How can I tell the console tool to just run *.Unit.Tests.csproj Projects?
I don't really find that in the documentation. Honestly I don't really get a clear overview over the documentation.

Greetings,
Regenhardt
michaelkroes
#2 Posted : Friday, April 5, 2019 8:25:35 AM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 208
Location: Netherlands

Thanks: 94 times
Was thanked: 43 time(s) in 41 post(s)
Hi thanks for posting!

The easiest way to do this is via an engine mode. What I find easiest is to create the engine mode for the solution. This can be done via the menu NCrunch->Set Engine Mode->Customise Engine Modes. You can create a new engine mode there. This engine mode needs to have a filter defined. There is a filter section "Tests to execute automatically". You can specify a filter many ways, a regex seems the best here. Make sure it matches the test project names you want to run.

You can now test the engine inside visual studio. If it runs the tests as desired you can proceed to configure the console tool to use that engine mode. This can be done by adding /E "Name of the new engine mode".

I hope this helps! If you need more information, please let me know.
1 user thanked michaelkroes for this useful post.
Regenhardt on 4/5/2019(UTC)
Regenhardt
#3 Posted : Thursday, April 11, 2019 1:43:28 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/5/2019(UTC)
Posts: 6
Location: Germany

Thanks: 3 times
Was thanked: 1 time(s) in 1 post(s)
michaelkroes;13339 wrote:
Hi thanks for posting!

The easiest way to do this is via an engine mode. What I find easiest is to create the engine mode for the solution. This can be done via the menu NCrunch->Set Engine Mode->Customise Engine Modes. You can create a new engine mode there. This engine mode needs to have a filter defined. There is a filter section "Tests to execute automatically". You can specify a filter many ways, a regex seems the best here. Make sure it matches the test project names you want to run.

You can now test the engine inside visual studio. If it runs the tests as desired you can proceed to configure the console tool to use that engine mode. This can be done by adding /E "Name of the new engine mode".

I hope this helps! If you need more information, please let me know.



Hi Michael, I do indeed need some more information.

I now have it in my pipeline, and as planned only the Unit.Test projects get tests, anything else gets skipped.
Unfortunately, because of the the skipped tests, the end result is always "Tests skipped". For the pipeline to succeed though I need a Successful return code. Think I have to trick the pipeline, or can I tell ncrunch to only inform me about the run tests, without noting the skipped tests in the end result?
GreenMoose
#4 Posted : Thursday, April 11, 2019 5:16:21 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 469

Thanks: 130 times
Was thanked: 60 time(s) in 58 post(s)
I think what you are looking for is switch /TeamCityDisableTestNotRunFailureReporting, right?
Quote:

Process Exit/Return Codes
The exit codes returned by the console tool will vary depending upon the result of the execution run.

Exit Code Description
0 OK
1 Build Failure
2 Test Failure
3 General Failure (caused by internal failures and/or incorrect parameters)
4 License Failure
5 Some tests were not run (Introduced in v2.24. For many people, tests not running on a build server is considered an error case). If you are running on NCrunch v3.16 or above under Team City, you can specify /TeamCityDisableTestNotRunFailureReporting to disable this return code for Team City reporting.
2 users thanked GreenMoose for this useful post.
michaelkroes on 4/11/2019(UTC), Regenhardt on 5/3/2019(UTC)
michaelkroes
#5 Posted : Thursday, April 11, 2019 6:49:15 PM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 208
Location: Netherlands

Thanks: 94 times
Was thanked: 43 time(s) in 41 post(s)
Thanks GreenMoose! That's exactly the setting that needs to be set in this case.
Regenhardt
#6 Posted : Friday, May 3, 2019 9:21:22 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/5/2019(UTC)
Posts: 6
Location: Germany

Thanks: 3 times
Was thanked: 1 time(s) in 1 post(s)
Thanks a lot, exactly what I needed :D
Regenhardt
#7 Posted : Wednesday, January 29, 2020 2:48:39 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/5/2019(UTC)
Posts: 6
Location: Germany

Thanks: 3 times
Was thanked: 1 time(s) in 1 post(s)
So I finally got the time to finish this. Many green unit tests. CI-Job runs unit tests and doesn't run other tests.
For some reason though, it now reports result 5 (Some tests not run) in the end, despite having the argument /TeamCityDisableTestNotRunFailureReporting in the call.
Any idea why?
The report correctly shows green tests and "not run" tests only.

Edit: We're on console tool 3.30.0.1
Remco
#8 Posted : Wednesday, January 29, 2020 10:17:07 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,943

Thanks: 777 times
Was thanked: 998 time(s) in 951 post(s)
Regenhardt;14407 wrote:

Edit: We're on console tool 3.30.0.1


Can you confirm whether this happens for you with the latest version? (v4.2). We've seen quite a bit of development since 3.30, including a whole major version bump.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.058 seconds.