Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Console tool reports "weird" test count with TeamCity
GreenMoose
#1 Posted : Tuesday, January 13, 2015 1:21:59 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 503

Thanks: 142 times
Was thanked: 66 time(s) in 64 post(s)
v2.10.0.4

I've set up a new trivial project with some tests.
In my vstudio NCrunch tells me I have 64 tests via "Monitoring 64 tests, with no tests ignored".
When I start console tool on teamcity in a clean workspace , I get 75 tests.
When I start console tool on teamcity in a non-clean workspace (i.e. same as above but not cleaned), I get 71 tests.

Some observations:
* The AllResults.html from the 2 scenarios with 71 resp. 75 tests have 0,09% difference in code coverage where the cleaned execution with 75 tests has less code coverage, where 1 line is not covered. I can't tell which line since the build log from the 2 cleaned vs non-cleaned test runs are indicating all tests within that particular file are run.
* If I do plain "text copy" in tab "Test results" for the 2 cleaned vs non-cleaned test results I see no orphan tests in either one.

Any clues what is going on?

(I use 3 grid nodes)

Thanks.
Remco
#2 Posted : Tuesday, January 13, 2015 9:36:33 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Thanks for sharing this.

When using the console tool, where do you see the reported number of tests? Is this in the HTML report exported from the tool, or perhaps something inside the trace output?

There are several different ways of counting tests. For example, one method may include fixtures as being physical tests, while another disregards fixtures and only counts child tests. Then there are ignored tests, tests excluded by engine mode, etc.

The difference between the clean vs non-clean workspace is interesting ... are you able to narrow this down to the 4 tests that are different? Note that NCrunch's console tool will completely rebuild new NCrunch workspaces for each execution, so in this case there may be a difference in the source itself.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.040 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download