Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Console tool and impacted tests, failed tests becomes ignored?
GreenMoose
#1 Posted : Friday, June 08, 2018 11:04:47 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 377

Thanks: 82 times
Was thanked: 40 time(s) in 39 post(s)
[NCrunchConsoleToolv3.17.0.2]

I am doing som trials with NCrunch console with a rather heavy test solution (14k tests) on TeamCity. But I am wondering why some of the failing tests are "disappearing" from the test results? See below:
TestHistory

When I look at the failed tests from #24, they do not have any records in build #25 and #26 (they were not executed since they were not impacted, I should have "Or is failing" in my filter), but shouldn't they be part of the "Ignored" tests?
SpecificTestHistory

Side note: I noticed the NCrunch cache directory is not generating a unique subfolder layout, I guess this means I should not share the NCrunch cache folder for different build configurations? Does this rule apply to all NCrunch's workspace and snapshot directory settings as well?

Thanks
Remco
#2 Posted : Friday, June 08, 2018 11:46:07 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,234

Thanks: 691 times
Was thanked: 843 time(s) in 803 post(s)
Where tests are excluded using the 'Tests to execute automatically' filter, the engine won't queue them for execution. Thus they won't have a result reported by the run as they also aren't considered to be 'ignored'. I guess this is probably also related to your other question, since essentially the use of the filter (which is very useful and powerful) results in a situation where you have tests missing from the result set. This then gives the 'TestsNotRun' 'Error' situation. I wonder if it might be better to consider such tests as ignored and not trigger the error at all when it is because of a filter.

GreenMoose
#3 Posted : Friday, June 08, 2018 12:01:28 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 377

Thanks: 82 times
Was thanked: 40 time(s) in 39 post(s)
Remco;12307 wrote:
Where tests are excluded using the 'Tests to execute automatically' filter, the engine won't queue them for execution. Thus they won't have a result reported by the run as they also aren't considered to be 'ignored'.
...


But all my tests excluded via filter are treated as "Ignored" (which is why I end up with 14 982 ignored tests). But let's leave this particular issue since I want to rerun failing tests anyhow so the behavior of failed tests vs ignored tests doesn't really matter.
Users browsing this topic
Guest (2)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.030 seconds.