Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Running tests manually while engine mode is in Impacted does not run all tests
Options
Previous Topic Next Topic
nrjohnstone
#1 Posted : Friday, March 18, 2016 1:55:34 AM(UTC)
Rank: Member

Groups: Registered
Joined: 7/1/2015(UTC)
Posts: 12
Location: New Zealand

Thanks: 1 times
Was thanked: 2 time(s) in 2 post(s)
I've noticed recently that NCrunch is not running all the tests when I right click on a node and run all selected tests.

The enging mode is set to "Impacted tests and others manually" I have the test output window grouping by Namespaces, Fixture, Test and if I right click on say a top level name space that contains 2000 tests, then select "Run selected tests in new task runner process" it only queues up a seemingly random amount, eg 700 sometimes, 753 etc.. but never the full 2000 and there are tests that end up with the ? symbol on them.

As soon as I change the engine to "Manual only" and then do the same thing, all 2000 run.

I think in previous versions, when I had "Run impacted tests and others manually" it would always run all of the tests I had selected manually.

Is this behavior you can repro Remco? It's pretty consistent for me.

NCrunch is at 2.19.0.4 and Visual Studio 2015 with Update 1.


UPDATE

Sorry I just noticed this sounds very similar to this issue here

http://forum.ncrunch.net...-Running-All-Tests.aspx

Maybe this provides some more information Remco about the Engine Mode having an impact?
Back to top
 |  Edit by user
Remco
#2 Posted : Friday, March 18, 2016 2:12:10 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,000

Thanks: 932 times
Was thanked: 1259 time(s) in 1172 post(s)
Hi,

This looks suspiciously like a problem that has been identified in the 2.19 (and possibly versions prior) release. NCrunch has a problem where it is clearing things out of the processing queue in response to certain events (such as builds being completed or projects being changed).

There are some fixes pending in 2.20 that I hope should resolve this. 2.20 is expected to be out within the next week or so. I think the best plan of action here would be to wait for the 2.20 release, then see if you can reproduce this problem. If it still exists, I'll investigate further :)
Back to top
Remco
#3 Posted : Monday, March 21, 2016 2:13:00 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,000

Thanks: 932 times
Was thanked: 1259 time(s) in 1172 post(s)
2.20 has just been released if you'd like to give this a try.
Back to top
nrjohnstone
#4 Posted : Monday, March 21, 2016 2:21:56 AM(UTC)
Rank: Member

Groups: Registered
Joined: 7/1/2015(UTC)
Posts: 12
Location: New Zealand

Thanks: 1 times
Was thanked: 2 time(s) in 2 post(s)
downloading right now :-) fingers crossed
Back to top
nrjohnstone
#5 Posted : Monday, March 21, 2016 2:25:45 AM(UTC)
Rank: Member

Groups: Registered
Joined: 7/1/2015(UTC)
Posts: 12
Location: New Zealand

Thanks: 1 times
Was thanked: 2 time(s) in 2 post(s)
actually... I can't test it as I've just gotten a message that our licence key is now 1 year out of date... that's a bit harsh isn't it? Not being able to manually run the tests reliably is a show stopper, should I really have to purchase a new licence for this Remco ??
Back to top
Remco
#6 Posted : Monday, March 21, 2016 2:32:58 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,000

Thanks: 932 times
Was thanked: 1259 time(s) in 1172 post(s)
nrjohnstone;8451 wrote:
actually... I can't test it as I've just gotten a message that our licence key is now 1 year out of date... that's a bit harsh isn't it? Not being able to manually run the tests reliably is a show stopper, should I really have to purchase a new licence for this Remco ??


Sorry, this is, unfortunately, the way the support/maintenance works.

To try and draw a line between what is a 'reasonable' or 'unreasonable' bug to require a purchase of an upgrade is subjective, and is a debate that the 12 month support/maintenance period was expected to remove. I can't make exceptions to this. I have, however, issued you a temporary 14 day license to allow you to test whether the problem is solved so that you can decide whether or not you wish to buy an upgrade.
Back to top
nrjohnstone
#7 Posted : Monday, March 21, 2016 2:37:28 AM(UTC)
Rank: Member

Groups: Registered
Joined: 7/1/2015(UTC)
Posts: 12
Location: New Zealand

Thanks: 1 times
Was thanked: 2 time(s) in 2 post(s)
I've installed it using the 30 day trial and it definitely seems to have fixed the issue... time to talk to the boss man now about upgrades...

Although, we have around 16000 tests across 100+ projects in our solution, and when you hit the "run all" button it has a noticable hang in VisualStudio now for around 10-15 seconds
Back to top
 |  Edit by user
Remco
#8 Posted : Monday, March 21, 2016 2:47:51 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,000

Thanks: 932 times
Was thanked: 1259 time(s) in 1172 post(s)
nrjohnstone;8453 wrote:

Although, we have around 16000 tests across 100+ projects in our solution, and when you hit the "run all" button it has a noticable hang in VisualStudio now for around 10-15 seconds


Is this a new issue that's just appeared with 2.20? Can you confirm that its VS itself hanging and not the NCrunch engine? i.e. if you try to click around the IDE, is it responsive?
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.048 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download