I've noticed recently that NCrunch is not running all the tests when I right click on a node and run all selected tests.
The enging mode is set to "Impacted tests and others manually" I have the test output window grouping by Namespaces, Fixture, Test and if I right click on say a top level name space that contains 2000 tests, then select "Run selected tests in new task runner process" it only queues up a seemingly random amount, eg 700 sometimes, 753 etc.. but never the full 2000 and there are tests that end up with the ? symbol on them.
As soon as I change the engine to "Manual only" and then do the same thing, all 2000 run.
I think in previous versions, when I had "Run impacted tests and others manually" it would always run all of the tests I had selected manually.
Is this behavior you can repro Remco? It's pretty consistent for me.
NCrunch is at 2.19.0.4 and Visual Studio 2015 with Update 1.
UPDATE
Sorry I just noticed this sounds very similar to this issue here
This looks suspiciously like a problem that has been identified in the 2.19 (and possibly versions prior) release. NCrunch has a problem where it is clearing things out of the processing queue in response to certain events (such as builds being completed or projects being changed).
There are some fixes pending in 2.20 that I hope should resolve this. 2.20 is expected to be out within the next week or so. I think the best plan of action here would be to wait for the 2.20 release, then see if you can reproduce this problem. If it still exists, I'll investigate further :)
actually... I can't test it as I've just gotten a message that our licence key is now 1 year out of date... that's a bit harsh isn't it? Not being able to manually run the tests reliably is a show stopper, should I really have to purchase a new licence for this Remco ??
nrjohnstone wrote:actually... I can't test it as I've just gotten a message that our licence key is now 1 year out of date... that's a bit harsh isn't it? Not being able to manually run the tests reliably is a show stopper, should I really have to purchase a new licence for this Remco ??
Sorry, this is, unfortunately, the way the support/maintenance works.
To try and draw a line between what is a 'reasonable' or 'unreasonable' bug to require a purchase of an upgrade is subjective, and is a debate that the 12 month support/maintenance period was expected to remove. I can't make exceptions to this. I have, however, issued you a temporary 14 day license to allow you to test whether the problem is solved so that you can decide whether or not you wish to buy an upgrade.
I've installed it using the 30 day trial and it definitely seems to have fixed the issue... time to talk to the boss man now about upgrades...
Although, we have around 16000 tests across 100+ projects in our solution, and when you hit the "run all" button it has a noticable hang in VisualStudio now for around 10-15 seconds
nrjohnstone wrote:
Although, we have around 16000 tests across 100+ projects in our solution, and when you hit the "run all" button it has a noticable hang in VisualStudio now for around 10-15 seconds
Is this a new issue that's just appeared with 2.20? Can you confirm that its VS itself hanging and not the NCrunch engine? i.e. if you try to click around the IDE, is it responsive?