I seem to recall that NCrunch favors failing tests, understandably, because the programmer would want to know whether a fix will have had impact on such failing tests.

However, I notice that, at least when there are a lot of failing tests, this doesn't happen. For instance, all successful tests are put in batch jobs many together (jobs of 200-600 tests each) and those jobs get a priority of 141. At the same, all failing tests are put in jobs of 8 tests each, with priority 90.

Result: slow running because of many jobs (dozens of minutes vs 1-2 minutes when everything is successful).

It is probably important to note that failing tests report an "actual processing time" many times higher than when run by hand (between 200-500ms). This may be because NCrunch requires a lot of time to collect the data on the failing test (like unwinding the stacktrace). But still, this time seems larger than necessary (most tests run under 5ms, also the ones that test exception-throwing, meaning that these failing tests do not run so slow if they aren't run in NCrunch).

This particular test set has over 21,000 tests and only 5,000 are green/successful. Still, I don't see why NCrunch's heuristic ends up this way, as it seems such a detrimental outcome (as opposed to running many more tests at once in parallel).

PS (edit): (not sure this is related, but it caught my attention) I took a look at the I/O delta read p/sec, which shows this, which is roughly 10x lower than if I only run the green (successful) tests. So *something* seems to be slowing these down to a crawl:

Thanks for your response. This is NUnit 3.6. Running it from NUnit's test runner, it runs all tests in approx. 3 minutes. This test runner runs them synchronously. Looking at the failing tests, most run in 3-5ms it seems. Only a handful of tests seem to have a genuinely slow execution speed (above 10sec). The average of all tests (success and failing) is 9ms per test.

Bottom line, there seems to be no significant difference between the failing and successful tests. It would indeed be interesting to find out what happens here. I can send you a compiled executable, if that helps, or the sources of the test project (but in a previous thread you told me that such large projects aren't of much use for you to analyze).

Here's a capture of running it in NUnit:

[img](- BROKEN LINK -)[/img]

Totally forgot about this thread, but I think that presently, the test prioritization works as expected. I did notice the "bucketing system", where it kind-of depends how many tests (incl. failing tests) end up in the same task or bucket. The Processing Queue shows this properly.

My modus-operandi, if I have a few failing tests that I am investigating, is to pin them, and instruct NCrunch to run Pinned tests. This appears often to be the simplest way to focus on failing tests when the project contains a (very) large set of many quick tests (10k and more tests in my case).