Hi Sam,

Thanks for posting. The best way I can think of to handle this would be to have a second engine mode that only runs tests that have been marked as failed.

Once the first run completes and the intermittent tests fail, they'll be stored in the NCrunch .cache file with a failed status.

The second build run then starts up to target only the failed tests, and runs only those ones again.

The test results from the second build will be the results you're after. If a test fails in both builds, then it's much less likely to be an intermittent failure.

The cache file is usually stored on the machine running NCrunch.exe. This file won't be stored on the grid nodes, although they do have cached data of a different kind (not relevant in this scenario).

The contents of the cache file can be quite important when using the NCrunch console tool in the build system, because NCrunch will use the data here to form a picture of what has changed in the solution since the last time the tests were run. This means that if you're running NCrunch.exe on different TC build agents, the cache file won't be consistent between runs and your build will function less efficiently. It does also mean that my suggestion wouldn't work, because if the runs were being handled by different build agents then the list of failed tests might not be available on the second run.

So you probably have a couple of options here. Either you can restrict the running of NCrunch.exe so it only happens on one of your build agents, or you can modify the storage directory of the NCrunch .cache file so that it lands on a shared network drive that is used by all the build agents. Note that sharing the cache file between build agents does give a slight chance of concurrency issues.

Brilliant! If I were to give an award for the most creative solution to an NCrunch related problem, you've definitely taken it for this year.

we came across the issue of partial regex matches using FullNameMatchesRegex
example test names:

• Import Test
  
• Import Test to backup
  
• AutoImport Test

then it will run all 3 example tests (as they all match that partial regex)


then it will run 2 example tests (Import Test and AutoImport Test)


then it runs 0 tests

what are we missing to do an exact name regex match?

thanks for the explanation Remco
we use FullNameMatchesRegex with spaces and that works

i've updated our regex match to include the fixture name and that works better


also, extending on samholder's solution, to reduce the total character length when there's 100s of tests to rerun