Hi Peter -

From what you've described, the setup you're using should allow NCrunch to run these tests in parallel. Because the tests are always in separate processes and you're testing in-process, there shouldn't be any interference around the parallelisation of the tests. The randomisation of port numbers is extremely unlikely to give you trouble - if it does, you can try seeding the random values using the test runner's process ID, as this will always be unique between parallel threads.

I think that this problem is being caused somehow by the tests themselves being sequence dependent. Where the sequence dependency exists is an interesting question and it may take some diving into SS.NET to figure out. If there is something in your test code that makes a call to HttpListenerBase.Init (directly or indirectly), you may wish to try controlling access by checking against another static member beforehand, i.e. only call this method if it hasn't been called before in the current application domain/process.

I'm wondering if it may be possible to easily reproduce this problem by running the same test twice. If you have a test throw this out at you, try right clicking on the test in NCrunch, going to Advanced, then choosing to run it using an existing task runner process. Try this a few times to make sure you do actually get the same task runner twice. If its a simple static reinitialise call, then this should surface the problem and make it easier to isolate/fix.

The regex idea on retrying for particular stack traces is a really good one. I'd been looking to retry options for flakey tests and the regex concept will definitely help with this - I'll note it down :)

Cheers,

Remco