Hi, thanks for sharing this issue.

When the analysis fails, how does NCrunch behave for you? Do you receive an error of some kind? Or do all your tests just suddenly disappear?

Are you using distributed processing?

Which test framework are you using?

Are you making use of any complex code to dynamically construct your tests? (i.e. NUnit TestCaseSource)

Try the following:

When you start your NCrunch session, go to your configuration and set your 'Log Verbosity' to 'Detailed'. You'll need to do this at the start of each session, as the engine will automatically reset this value when it initialises (as this setting has a big impact on performance).

When the problem appears, go to your Processing Queue Window and look for the Analysis Task representing the test project experiencing the problem. Click on the task so its trace data shows in the pane below. Do you see anything interesting in this trace data? It would be great if you could copy/paste it here or submit it through the contract form.

I'm also interested in knowing whether the problem appears when you're not using distributed processing. If the conditions on the grid node server are somehow different to your client machine in a way that causes it to not detect the tests, this could explain the intermittent behaviour. When you are using distributed processing, sometimes NCrunch will take the discovery results from the node, and sometimes it will take them from the client machine (depending on which finishes first). Trying to work exclusively using certain nodes or the local client machine would be a good way of narrowing down a problem like this.

You mentioned that you are using TestCaseSource, are you using it for all the tests that are not being reported? Is there much complexity behind the generation of the tests?

From the top of my head, I don't think it's possible to feed additional information into the NCrunch trace log during its discovery step .. but there would be nothing stopping you from creating your own log file in a known location.

Do you have other tests in the same suite that are not using TestCaseSource? If so, I would find it strange if you've managed to find a TestCaseSource error that could prevent NUnit from discovering everything within the suite ... there must be an exception kicked up somewhere. I think that obtaining a diagnostic log of the discovery task would be very helpful in analysing this problem.