Hi Garry,

Right now NCrunch uses its own custom static analysis method to find MSpec tests. This approach relies on the full type/namespace name of the 'It' type. It is extremely fast, although there is no support for more complex test declarations like what you've described.

This sort of support would require NCrunch to execute a dynamic analysis step against the assembly, calling into MSpec and asking it to identify the tests within the assembly. This would be much slower, as it involves the use of run-time reflection inside a specially constructed application domain.

Would you be interested in seeing such a feature introduced?

Hi Garry -

You're correct in your understanding that the static analysis could be modified to include support for this feature. However, the extra search logic would impact the performance of the algorithm.

There is also the evidence you've presented that the current analysis approach with MSpec does not cover all features within MSpec. Rather than trying to directly add support for edge case features such as this one piece-by-piece (and effectively rebuild MSpec), the sensible approach seems to be to simply ask MSpec which tests/specs exist within the assembly. This is the approach currently taken with Gallio and NUnit, and it does work very well. This makes it possible to choose which approach you'd like to take - the one that provides the best support, or the one that provides the best performance.

Performance isn't usually much of a problem if you have a test project with just a few dozen classes, but it becomes a much bigger factor when working with extensive test libraries with thousands of tests. The ability to make the choice between performance and compatibility was the driving motive behind the framework utilisation type configuration setting. MSpec has been a step behind the other frameworks in NCrunch's development because compared with the other major frameworks (NUnit, MSTest, xUnit), it has a smaller user base.

I'm looking at putting some feature voting up on the website here soon. If you like, you're welcome to vote for improved MSpec support as a feature when this is online.

I've just set up a uservoice page for NCrunch to help collect feature requests. You're welcome to share your thoughts on both the SDK/API and the MSpec dynamic analysis if you like - https://ncrunch.uservoice.com/. It would be useful to have these features listed to help gauge their popularity.