Custom MSpec delegates not recognized...

Greetings

I've been playing around with customizing MSpec for our environment and wanted to change the It delegate to a Then to match a BDD style. Therefore I'd have the following:


original:
It should_have_a_value_of_twenty;

new:
Then it_should_have_a_value_of_twenty

Unfortunately NCrunch doesn't recognize the new delegate, however ReSharper does so it would seem MSpec supports this substitution. To create a new version of It you can use this code:

[AssertDelegate]
public delegate void Then();

I'm not sure how you've implemented support for MSpec, but it seems if I name the new delegate It it's still not recognized. The other keywords of Because and Establish etc also suffer from this issue.

Thanks
Garry Mc

Hi Garry,

Right now NCrunch uses its own custom static analysis method to find MSpec tests. This approach relies on the full type/namespace name of the 'It' type. It is extremely fast, although there is no support for more complex test declarations like what you've described.

This sort of support would require NCrunch to execute a dynamic analysis step against the assembly, calling into MSpec and asking it to identify the tests within the assembly. This would be much slower, as it involves the use of run-time reflection inside a specially constructed application domain.

Would you be interested in seeing such a feature introduced?

Remco wrote:Hi Garry,

Right now NCrunch uses its own custom static analysis method to find MSpec tests. This approach relies on the full type/namespace name of the 'It' type. It is extremely fast, although there is no support for more complex test declarations like what you've described.

Thanks for the prompt response, I wont try and pretend I know a lot about dynamic vs static code analysis. Though I would have thought you'd be able to search for types that have the inherited attribute AssertDelegate (and the other custom attributes for the other MSpec types), using TypeNode.Attributes?. However, if this is not the case and its a lot of extra work to implement, then I guess its not worth the effort given I may be the only one who uses it! :) Though it would have made the extensions I'm making look more BDDish.

Let me know if this is possible,

Thanks!
Garry Mc

Hi Garry -

You're correct in your understanding that the static analysis could be modified to include support for this feature. However, the extra search logic would impact the performance of the algorithm.

There is also the evidence you've presented that the current analysis approach with MSpec does not cover all features within MSpec. Rather than trying to directly add support for edge case features such as this one piece-by-piece (and effectively rebuild MSpec), the sensible approach seems to be to simply ask MSpec which tests/specs exist within the assembly. This is the approach currently taken with Gallio and NUnit, and it does work very well. This makes it possible to choose which approach you'd like to take - the one that provides the best support, or the one that provides the best performance.

Performance isn't usually much of a problem if you have a test project with just a few dozen classes, but it becomes a much bigger factor when working with extensive test libraries with thousands of tests. The ability to make the choice between performance and compatibility was the driving motive behind the framework utilisation type configuration setting. MSpec has been a step behind the other frameworks in NCrunch's development because compared with the other major frameworks (NUnit, MSTest, xUnit), it has a smaller user base.

I'm looking at putting some feature voting up on the website here soon. If you like, you're welcome to vote for improved MSpec support as a feature when this is online.

Thanks, appreciate the explanation. The feature is really a 'nice to have' rather than a must have and I understand MSpec isn't quite as popular as the other major frameworks. One thing to think about, is to release a SDK for NCrunch that will allow framework developers to add the support themselves. I believe ReSharper does something like that with their product. Hopefully this would increase your frameworks support while reducing your development costs (well once you've written the SDK). Anyway something to consider.

Thanks...

I've just set up a uservoice page for NCrunch to help collect feature requests. You're welcome to share your thoughts on both the SDK/API and the MSpec dynamic analysis if you like - https://ncrunch.uservoice.com/. It would be useful to have these features listed to help gauge their popularity.

done