NCrunch seems to behave differently to the Resharper and MSTest runners when it encounters test methods annotated with ExpectedException annotations.
When the normal ExpectedException annotation is encountered, NCrunch works fine. When an annotation derived from ExpectedExceptionBaseAttribute is encountered however, NCrunch ignores the exception expectation whereas the Resharper and MSTest runners work fine.
A minimal reproduction is shown below.
[TestClass]
public sealed class ExpectedExceptionTests
{
[TestMethod]
[ExpectedException(typeof(InvalidOperationException))]
public void ExpectedExceptionAttributeTest()
{
throw new InvalidOperationException("This is fine");
}
[TestMethod]
[DerivedExpectedExceptionAttribute(typeof(InvalidOperationException))]
public void ExpectedExceptionDerivedAttributeTest()
{
throw new InvalidOperationException("This is fine in Resharper and MSTest, but fails in NCrunch because the derived expected exception annotation is ignored");
}
private sealed class DerivedExpectedExceptionAttribute : ExpectedExceptionBaseAttribute
{
private readonly Type _expectedExceptionType;
public DerivedExpectedExceptionAttribute(Type expectedExceptionType)
{
_expectedExceptionType = expectedExceptionType;
}
protected override void Verify(Exception exception)
{
Assert.IsInstanceOfType(exception, _expectedExceptionType);
}
}
}
Could NCrunch be updated to support annotations that derive from ExpectedExceptionBaseAnnotation the way the other runners do?
Unfortunately, we aren't able to support custom test metadata types under MSTest. This is due to the design of MSTest itself where they've made it very difficult for 3rd party software (like NCrunch) to implement custom behaviour around MSTest types like this one.
NCrunch's MSTest adapter is emulated rather than integrated. Trying to make our features work for MSTest by integrating with all versions of it released over the last 20 years simply wasn't something we were able to do, so we instead have our own adapter that behaves as much like MSTest as we were easily able to implement. Custom exception attributes is one of the areas where the effort required to handle it correctly makes it infeasible for us.
Remco wrote:NCrunch's MSTest adapter is emulated rather than integrated. Trying to make our features work for MSTest by integrating with all versions of it released over the last 20 years simply wasn't something we were able to do, so we instead have our own adapter that behaves as much like MSTest as we were easily able to implement. Custom exception attributes is one of the areas where the effort required to handle it correctly makes it infeasible for us.
No worries. I have a workaround so I'll just run with that. ExpectedException is considered pretty meh compared to Assert.ThrowsException these days anyway so I'll just update these legacy tests.