Analysis fails on Attribute

NCrunch is reporting an analysis failure with a NPE in the constructor of an attribute. Below is the stack trace. Because of this no tests are run.

Do attributes need to be handled differently?

There is no error in the attribute itself. The project builds and I have used the attribute successfully.

Stack Trace

An error occurred while analysing this project after it was built: System.NullReferenceException: Object reference not set to an instance of an object.
   at eSante.DataGenerator.Core.Identities.IdentityAttribute..ctor(Type identityType) in D:\FoxSystems\Arms2\TestData\eSante.DataGenerator\eSante.DataGenerator.Core\eSante.DataGenerator.Core\Identities\IdentityAttribute.cs:line 16
   at System.Reflection.CustomAttribute._CreateCaObject(RuntimeModule pModule, IRuntimeMethodInfo pCtor, Byte** ppBlob, Byte* pEndBlob, Int32* pcNamedArgs)
   at System.Reflection.CustomAttribute.CreateCaObject(RuntimeModule module, IRuntimeMethodInfo ctor, IntPtr& blob, IntPtr blobEnd, Int32& namedArgs)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeType type, RuntimeType caType, Boolean inherit)
   at System.RuntimeType.GetCustomAttributes(Boolean inherit)
   at NUnit.Core.Reflect.GetAttributes(ICustomAttributeProvider member, Boolean inherit)
   at NUnit.Core.Reflect.HasAttribute(ICustomAttributeProvider member, String attrName, Boolean inherit)
   at NUnit.Core.Builders.SetUpFixtureBuilder.CanBuildFrom(Type type)
   at NUnit.Core.Extensibility.SuiteBuilderCollection.CanBuildFrom(Type type)
   at NUnit.Core.TestFixtureBuilder.CanBuildFrom(Type type)
   at NUnit.Core.Builders.TestAssemblyBuilder.GetFixtures(Assembly assembly, String ns)
   at NUnit.Core.Builders.TestAssemblyBuilder.Build(String assemblyName, Boolean autoSuites)
   at NUnit.Core.Builders.TestAssemblyBuilder.Build(String assemblyName, String testName, Boolean autoSuites)
   at NUnit.Core.TestSuiteBuilder.Build(TestPackage package)
   at nCrunch.Module.NUnit.Integration.NUnitDynamicTestFinder.FindFrameworkTestsUsingRuntimeInvoke(TestPackage package, ILogger logger, List`1 tests, Factory testNameFactory)
   at nCrunch.Module.NUnit.Integration.NUnitTestFramework.FindFrameworkTestsInAssembly(ReflectedAssembly assembly, FilePath assemblyFilePath, IList`1 referencedAssemblyFilePaths, ComponentUniqueName testComponentUniqueName)
   at nCrunch.TestExecution.TestFinder.FindTestsForFrameworks(ReflectedAssembly assembly, FilePath assemblyFilePath, IList`1 referencedAssemblyFilePaths, TestFrameworkDescription[] frameworks, ComponentUniqueName testComponentUniqueName)
   at nCrunch.TestExecution.RemoteTaskRunner.AnalyseAssembly(TestFrameworkDescription[] applicableFrameworks, ComponentUniqueName testComponentUniqueName)

Class causing issues

    public class IdentityAttribute : Attribute
    {
        public IdentityAttribute( Type identityType )
        {
            var identity = (Identity)(Assembly.GetExecutingAssembly().CreateInstance( identityType.FullName ));

            this.Identifier = identity.Identifier; // Line 16, error here
            this.Name = identity.Name;
            this.Description = identity.Description;
        }

        public string Identifier { get; private set; }
        public string Name { get; private set; }
        public string Description { get; private set; }
    }

Hi, thanks for sharing this issue with the code involved. It's great to be able to see exactly where the failure happened.

When set to use a framework utilisation type of 'DynamicAnalysis' (the default for NUnit2), NCrunch creates a runtime environment and invokes NUnit to discover the tests in your project. NUnit loads your assembly and uses reflection to iterate over the types, identifying which types/methods represent executable tests.

Custom attributes can create a bit of a problem in this context as they contain executable user code that is invoked automatically by the CLR when the attributes are created. This effectively means that NUnit is executing your code while it searches for tests, even though it may not actually be intending to do this. Because attributes can inherit their way across assemblies, it becomes possible for NUnit to execute code within your production assemblies, even though it's only searching through the types in your test assemblies.

There are two potential solutions to this problem:

1. Change the 'Framework utilisation type for NUnit' solution-level NCrunch configuration setting to 'StaticAnalysis'. This will cause NCrunch to discover tests using metadata analysis rather than invoking NUnit, thus allowing it to avoid creating custom attributes during test discovery. Note that NUnit will still eventually need to be able to construct the attributes later when it builds its runtime environment to execute tests, so this may only serve to push the issue downstream.

2. Redesign the code involved. I would suggest making the custom attribute more resilient to partially loaded environments. As long as the attribute can construct without throwing an exception, you shouldn't see any problems here.

I figured out the issue. I was using GetExecutingAssembly, but the type wasn't in that assembly. I switched to using identityType.Assembly and it all works now.

Thanks for the explanation. That makes sense. I initially set NCrunch to StaticAnalysis, then I could test the attribute and figured out the issue, fixed the code, and am back to DynamicAnalysis.