Different test results with ncrunch and NUnit tool

I am testing this method:

public static Base<T> Create(string typeName, string assemblyName) where T : class
        {
            string name = string.Format(
                CultureInfo.InvariantCulture, "{0}`1[[{1}]], {2}", typeName, typeof(T), assemblyName);
            Type type = Type.GetType(name, true);

            return new Derived<T>((BaseImpl<T>)type.GetProperty("Instance").GetValue(null, null));
        }

This test succeeds with NUnit but fails with NCrunch:

[Test]
        [ExpectedException(typeof(FileNotFoundException))]
        public void Create_throws_FileNotFoundException_if_assembly_can_not_be_found()
        {
            using (Factory.Create<string>(this.typeName, "This.File.Does.Not.Exist"))
            {
            }
        }

Ncrunch expects a FileLoadException but the assembly does not exist??

Hi Mrt181,

This is caused by a difference in assembly resolution logic between NCrunch and NUnit.

NCrunch throws a different exception with extra information about the paths checked for the missing assembly. This was implemented for diagnostics, as assembly resolution issues can be horrifically hard to troubleshoot without it.

You can work around this by generalising your exception handling a bit, i.e:

[Test]
public void Create_throws_FileNotFoundException_if_assembly_can_not_be_found()
{
	try
	{
		Create<string>(this.GetType().Name, "This.File.Does.Not.Exist");
	}
	catch (Exception ex)
	{
		Assert.That(ex is FileLoadException || ex is FileNotFoundException);
		return;
	}

	Assert.Fail();
}

I hope this helps.


Cheers,

Remco

Ok, that works but feels like a hack.
Shouldn't it behave like nunit, no matter if this is worse or better?
Shouldn't this diagnostic behavior be optional?

Agreed - it's not pretty.

There are some strong arguments in favour of aligning this with NUnit. Likely there will be some adjustments made around this in a future release.

Thanks for reporting it :)

Arguably, you should use the `Assert.That(() => { }, Throws.InstanceOf<>());` method, anyways.

You should be able to write your own constraint that would support subclasses, too.

For anyone interested, this issue should be fixed in the recently released 1.33b.