I have a dotnet core solution targeting .Net46 and just moved to vs2017, mostly to get ncrunch working. I have 97 xunit tests and the VS test runner runs them fine. For 25 of them, ncrunch (3.6.0.2) gives me
System.BadImageFormatException: Could not load file or assembly 'file:///C:\Users\lala\.nuget\packages\system.net.http\4.3.0\ref\net46\System.Net.Http.dll' or one of its dependencies. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflection-only loader context. (Exception from HRESULT: 0x80131058)
or
System.BadImageFormatException: Could not load file or assembly 'file:///C:\Users\lala\.nuget\packages\system.runtime.interopservices.runtimeinformation\4.3.0\ref\netstandard1.1\System.Runtime.InteropServices.RuntimeInformation.dll' or one of its dependencies. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflection-only loader context. (Exception from HRESULT: 0x80131058)
at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
at Microsoft.AspNetCore.Hosting.WebHostBuilder.Build()
This latter one seems to be from using TestHost for self-hosted testing of REST interfaces.
This exception is thrown by the .NET Core CLR when an attempt is made to load an assembly reference directly from the Nuget packages directory. Reference assemblies should normally be loaded automatically by the CLR using the NCrunch.TaskRunner.DotNetCore.deps.json file generated by NCrunch in the build output directory of your test project.
Are you able to share any more details about how these Nuget packages are referenced from your test project? Do you see any reference to them in the .deps file?
I see references to both, many more to the RuntimeInformation than to the Http one. I dont mind uploading the file somewhere. Do you have a favorite place to do that? It's 185kb.
EDIT: Sorry, I did not read your comment properly. I do not have such a file (NCrunch.TaskRunner.DotNetCore.deps.json) in the ncrunch test output dir.
I do not reference these dlls explicitly from my test project.
Sorry, let me confirm the exact name of the file. It should normally be nCrunch.TaskRunner.DotNetCore.x64.deps.json. This file will exist in the build output directory of the workspace used to execute your tests. You should be able to find it by right-clicking on the test in the Tests Window, then choosing Advanced->Browse to workspace.
Thinking that not all of your tests are failing, can you notice any pattern around which tests seem to fail vs the ones that pass? I'm wondering if there might be some test code here that is deliberately trying to load these assemblies and is failing because of it. Is this problem being reported as a test failure, or an internal problem within NCrunch?
Sorry, let me confirm the exact name of the file. It should normally be nCrunch.TaskRunner.DotNetCore.x64.deps.json. This file will exist in the build output directory of the workspace used to execute your tests. You should be able to find it by right-clicking on the test in the Tests Window, then choosing Advanced->Browse to workspace.
Yes. I do not have such a file there. This is the dir /b output from one such folder:
I have a .deps.json file in my 'normal' test bin folder.
Remco wrote:Thinking that not all of your tests are failing, can you notice any pattern around which tests seem to fail vs the ones that pass? I'm wondering if there might be some test code here that is deliberately trying to load these assemblies and is failing because of it. Is this problem being reported as a test failure, or an internal problem within NCrunch?
The System.Runtime.InteropServices.RuntimeInformation ones are all in the self-hosted tests that uses TestHost/Kestrel. And all of those tests fail. The System.Net.Http.dll ones are more or less w/o rhyme or reason. These are mocked tests (moq) and hence quite self-contained.
I do not have any sneaky code that loads assemblies explicitly.
It is being reported as a test failure, I believe. I see no indication of internal error at least.
Rasmus wrote:
The System.Runtime.InteropServices.RuntimeInformation ones are all in the self-hosted tests that uses TestHost/Kestrel. And all of those tests fail. The System.Net.Http.dll ones are more or less w/o rhyme or reason. These are mocked tests (moq) and hence quite self-contained.
By self-hosted, do you mean that the code that is exploding is running in its own process that is not hosted by NCrunch? If so, this could explain the behaviour you're seeing. In its first releases for .NET Core, NCrunch had a number of problems with dependency resolution that were very similar to what you've described. However, these problems always appeared outside of the test execution, as they were related to building the environment around the test. If the resolution issues are appearing inside code being run by the tests themselves, then this is technically a problem that is downstream from NCrunch.
Can you give me any more detail on how these tests are hosting their own code? Is this being done by an MS library or framework of some kind?
Remco wrote:
By self-hosted, do you mean that the code that is exploding is running in its own process that is not hosted by NCrunch? If so, this could explain the behaviour you're seeing. In its first releases for .NET Core, NCrunch had a number of problems with dependency resolution that were very similar to what you've described. However, these problems always appeared outside of the test execution, as they were related to building the environment around the test. If the resolution issues are appearing inside code being run by the tests themselves, then this is technically a problem that is downstream from NCrunch.
Can you give me any more detail on how these tests are hosting their own code? Is this being done by an MS library or framework of some kind?
Yes, you are quite right. Sorry for being unclear here. Per test run, we start a WebHost like this
var hostBuilder = new WebHostBuilder()
.UseEnvironment("Development")
.UseKestrel()
.UseUrls("(- BROKEN LINK -)")
.UseIISIntegration()
.UseStartup<Startup>();
var webHost = hostBuilder.Build();
webHost.Start();
where webhostbuilder is from microsoft.aspnetcore.hosting\1.1.1\lib\net451\Microsoft.AspNetCore.Hosting.dll
Ok, we're now stepping outside the boundary within which I feel I can be of much help here. Under .NET Core, it's the responsibility of the host process to declare the dependencies it requires and to load them in a way that complies with the 'rules' (which are to my knowledge completely undocumented at this point in time). Because this host process isn't owned by NCrunch, I have no idea how to analyse this problem, or even whether it is actually possible to solve it.
As you've mentioned that the problem appears when NCrunch runs the test but not under VSTest, there must therefore be something in the environment of the test runner itself that is being handled differently. It's possible that there may be some special magic that VSTest does in advance that somehow makes the webhost work. The only thing that the moment I could suggest trying would be to turn on compatibility mode to see if there is an NCrunch feature that is clashing with something here.
If you have a self contained code sample you can share that can reproduce this problem, I'd be happy to look at it in more detail to see if I can discover the difference between VSTest and NCrunch.
I just had an idea. Could you check whether your test project has a direct dependency on the two packages reported by the exceptions? If not, does adding the dependency directly resolve the problem?
Remco wrote:I just had an idea. Could you check whether your test project has a direct dependency on the two packages reported by the exceptions? If not, does adding the dependency directly resolve the problem?
There is two projects in play, one without self-hostedness (classic unit test with moq) and one with (more integration-test'ish). We have mostly been talking about the latter but I tried this for both. It did not help for any of them. Both adds (System.Runtime.InteropServices.RuntimeInformation nuget for the integration test and a direct ref to System.Net.Http for the unit test) resulted in a yellow triangle on the ref and no difference for NCrunch. VS is not forthcoming about its issue with the references.
I have also tried to reproduce the self-hosting issue in a new project but is unable to :/ NC is quite happy so far, with both self-hosting, swagger and other things we stuff into the build pipeline. I pursue these more when I get some more me-time. In the meanwhile I am open for suggestions.
I also tried toggling the various compat-flags without luck.
I will pursue these more when I get some more me-time.
Ok, looked at the unit-test error. I could not reproduce in my test solution :/
But I establisted that the issue is with the following code snippet:
client.Setup(
c =>
c.GetXXXAsync(
It.IsAny<int>(),
It.IsAny<bool>(),
It.IsAny<bool>(),
It.IsAny<string>(),
It.IsAny<Dictionary<string, List<string>>>(),
It.IsAny<CancellationToken>()))
.Throws(
new ValidationException(
new Validation.ValidationErrors()
{
Errors =
{
new Validation.ValidationError()
{
Code = "100",
Member = "TestMe",
Reason = "Unit tests"
}
}
},
new HttpRequestMessageWrapper(new HttpRequestMessage(), null),
new HttpResponseMessageWrapper(new HttpResponseMessage(HttpStatusCode.BadRequest), null)));
where 'client' is a Moq of a AutoRest-generated swagger client (and the rest is fairly vanilla stuff). If I comment out the two HttpR*Wrapper at the end (swap with null), NCrunch will run the test. These two classes come from Microsoft.Rest.ClientRuntime, which is directly referenced by the test project.
This is reminicent of issues I had just after converting from VS2015/project.json, which was fixed by making the test project a <Project Sdk="Microsoft.NET.Sdk.Web"> instead of a <Project Sdk="Microsoft.NET.Sdk">. If I remember correctly, we had a couple of days of issues converting..
Have the same problem with Version 4.3.1 of System.Net.Http, i believed it work with NCrunch 3.5, does not work with 3.6 but i'm not sure.
There is a strange behaviour. A dotnet restore, restores 4.1.0, 4.3 and 4.3.1 nuget packages while restoring, in the application is of course only 4.3.1 in the dependencies.
ASP.NET Core 1.1.1, and .NET 46.
ReSharper 2017.1 EAP 5 runs the tests, dotnet test/watch test also works.
Can you confirm whether this is appearing for you with self-hosting tests or custom application domains? I'm trying to establish if this is happening the same way for you as it is for Rasmus, or if it is a similar looking issue with a different cause.
I hope that i can provide a simple repro for that problem. I've identified some thing in the *.deps.json, if i use the right Dependencies it should be reproducable. But this will take 1 or 2 days until i can take a spin on that.
Der-Albert.com wrote:I hope that i can provide a simple repro for that problem. I've identified some thing in the *.deps.json, if i use the right Dependencies it should be reproducable. But this will take 1 or 2 days until i can take a spin on that.
Thanks, this does definitely look like a .deps related issue. v3.6 introduced a key change in how this file is constructed, so there must be something in your build system's .deps generation that NCrunch isn't aligned with. If you can prepare for me a test case, I am fairly certain that I can solve this problem.