Unable to run tests with .NET Core 2.0 referencing .NET Standard 2.0

I have a simple solution with a .NET Standard 2.0 (RTM) project and a .NET Core 2.0 (RTM) project that depends on the .NET Standard project. The solution builds fine in Visual Studio, but NCrunch simply shows a blue question-mark next to the test project. It also fails to build the .NET Standard 2.0 project, but that isn't a problem for me because it doesn't contain tests.

When I remove the dependency from the test project to the .NET Standard project and comment out the test body, the project builds in NCrunch and the tests run.

Is .NET Standard 2.0 / .NET Core 2.0 compatibility coming soon?

UPDATE: this problem also occurs when referencing a .NET Standard 2.0 project from a .NET Framework 4.7 project. I'm running NCrunch 3.10.0.20.

Hi, thanks for sharing this problem.

Handling of .NET Core/Standard 2.0 is hit and miss at the moment. The reason for this is because every time NCrunch gets updated to handle this framework, MS push out a new release and it all gets broken again. Although .NET Core 2.0 is now announced as 'RTM' by Microsoft, NCrunch is still totally reliant on the MS tool stack that supports it, and this tool stack isn't expected to be released until mid September.

I appreciate you reporting a use case here that doesn't work correctly, I'll make sure I have a chance to review it before the next NCrunch update.

If it helps, I have a solution at https://github.com/DavidArno/Standard.Abstractions that has such problems (though may not be directly related to the OP's issue).

Using VS2017 Update 3, netstandard 2.0 and .NET Core 2.0 RTW's (all updated today), with NCrunch v3.10, NCrunch gives the following error when it attempts to build:

System.ArgumentNullException: Value cannot be null.
Parameter name: type
at _Mono.Cecil.ModuleDefinition.Import(TypeReference type)
at nCrunch.Compiler.StaticManipulation.StaticInstrumentationImportMethodSet..ctor(CustomCecilAssemblyResolver resolver, ModuleDefinition module)
at nCrunch.Core.PlatformTypes.DotNetCorePlatformType.GetInstrumentationMethodsForFramework(CustomCecilAssemblyResolver resolver, AssemblyDefinition assembly)
at nCrunch.Compiler.StaticManipulation.BuiltAssembly.Instrument(ComponentSourceLineMap sourceLineMap, IDictionary`2 instrumentationDirectivesByCodeFileId, IDictionary`2 codeFileIDsByFilePath, Boolean proxyProcessIsActive, Int32 componentId, IInstrumentationFilter[] instrumentationFilters, IBuildPlatformType platformType, PerfTracker perfTracker)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentBuildParameters , FilePath , BuildOutput , DirectoryPath[] , FilePath[] )
at nCrunch.Compiler.RemoteBuildRunner..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentBuildParameters )

Apologies though if this is an unrelated issue.

Yes, I have the same error, too.

DavidArnoDandG wrote:If it helps, I have a solution at https://github.com/DavidArno/Standard.Abstractions that has such problems (though may not be directly related to the OP's issue).

Using VS2017 Update 3, netstandard 2.0 and .NET Core 2.0 RTW's (all updated today), with NCrunch v3.10, NCrunch gives the following error when it attempts to build:

System.ArgumentNullException: Value cannot be null.
Parameter name: type
at _Mono.Cecil.ModuleDefinition.Import(TypeReference type)
at nCrunch.Compiler.StaticManipulation.StaticInstrumentationImportMethodSet..ctor(CustomCecilAssemblyResolver resolver, ModuleDefinition module)
at nCrunch.Core.PlatformTypes.DotNetCorePlatformType.GetInstrumentationMethodsForFramework(CustomCecilAssemblyResolver resolver, AssemblyDefinition assembly)
at nCrunch.Compiler.StaticManipulation.BuiltAssembly.Instrument(ComponentSourceLineMap sourceLineMap, IDictionary`2 instrumentationDirectivesByCodeFileId, IDictionary`2 codeFileIDsByFilePath, Boolean proxyProcessIsActive, Int32 componentId, IInstrumentationFilter[] instrumentationFilters, IBuildPlatformType platformType, PerfTracker perfTracker)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentBuildParameters , FilePath , BuildOutput , DirectoryPath[] , FilePath[] )
at nCrunch.Compiler.RemoteBuildRunner..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentBuildParameters )

Apologies though if this is an unrelated issue.

Same here.

I've prepared a build with a fix targeted to resolve this issue. Download links are below:

http://downloads.ncrunch.net/NCrunch_Console_3.11.0.4.msi
http://downloads.ncrunch.net/NCrunch_Console_3.11.0.4.zip
http://downloads.ncrunch.net/NCrunch_GridNodeServer_3.11.0.4.msi
http://downloads.ncrunch.net/NCrunch_GridNodeServer_3.11.0.4.zip
http://downloads.ncrunch.net/NCrunch_LicenseServer_3.11.0.4.zip
http://downloads.ncrunch.net/NCrunch_VS2008_3.11.0.4.msi
http://downloads.ncrunch.net/NCrunch_VS2010_3.11.0.4.msi
http://downloads.ncrunch.net/NCrunch_VS2010_3.11.0.4.zip
http://downloads.ncrunch.net/NCrunch_VS2012_3.11.0.4.msi
http://downloads.ncrunch.net/NCrunch_VS2012_3.11.0.4.zip
http://downloads.ncrunch.net/NCrunch_VS2013_3.11.0.4.msi
http://downloads.ncrunch.net/NCrunch_VS2013_3.11.0.4.zip
http://downloads.ncrunch.net/NCrunch_VS2015_3.11.0.4.msi
http://downloads.ncrunch.net/NCrunch_VS2015_3.11.0.4.zip
http://downloads.ncrunch.net/NCrunch_VS2017_3.11.0.4.msi
http://downloads.ncrunch.net/NCrunch_VS2017_3.11.0.4.zip

Hi Remco,

I have installed http://downloads.ncrunch.net/NCrunch_VS2017_3.11.0.4.msi and the net standard project now compiles just fine. Thanks.

However, the tests now fail with the error:

System.BadImageFormatException: Could not load file or assembly 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflection-only loader context. (Exception from HRESULT: 0x80131058)
File name: 'netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' ---> System.BadImageFormatException: Could not load file or assembly 'file:///C:\Program Files\dotnet\sdk\NuGetFallbackFolder\netstandard.library\2.0.0\build\netstandard2.0\ref\netstandard.dll' or one of its dependencies. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflection-only loader context. (Exception from HRESULT: 0x80131058)
File name: 'file:///C:\Program Files\dotnet\sdk\NuGetFallbackFolder\netstandard.library\2.0.0\build\netstandard2.0\ref\netstandard.dll' ---> System.BadImageFormatException: Cannot load a reference assembly for execution.
at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadFrom(String assemblyFile, Evidence securityEvidence, Byte[] hashValue, AssemblyHashAlgorithm hashAlgorithm, Boolean forIntrospection, Boolean suppressSecurityChecks, StackCrawlMark& stackMark)
at System.Reflection.Assembly.LoadFrom(String assemblyFile)
at nCrunch.Common.CustomAssemblyResolver.ResolveAssembly(ParsedAssemblyName parsedAssemblyName)
at nCrunch.Common.CustomAssemblyResolver.(Object , ResolveEventArgs )
at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)


at Standard.Abstractions.Tests.UnitTest1.Directory_CanBeCreatedAndCorrectlyShowsItExists()

I've tried targeting netcoreapp2.0, net461 and net47. I've tried adding a reference to NETStandard.Library 2.0.0 nuget package. None of these makes a difference though.

Am I doing something wrong. or is there still an issue?

Hi David,

If I were to guess, this looks to be the result of confusion in the test environment between the netstandard compile-time assembly and run-time assembly. Probably this is the result of another late-game change to the .NET Core toolset.

Unfortunately I'm not sure how to produce this. It doesn't look to be specific to a default netstandard project, so I guess there must be another variable here. Is there any chance you could share with me a self-contained code sample that I can use to reproduce the problem? You're welcome to submit any code through the contact form.

Hi Remco,

The code that exhibits this behaviour can be git pulled from https://github.com/DavidArno/Standard.Abstractions. But if it's easier for you if I send it directly, please let me know.

David.

Found a partial workaround: as long as the assembly with the actual tests target .net framework (net461 in my case), and the csproj file contains a <DependsOnNETStandard>true</DependsOnNETStandard> tag, I could get it to run code from a NetStardard2.0 project.

Thanks EsbjornRedmo, that fixes the issue for me.

I've prepared a new build with a fix for the above netstandard.dll problem. This build also has a couple of fixes around .NET Core usage on grid nodes.

http://downloads.ncrunch.net/NCrunch_Console_3.11.0.5.msi
http://downloads.ncrunch.net/NCrunch_Console_3.11.0.5.zip
http://downloads.ncrunch.net/NCrunch_GridNodeServer_3.11.0.5.msi
http://downloads.ncrunch.net/NCrunch_GridNodeServer_3.11.0.5.zip
http://downloads.ncrunch.net/NCrunch_LicenseServer_3.11.0.5.zip
http://downloads.ncrunch.net/NCrunch_VS2008_3.11.0.5.msi
http://downloads.ncrunch.net/NCrunch_VS2010_3.11.0.5.msi
http://downloads.ncrunch.net/NCrunch_VS2010_3.11.0.5.zip
http://downloads.ncrunch.net/NCrunch_VS2012_3.11.0.5.msi
http://downloads.ncrunch.net/NCrunch_VS2012_3.11.0.5.zip
http://downloads.ncrunch.net/NCrunch_VS2013_3.11.0.5.msi
http://downloads.ncrunch.net/NCrunch_VS2013_3.11.0.5.zip
http://downloads.ncrunch.net/NCrunch_VS2015_3.11.0.5.msi
http://downloads.ncrunch.net/NCrunch_VS2015_3.11.0.5.zip
http://downloads.ncrunch.net/NCrunch_VS2017_3.11.0.5.msi
http://downloads.ncrunch.net/NCrunch_VS2017_3.11.0.5.zip

Still seem to be having a problem, unfortunately. Tried setting logging level to verbose but still only get the message below. This comes from Visual Studio 2017 Community version 15.3.1 using NCrunch 3.11.0.5 linked to above.

An error occurred while analysing this project after it was built: NCrunch encountered an unexpected error occurred while building an environment to analyse an assembly: nCrunch.TaskRunner.Ipc.IpcConnectionClosedException: The connection has been closed
at nCrunch.Core.ProcessManagement.ExternalProcessManager.(ProcessorArchitecture , ProcessLoadParameters )
at nCrunch.Core.ProcessManagement.ExternalProcessManager.LoadExternalProcess(ProcessLoadParameters parameters, GridClientId client)
at nCrunch.Core.TestManagement.TestRunnerProcess..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.TestManagement.TestRunnerProcess.(Nullable`1 , FilePath , GridClientId , CustomEnvironmentVariable[] )
at nCrunch.Core.TestManagement.TestRunnerProcess..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.Processing.AnalysisTaskLogic.DoProcessTaskAndReturnSuccessFlag()

Not sure if this helps but I'm also using Resharper Ultimate as well as OzCode.

sextondjc wrote:Still seem to be having a problem, unfortunately. Tried setting logging level to verbose but still only get the message below. This comes from Visual Studio 2017 Community version 15.3.1 using NCrunch 3.11.0.5 linked to above.

An error occurred while analysing this project after it was built: NCrunch encountered an unexpected error occurred while building an environment to analyse an assembly: nCrunch.TaskRunner.Ipc.IpcConnectionClosedException: The connection has been closed
at nCrunch.Core.ProcessManagement.ExternalProcessManager.(ProcessorArchitecture , ProcessLoadParameters )
at nCrunch.Core.ProcessManagement.ExternalProcessManager.LoadExternalProcess(ProcessLoadParameters parameters, GridClientId client)
at nCrunch.Core.TestManagement.TestRunnerProcess..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.TestManagement.TestRunnerProcess.(Nullable`1 , FilePath , GridClientId , CustomEnvironmentVariable[] )
at nCrunch.Core.TestManagement.TestRunnerProcess..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.Processing.AnalysisTaskLogic.DoProcessTaskAndReturnSuccessFlag()

This looks to be a different problem. This exception is thrown when the .NET Core test process can't initialise and automatically terminates.

Where I've seen this in the past, it's been because of missing dependencies. It's possible there's something in your environment that NCrunch isn't accounting for.

Could you submit a bug report after you've seen this? The log file may yield something useful.

Remco wrote:
This looks to be a different problem. This exception is thrown when the .NET Core test process can't initialise and automatically terminates.

Where I've seen this in the past, it's been because of missing dependencies. It's possible there's something in your environment that NCrunch isn't accounting for.

Could you submit a bug report after you've seen this? The log file may yield something useful.

Done. Can't see any missing dependencies but seems plausible as R# is also complaining about an unresolved reference (which it shouldn't).

I may well be doing something stupid, but can't see what that might be.

Thanks for sending through the log. This contained an interesting exception:

System.Exception: Could not resolve assembly: System.Threading.Thread, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3aSystem.IO.FileNotFoundException: Could not load file or assembly 'System.Threading.Thread, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
File name: 'System.Threading.Thread, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
at nCrunch.TaskRunner.Ipc.Fast.FastIpcServer..ctor(SerializationContract contract, String ipcName, Boolean stringCachingAllowed)
at Program.startTaskRunner(Int32 ownerProcessId, String callbackUri, String externalProcessId, FilePath taskAssemblyPath, String taskComponentFullName, DirectoryPath appDomainBaseDirectory, Boolean stringCachingAllowed)
at Program.Initialise(String[] args)
at Program.Main(String[] args)

NCrunch should normally be trying to restore its dependencies earlier in the session. Has it given you any other warnings or errors worthy of note?

Simply creating a new project and adding the System.Threading.Thread v4.1 Nuget package should make VS restore this and place it in your Nuget local storage directory. NCrunch should then work for you, though it is still a mystery to me why NCrunch couldn't handle this earlier in the process.

Remco wrote:
NCrunch should normally be trying to restore its dependencies earlier in the session. Has it given you any other warnings or errors worthy of note?

Simply creating a new project and adding the System.Threading.Thread v4.1 Nuget package should make VS restore this and place it in your Nuget local storage directory. NCrunch should then work for you, though it is still a mystery to me why NCrunch couldn't handle this earlier in the process.

No joy. Tried that, same result. I think that exception was actually from ReSharper (2017.1.3, have since moved to EAP version as recommended here).

Does appear to be an NCrunch problem. I created a small test solution to try out against a couple of different test frameworks and runners. Observations below.

xUnit (fails)
• Build fails immediately as soon as the xUnit nuget package is reference.
• NCrunch reports the build as failed with the IpcConnectionClosedException pasted above. This is before writing a single line of code.
• I've tried this both with using the xUnitProject template as well as with a blank Core assembly and then adding xUnit via nuget.
• The test assembly is actually built and available in the test folder (AppData\Local\NCrunch\....).
• Needs Microsoft.NET.Test.Sdk to show up in Visual Studio Test Explorer.
• Test runs successfully with Visual Studio Test Explorer.
• Test runs successfully with ReSharper Dot Cover.

MsTest (fails)
• Builds successfully but fails the test with same IpcConnectionClosedException before actually executing the test.
• Needs Microsoft.NET.Test.Sdk to show up in Visual Studio Test Explorer.
• Test runs successfully with Visual Studio Test Explorer.
• Test runs successfully with ReSharper Dot Cover.

NUnit (succeeds)
• Seems to be unaffected, provided the NUnit3TestAdapter v3.8.0 package is installed.
• Needs Microsoft.NET.Test.Sdk to show up in Visual Studio Test Explorer.
• Test runs successfully with Visual Studio Test Explorer.
• Test runs successfully with ReSharper Dot Cover.

I've copied the test solution along with verbose NCrunch Diagnostic Output to here.

Hope that helps.

I don't seem to be seeing the same thing as you are for these scenarios, so probably this problem is related to your system installstate or environment. Could you check which versions of the System.Threading.Thread library you have installed in your user's Nuget package storage directory? (i.e. C:\Users\[USER]\.nuget\packages\System.Threading.Thread). I think this is being caused by changes to .NET Core's assembly resolution system.

Remco wrote:I don't seem to be seeing the same thing as you are for these scenarios, so probably this problem is related to your system installstate or environment. Could you check which versions of the System.Threading.Thread library you have installed in your user's Nuget package storage directory? (i.e. C:\Users\[USER]\.nuget\packages\System.Threading.Thread). I think this is being caused by changes to .NET Core's assembly resolution system.

I have 4.0.0 and 4.3.0. Same ones as on nuget.

Have reinstalled core SDK. Issue remains. Reckon I'll just re-image my machine and try again. Maybe an unrelated dodgy install broke something somewhere. Will start from scratch and try again.

sextondjc wrote:
I have 4.0.0 and 4.3.0. Same ones as on nuget.

Have reinstalled core SDK. Issue remains. Reckon I'll just re-image my machine and try again. Maybe an unrelated dodgy install broke something somewhere. Will start from scratch and try again.

I'm getting reports of a whole range of assembly resolution issues related to changes in the latest version of .NET Core that was released a week or so ago. I'm going to have a dig around to see if I can reproduce any of these to provide closer alignment. I'll update you as soon as I have a build available.