Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Debugging tests that require an external dll
kalebpederson
#1 Posted : Friday, July 6, 2012 5:05:09 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/1/2012(UTC)
Posts: 25
Location: US

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
I've read through the test troubleshooting page but I'm still having problems getting a few (integration-like) tests to work in NCrunch. The tests run fine in the nunit.console test runner as well as the Resharper test runner.

I have the following projects and dependencies:

Code:
MyProject.Interop [c++/cli] --> {MyProject.Contracts [c#], Other.dll}
MyProject.Interop.Test [c#] --> MyProject.Interop
Other.dll is a standard dll loaded using ::LoadLibrary


I'm trying to run a few tests in MyProject.Interop.Test but a few tests still fail.

In order to make the other test runners work, I included Other.dll via a link to ..\..\libs\other.dll and set the file to Copy if Newer. This dll shows up in the NC output directory. Regardless of whether I tell NC to "copy referenced assemblies to workspace," the tests still fail. The MyProject.Interop assembly isn't copied to the output directory in either configuration, so I'm guessing it's trying to load other.dll from the same directory it's located in, which then fails.

I tried using the following line of code:

Code:
var referencedAssemblyPathsInOuterAppDomain = 
    AppDomain.CurrentDomain.GetAssemblies().Select(x => x.Location).ToArray()


But that gives me a "The invoked member is not supported in a dynamic assembly" exception. In manually examining the returned assemblies, I see nothing out of the ordinary (although I don't know exactly what I should be looking for either).

Suggestions?
Remco
#2 Posted : Monday, July 9, 2012 1:53:59 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,143

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for posting!

In some application domains, the GetAssemblies().Select(x => x.Location) code will cause the exception you've described to be thrown. I've updated the documentation with an adjustment to this to avoid referencing System.Reflection.Emit assemblies which often are responsible for the exception. Try the following to see if it works any better:

var referencedAssemblyPathsInOuterAppDomain = AppDomain.CurrentDomain.GetAssemblies().Where(a => a.ManifestModule.GetType().Namespace != "System.Reflection.Emit").Select(x => x.Location).ToArray();

If I correctly understand the relationship between these DLLs you've described, the Test project has an indirect dependency on Other.dll and you are working around this by introducing a relative cross-project dependency in the Test .csproj file. This sort of setup unfortunately can't work well with NCrunch's workspacing as the workspacing treats projects as being independent units that have been extracted from their parent solution (and thus they cannot indirectly rely on each other's build output outside of NCrunch's test environment wiring).

Using the referenced assembly path from the current application domain is by far the best solution to this problem. You may be able to introduce a solution where the output of Other.dll is treated as a statically included file in your Test project. This could be done by using the 'Additional files to include' setting, although the result would not be ideal as changes to Other.dll would not be automatically picked up in your continuous testing environment and would require a manual rebuild of your solution before NCrunch would notice them.
kalebpederson
#3 Posted : Monday, July 9, 2012 5:02:54 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/1/2012(UTC)
Posts: 25
Location: US

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
Remco wrote:
Hi, thanks for posting!


And thanks for responding.

Remco wrote:

In some application domains, the GetAssemblies().Select(x => x.Location) code will cause the exception you've described to be thrown. I've updated the documentation with an adjustment to this to avoid referencing System.Reflection.Emit assemblies which often are responsible for the exception. Try the following to see if it works any better:

var referencedAssemblyPathsInOuterAppDomain = AppDomain.CurrentDomain.GetAssemblies().Where(a => a.ManifestModule.GetType().Namespace != "System.Reflection.Emit").Select(x => x.Location).ToArray();


Yes, this fixes the debugging problem and helped point me in the right direction.

Remco wrote:

If I correctly understand the relationship between these DLLs you've described, the Test project has an indirect dependency on Other.dll and you are working around this by introducing a relative cross-project dependency in the Test .csproj file. This sort of setup unfortunately can't work well with NCrunch's workspacing as the workspacing treats projects as being independent units that have been extracted from their parent solution (and thus they cannot indirectly rely on each other's build output outside of NCrunch's test environment wiring).


Not quite. Other.dll is 3rd party dll that I have placed in my solution's libs directory. I have made NC aware of that dependency by linking to the libs\other.dll file within MyProject.Interop.Test and setting it to copy when newer. When I check the NC output directory for the test project, other.dll is present.

Problem 1: CPU architecture autodetect failure.

With the code changes above, I was able to see all the assembly locations which made me wonder where the test runner was searching for other.dll. I pulled up ProcessMonitor and discovered that the dll was being found and read but not loaded, which made me realize that the platform/architecture must be wrong on the test project (as it's a 64-bit only dll). I had set the Interop project to a CPU architecture of x64, so I assumed it would have been picked up correctly, but it wasn't. So, I hard coded ALL the projects to a CPU Architecture of x64. I then received an error message about my interop assembly failing to load. My interop project, a C++/CLI project and has its configuration hard coded to x64 within the solution as that's the only thing we support.

Problem 2: CPU architecture ignored at compile time

At this point my interop assembly was failing to load with a "Bad image format exception," which suggests it's not being loaded as a 64-bit assembly. I did a NC reset but the tests still failed. I pulled up the generated interop and test assemblies and despite the NC CPU architecture being set to x64, both assemblies are compiled as AnyCPU. As a test, I replaced the NC-generated Interop.dll with the dll from my standard build output and re-ran the NC tests and all the tests passed.

Summary:

NC didn't autodetect the architecture correctly. Except my interop project which is compiled x64, each of my assemblies is compiled AnyCPU. Despite hard-coding the CPU architecture to x64 in NC, all the assemblies output by NC are compiled as AnyCPU which causes my other.dll, which is 64-bit only, to fail to load.

I'm still new enough to .NET that I don't know how the assemblies being compiled as AnyCPU instead of x64 interacts with DLL loading when a 64-bit DLL is required. I would think that it would be fine to compile an assembly as AnyCPU if it were being ran as x64, but perhaps that's not the case. Like I said, replacing the NC-generated AnyCPU compiled Interop dll with my standard x64-compiled interop dll allows the tests to pass.

Thank you again for all your help.

--Kaleb
Remco
#6 Posted : Tuesday, July 10, 2012 6:01:07 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,143

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi Kaleb,

Thanks for the detailed analysis of the issue. Differing CPU architectures have a history of causing problems with test runners, as it can sometimes be difficult to establish what is the correct behaviour in setting up the test environment. The 'Use CPU architecture' configuration option in NCrunch is used to establish the preferred CPU architecture for the test execution environment for AnyCPU test projects, but it does not have any effect on the build process itself. This means that if the build settings for a test project are set to build an x64 assembly, then NCrunch should build an x64 test environment in order to test the assembly (regardless of how the configuration flag is set).

The reasoning behind this is to try to align NCrunch's build process as closely with the Visual Studio build process as possible, and to build to different CPU architectures could create some very strange problems (I guess similar to the one you've described).

Something to watch out for is that NCrunch will only ever use the default build configuration that is specified in the .proj file, and it will ignore the currently selected build configuration in Visual Studio. I wonder if this may have been the source of some of your problems? Normally NCrunch should detect the correct CPU architecture when loading projects.
1 user thanked Remco for this useful post.
kalebpederson on 7/10/2012(UTC)
kalebpederson
#7 Posted : Tuesday, July 10, 2012 10:17:15 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/1/2012(UTC)
Posts: 25
Location: US

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
Remco;2409 wrote:
Hi Kaleb,
...
Something to watch out for is that NCrunch will only ever use the default build configuration that is specified in the .proj file, and it will ignore the currently selected build configuration in Visual Studio. I wonder if this may have been the source of some of your problems? Normally NCrunch should detect the correct CPU architecture when loading projects.


You're right! I did some digging and it turns out that my interop project defaulted to building an AnyCPU assembly which was causing problems. I still have to manually set the CPU architecture to x64 on the two test projects that depend on the interop project, but now everything works beautifully again.

Thank you for all the help.

--Kaleb
1 user thanked kalebpederson for this useful post.
Remco on 7/11/2012(UTC)
Users browsing this topic
Guest (2)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.070 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download