When running NCrunch using test-files on MSTest, the outcome is not consistent.
Almost always upon loading the solution, or starting NCrunch the unit tests that require deployed files fail.
Upon automated execution when a code change has been detected, they are successful.

I have applied deploymentitem attributes at class level as:


It seems to have to do with the location of the directory the testrunner uses, which is varying (maybe dependent on the last build performed, being tracked by NCrunch?)

It switches between (success):
and (failure):

Some cases that some reproducible:


Within the failing tests' folder: [NAMESPACE OF MY UNIT TEST LIBRARY]\bin the deployed files are actually present at the compiler output folder but are somehow not copied to the 'Out' folder.

• Debug
  
  • testfiles (containing testfiles)
  
  
• In
  
  • [name of my machine] (empty)
  
  
• Out (empty)

Any suggestions?
The 'deployment' option is switched on in my .testsettings file, but that should not matter, as NCrunch state it ignores these file altogether.

I'm working with MSTest of VS2012, update 2
(located at: %ProgramFiles(x86)%\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe)

I also have Resharper installed, which installs and runs NUnit I believe. When disabling Resharper, the issue remains.

First I thought that maybe Directory.GetCurrentDirectory was the culprit, so I removed it.
Now it seems that the behaviour of Directory.GetCurrentDirectory is consistent with previous mentioned issue:

When failing it states:
[MyAppData]\Local\NCrunch\[SOMENUMBER]\[SOMENUMBER]\[NAMESPACE OF MY UNIT TEST LIBRARY]\bin\Out

When succeeding it states:
[MyAppData]\Local\NCrunch\[SOMENUMBER]\[SOMENUMBER]\TestResults\[SOMEGUID]\Out

Let me see if I can make a copy of my project and strip it

Thanks! I'll take a look and see if I can figure it out. Perhaps if I shake it in the right way, I can reproduce the issue :)

Cheers,

Remco

I should mention that the files are not referenced in my solution (but are stored in tfs in a directory) and I am using a post build task to copy the files to my build directory.

Yes, you are right. Sorry, didn't see that in the documentation.

Thanks,

Matt

Hi Bob,

DeploymentItems are a bit of a complex mess with MSTest, as their behaviour tends to be slightly different depending upon the version of MSTest being used and how this is configured. Because NCrunch doesn't rely on testsettings files, it must infer the correct usage of this attribute and how to apply it during the test run. For this reason, I usually try to advise people avoid them and to instead reference the files relatively as they exist in the project in their pre-copied location.

Considering the behaviour around the copying seems inconsistent, I'm wondering if it may be possible that the DeploymentItems are making your tests sequence dependent. This can happen if you have one test without a deployment item (or an incorrectly specified deployment item) that is relying on the same file being copied using the deployment item of another test. This means the former test cannot run without failure until the latter test has been executed.

Something that can be helpful is to examine the full state of the workspace NCrunch has constructed for your project after it has been built, then again after tests have been executed. You can do this by right-clicking on the project file (or any tests within it), then choosing Advanced->Browse to workspace. This will let you easily examine which files exist in the workspace and which aren't there. This can also be a way to identify files that haven't been copied into the workspace by NCrunch and need to be added to the 'Additional files to include' setting.