Hi,

Thanks for sharing this issue.

Is it possible that the resources you are trying to access are in a different project to the one containing your tests? If so, this would be a cross-project dependency that will require some special handling on your part to get working with NCrunch. See Assembly Colocation Assumptions for more information on this sort of structure.

When you talk about the sandbox the UI is reporting, how are you obtaining this information? Is this by right clicking on the test/assembly in the Tests Window and going to Advanced->Browse to workspace?

Thanks for sharing these extra details. Understanding that the relative path to the TestFilesFolder is different under NCrunch than it is otherwise, I understand that there are two possibilities here:

1. The current directory used by NCrunch is not the same as you expect it to be relative to the project (.proj) file.
2. Or, the TestFilesFolder is not being placed by NCrunch in the right place relative to the project (.proj) file.

Which one of these two is wrong? It should be possible to establish this by examining the NCrunch workspace and comparing this with your project's directory.

Are you using the 'Run impacted tests automatically, others manually' engine mode? If so, the misleading directory path you've described would make sense. This is NCrunch reporting the results from the last time the test was executed, perhaps from a previous session. Because your source code wasn't changed and the test wasn't impacted, it wasn't rerun by NCrunch using any of the new paths.

Can you confirm which test framework you are using? If you are using MSTest, are you using the DeploymentItem attribute?

Usually, when NCrunch sets up to run your tests, it will set both the current directory and the application domain base directory to be the same directory that the test assembly was placed by the compiler (typically bin\debug, but this can be different depending on your compile settings).

The current directory of the application of course isn't immutable - if there is code anywhere executed by NCrunch that changes this directory, downstream tests may be affected. Do you experience this problem if you run the test in isolation? Or is the problem inconsistent?

Can you confirm that you are using the current directory to find your files and not another method? (i.e. the app domain base directory or the location of an assembly in the application domain).

This is definitely a sequence dependent issue then. Either the test code or the code under test is changing the current directory during a run. Because there are tests later in the execution pipeline that depend on the current directory being set consistently, they fail intermittently depending on which tests have been previously executed before them in the test process. See sequence dependent tests for more information on this kind of failure.

Under NCrunch there are many variables that affect the sequence in which tests are executed. NUnit V2 also has a different internal architecture to NUnit V3, so is likely that this problem will not present itself consistently between different runners or versions of NUnit.

I recommend performing a search of your codebase for anything making a call to Directory.SetCurrentDirectory. Note that the call may not actually be in your code at all (for example, you may be calling into compiled 3rd party code or a Win32 API that changes the current directory).

Creating an NUnit SetUpFixture with a teardown section that throws an exception if the current directory has been changed is one way to find the test causing the problem if you execute the tests manually independently or in groups.