Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

DirectoryNotFoundException
ericschmidt
#1 Posted : Friday, December 30, 2016 12:43:07 AM(UTC)
Rank: Member

Groups: Registered
Joined: 12/30/2016(UTC)
Posts: 12
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
I've recently started having an issue with NCrunch with tests that try to access files for testing. There are a couple of issues I'm running into.

First, the sandbox directory that the UI is reporting doesn't match up with the actual directory in the %LocalAppData%\NCrunch.

Secondly, I'm using "..\..\" to have the path relative to the directory that the actual test files are stored in. It should be going up two levels, but for some reason, NCrunch is automatically skipping over the project directory. I am using Visual Studio 2015 along with the NUnit and RhinoMocks framework.
Remco
#2 Posted : Friday, December 30, 2016 1:01:43 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 4,717

Thanks: 622 times
Was thanked: 725 time(s) in 690 post(s)
Hi,

Thanks for sharing this issue.

Is it possible that the resources you are trying to access are in a different project to the one containing your tests? If so, this would be a cross-project dependency that will require some special handling on your part to get working with NCrunch. See Assembly Colocation Assumptions for more information on this sort of structure.

When you talk about the sandbox the UI is reporting, how are you obtaining this information? Is this by right clicking on the test/assembly in the Tests Window and going to Advanced->Browse to workspace?
ericschmidt
#3 Posted : Tuesday, January 03, 2017 6:09:12 PM(UTC)
Rank: Member

Groups: Registered
Joined: 12/30/2016(UTC)
Posts: 12
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
In this instance, I am not trying to access files from a separate project. These are sample files contained within a subfolder of the TestProject itself (so I have DataStoreTests\TestFilesFolder). I've gone in and modified the configuration for this project to include the directory and all files, so in the sandbox, I have verified that %LocalAppData%\NCrunch\#####\#\DataStoreTests\TestFilesFolder exists, and exists two levels up from where the actual DLL being tested is. For some reason, when debugging the path, I've found that, instead of ending up in the DataStoreTests folder, it is ending up one level higher. Thus it cannot find the directory because it's not looking in the right place.

In referring to the Sandbox, I am looking at the failed test exception message. Currently, I am receiving a message "System.IO.DirectoryNotFoundException : Could not find a part of the path 'C:\Users\schmidte\AppData\Local\NCrunch\19696\18\TestProjects\ComponentTestFiles\'.", but in my %LocalAppData%\NCrunch, the only sandbox directory is 3760.
Remco
#4 Posted : Tuesday, January 03, 2017 10:31:34 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 4,717

Thanks: 622 times
Was thanked: 725 time(s) in 690 post(s)
Thanks for sharing these extra details. Understanding that the relative path to the TestFilesFolder is different under NCrunch than it is otherwise, I understand that there are two possibilities here:

1. The current directory used by NCrunch is not the same as you expect it to be relative to the project (.proj) file.
2. Or, the TestFilesFolder is not being placed by NCrunch in the right place relative to the project (.proj) file.

Which one of these two is wrong? It should be possible to establish this by examining the NCrunch workspace and comparing this with your project's directory.

Are you using the 'Run impacted tests automatically, others manually' engine mode? If so, the misleading directory path you've described would make sense. This is NCrunch reporting the results from the last time the test was executed, perhaps from a previous session. Because your source code wasn't changed and the test wasn't impacted, it wasn't rerun by NCrunch using any of the new paths.
ericschmidt
#5 Posted : Monday, January 09, 2017 8:52:00 PM(UTC)
Rank: Member

Groups: Registered
Joined: 12/30/2016(UTC)
Posts: 12
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
It's not those. The folder (and file) relative path to the project are both the same, but the test is not running with the appropriate path.

Say I have the following setup.

TestProjects
-TestProjectA
-TestDataFiles
-TestFile1.txt
-TestFile2.txt

When built, the DLL tests are run out of the TestProject\bin\Debug folder. So I setup the test to use the file ..\..\TestDataFiles\TestFile1.txt. Instead of using TestProjectA\TestDataFiles\TestFile1.txt, the test is trying to access TestProjects\TestDataFiles\TestFile1.txt
Remco
#6 Posted : Monday, January 09, 2017 10:47:40 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 4,717

Thanks: 622 times
Was thanked: 725 time(s) in 690 post(s)
Can you confirm which test framework you are using? If you are using MSTest, are you using the DeploymentItem attribute?

Usually, when NCrunch sets up to run your tests, it will set both the current directory and the application domain base directory to be the same directory that the test assembly was placed by the compiler (typically bin\debug, but this can be different depending on your compile settings).

The current directory of the application of course isn't immutable - if there is code anywhere executed by NCrunch that changes this directory, downstream tests may be affected. Do you experience this problem if you run the test in isolation? Or is the problem inconsistent?

Can you confirm that you are using the current directory to find your files and not another method? (i.e. the app domain base directory or the location of an assembly in the application domain).
ericschmidt
#7 Posted : Monday, January 16, 2017 10:23:54 PM(UTC)
Rank: Member

Groups: Registered
Joined: 12/30/2016(UTC)
Posts: 12
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
We are using NUnit v3.5.0.0.

Yes, as far as I can tell, the NCrunch environment matches the environment of the project and test files themselves. The assembly is located in Project\bin\Debug\TestingAssembly.dll and the test files are located in Projects\ComponentTestFiles\TestFile.txt. Thus, to me, setting Environment.CurrentDirectory = "..\..\ComponentTestFiles\" should work but it's not.

To do some more testing, I ended up rolling back to using NUnit v2.6.4, and found that this issue goes away. Also, in v3.5, tests like this only seem to fail when the tests are being run in parallel. If I rerun a failed test using the "Run selected test in new task runner process" that the test passes. However, if I click the "Run all tests" button at the top of the NCrunch Tests UI, these tests fail. I'm not sure how to further troubleshoot this.
Remco
#8 Posted : Tuesday, January 17, 2017 1:58:31 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 4,717

Thanks: 622 times
Was thanked: 725 time(s) in 690 post(s)
ericschmidt;9690 wrote:

To do some more testing, I ended up rolling back to using NUnit v2.6.4, and found that this issue goes away. Also, in v3.5, tests like this only seem to fail when the tests are being run in parallel. If I rerun a failed test using the "Run selected test in new task runner process" that the test passes. However, if I click the "Run all tests" button at the top of the NCrunch Tests UI, these tests fail. I'm not sure how to further troubleshoot this.


This is definitely a sequence dependent issue then. Either the test code or the code under test is changing the current directory during a run. Because there are tests later in the execution pipeline that depend on the current directory being set consistently, they fail intermittently depending on which tests have been previously executed before them in the test process. See sequence dependent tests for more information on this kind of failure.

Under NCrunch there are many variables that affect the sequence in which tests are executed. NUnit V2 also has a different internal architecture to NUnit V3, so is likely that this problem will not present itself consistently between different runners or versions of NUnit.

I recommend performing a search of your codebase for anything making a call to Directory.SetCurrentDirectory. Note that the call may not actually be in your code at all (for example, you may be calling into compiled 3rd party code or a Win32 API that changes the current directory).

Creating an NUnit SetUpFixture with a teardown section that throws an exception if the current directory has been changed is one way to find the test causing the problem if you execute the tests manually independently or in groups.

Users browsing this topic
Guest (2)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.059 seconds.