All Database tests fail with latest version

All Database tests fail with latest version. I have tried to eliminate this by setting NCrunch to a single CPU core, and process but it still fails. MS test passes all the tests so it is appears to be an issue with the latest version of NCrunch. All these tests worked fine under the previous version.

The Errors is:

System.Data.SqlClient.SqlException (0x80131904): An attempt to attach an auto-named database for file C:\Users\[USERNAME]\AppData\Local\NCrunch\7988\30\TestResults\26af65b2-2d7e-40ca-a962-74f0abaead72\Out\Resources\MdmTestDb.mdf failed. A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.

Hi Terry,

Can you share any more information about the life cycle of this .mdf file? Is it a resource that is stored with your solution and deployed into the MSTest working area, or is it a file generated from one of your tests?

If you right click on the failed test in the Tests Window, then go to Advanced->Browse to workspace, can you find this file in the NCrunch workspace?

Yes the mdf file is stored with the project, all operations/tests never changes the content of the DB, so it can remain read only.
the connection string in the project app.config file is

connectionString="Data Source=(localdb)\v11.0;AttachDbFilename=|DataDirectory|\Resources\MdmTestDb.mdf;Integrated Security=True"

yes if can see the mdf file in the test workspace.
we never had a problem until we upgraded to the new version.

The new version introduced support for the MSTest variable 'DataDirectory', so I wonder if this might be what is causing the issue. Does removing this value from the connection string and replacing it with an alternative value (i.e. perhaps constructing the .mdf path using the current directory) make any difference?

I'm curious as to why the value NCrunch is placing against DataDirectory is different to MSBuild. Is there any chance you'd be able to get me a capture of the difference in this value between NCrunch/MSTest on your solution? Try adding the following code and running your test using each runner:

Console.WriteLine(AppDomain.CurrentDomain.GetData("DataDirectory"));


Thanks!

Remco

DataDirectory is the issue; NCrunch uses the same folder across multiple tests.
as such, it is behaving badly to error out.

The preferred behavior is: If the database is already there, it should use it, not error out. Many test running against a single database.

We have multiple tests, against the same database, with the same connection string. Which I would think is how most tests run, you have many individual test.

NCrunch will make multiple calls into the same process to run sets of tests in batches. This is necessary in order to perform proper continuous testing, as the priorities of tests can constantly change as the source code is being altered. Unfortunately, this makes impossible the old traditional approach of running all tests from start to finish in a single process. Most tests are are OK with this - but some tests that perform setup actions on each test run without considering the behaviour of NCrunch can experience problems. Usually it's fairly easy to fix these tests using a static flag that is set when the setup code is first run, then compared at each time the code is invoked to make sure the setup code is never called twice.

Have a look at test atomicity for more information on this.

If you're experiencing problems with NCrunch running these database tests in parallel with each other over the same .mdf file, you might want to look at introducing some concurrency constraints using attributes.

Ya, that's really not a good solution, in this case if the DB is available it should just use it, and not bail, its a test database, not a resource file - the error message even recognizes it as the database. Would not the best solution be able to adjust the engine - so it sees it is a DB not a file and use it, and ignore the file copy error?

The concurrent example as I read it would create a copy of the db for each test - well given a typical solution may have 100's of tests .. I assume I've understood that wrong?

I will investigate the IsolatedAttribute and other methods - Thanks.

Hi Terry,

Sorry, I think I have misunderstood your problem.

NCrunch has no knowledge of databases or of the workings of your source code. It does not manipulate connection strings or anything around how your code connects to the database. Your code is basically just code that is executed under test. There is a line of code in NCrunch that explicitly sets the DataDirectory AppDomain variable to be the working directory of the executing test, it does so in this way:

AppDomain.CurrentDomain.SetData("DataDirectory", <working directory>);

This was added because some people make use of the DataDirectory appdomain value which is automatically set by MSTest. This value is automatically placed into connection strings during connection by ADO.NET, so if for some reason this value is different to that which is normally set by MSTest, then your code will be unable to find the database.

The .mdf file is just a file, treated like NCrunch the same as any other file. Because the file is being shared between tests, you should definitely look into making use of ExclusivelyUsesAttribute if you are using parallel execution for your database tests.

But anyway, the key question here is whether or not the database resource file is being referenced in the place that it exists in the NCrunch workspace.

Try adding the following two lines to your test code, then inspecting the results in the test output:

Console.WriteLine(AppDomain.CurrentDomain.GetData("DataDirectory") + @"\Resources\MdmTestDb.mdf");
Console.WriteLine(System.IO.File.Exists(AppDomain.CurrentDomain.GetData("DataDirectory") + @"\Resources\MdmTestDb.mdf"));

This will tell us whether or not the file is in its expected location in NCrunch's workspace. If you can let me know what the results are from the above two lines, I can help with further troubleshooting.

Cheers,

Remco

Here is the output:
C:\Users\terry.sansom\AppData\Local\NCrunch\8428\31\TestResults\bb89c3d7-2699-410d-bbc9-2bc0e38ad722\Out\Resources\MdmTestDb.mdf
False
Assembly initialize/cleanup failed: System.Data.SqlClient.SqlException (0x80131904): An attempt to attach an auto-named database for file C:\Users\terry.sansom\AppData\Local\NCrunch\8428\31\TestResults\bb89c3d7-2699-410d-bbc9-2bc0e38ad722\Out\Resources\MdmTestDb.mdf failed. A database with the same name exists, or specified file cannot be opened, or it is located on UNC share.

Some success. I added the attributes :

[DeploymentItem(@"Resources\MdmTestDb.mdf", "Resources")]
[DeploymentItem(@"Resources\MdmTestDb.ldf", "Resources")]

To the first class in the test project, and that seems to work, however, the ideal solution would be to have the attributes work on the method decorated with [AssemblyInitialize]
but that does not seem to work.

Thanks Terry, you've found the hole :)

NCrunch doesn't handle these attributes on the AssemblyInitialize method. I'll note this down for a fix. Thanks for your help in troubleshooting this!

To work around this, you may wish to consider referencing the .mdf file directly using relative paths that reach outside the MSTest working directory. If you can reference the file without needing the DeploymentItem attributes, you can then avoid needing to specify these on every test in your project.

On a further note,
when I add

AppDomain.CurrentDomain.SetData("DataDirectory", dataDirectory);

to a test class method with the [TestInitialize] attribute the class will find and use the Database, however when I do this on the method marked with
[AssemblyInitialize] the classes are unable to find the databases - does the DataDirectory item get reset?

Yes - NCrunch will specifically set it before each individual test run. It looks like this is also different to the behaviour in MSTest 2012, which sets it only at the beginning of the test run. I've also noted this down for a fix. You may wish to replace the DataDirectory with something else (perhaps a manually coded file path) until this can be resolved. Sorry for the trouble.

Here is what I have done...
I created a base test class, which all our other DB test classes will inherit from.
[TestClass]
public class RepositoryBaseTests
{
[TestInitialize]
public virtual void TestInitialize()
{
//-- Set the DataDirectory for NCrunch
if (NCrunchEnvironment.NCrunchIsResident())
{
AppDomain.CurrentDomain.SetData("DataDirectory", dataDirectory);
}
}
}

(Made it virtual to support the case where an override may be required.)

That seems to have it licked.

Nicely done! This will do the trick. Thanks again for reporting these problems.