I am seeing some different behaviour between the console and the VS plugin I think. I have class which creates a temporary database for each test. This class simply has a static id for the db which is a guid and relies on the fact that NCrunch runs each test in its own process to ensure that the guid is different in each test. This works fine in VS (and each guid is unique in each test as far as I am aware).

I have also been using NCrunch console on my build server, but have had some tests failing randomly with DB access timeouts and have noticed that I seem to get batches of tests all using the same guid, and I think it may be that the db is being destroyed/created by each test even though the other tests are still using it.

Am I imagining this difference? Or does the console work slightly differently to the VSPlugin in terms of test isolation? If it does are there any recommendations for tackling this?

**UPDATE**

I've just reduced the number of concurrent tests to 1 and all my tests have gone back to passing. I was originally using 4 concurrent tests on a 2 core machine and didn't see many test failures due to this issue. Reducing to 2 seemed to have upped the numbers of tests that failed. I could live with some tests being run in the same process as long as the ones scheduled in the same process never ran in parallel, and the parallelisation was always achieved by running tests in different processes together

Cheers

Sam

Thanks for the quick reply Remco.

I was under the (possibly incorrect) assumption that each test was run completely isolated so any static properties would be per test. I have a static field:

public static Guid DbId = Guid.NewGuid();

or similar and this field is used to generate the name of a mdf file which localdb uses as the db for the current test, and EF runs migrations on to create the schema. This ensures each test is running entirely isolated. In debug I think that each test always gets its own ID, although I'll need to check that, but it seems that this id is the same in some tests when I use the console runner.

I can't use the process id as I want each test to use its own unique db (hence the GUID). I can't just genberate a new guid everytime the class is created as EF creates multiple instances when its doing the migrations.

Hope this is clear.

I'm working round the issue by only runnning 1 test at a time currently, but part of the reason to use NCrunch on the server was to be able to parallelize the tests as they are reasonably slow (given each one creates its own db :))

Hi Remco.

Thanks for the response. I have been able to restore the 'each test usijng its own database' situation now, by doing both things suggested above (using the Isolated attribute on the entire assembly, and setting the guid in each test setup. I can probably remove the isolated attribute now), but I'm still not seeing the tests running in parallel. As each test creates its own db, if they were running in parallel then I'd see multiple db files being created at once, but I don't. and I upped the VM they are running on to an 8 core machine and tried to configure NCRunch to run using 8 threads, but the tests all took the same time still.

I'm not sure if its my config file, so this is the contents of that:

<GlobalConfiguration>
<CoverageMarkerStyle>Circle</CoverageMarkerStyle>
<AutoAdjustMarkerColors>true</AutoAdjustMarkerColors>
<MarkerPerformanceDisplaySensitivity>100</MarkerPerformanceDisplaySensitivity>
<MarkerPerformanceAggregationType>Average</MarkerPerformanceAggregationType>
<SpinnerInnerCircleColor>-1</SpinnerInnerCircleColor>
<SpinnerBuildFailureCircleColor>-4558080</SpinnerBuildFailureCircleColor>
<SpinnerTestFailureCircleColor>-65536</SpinnerTestFailureCircleColor>
<SpinnerInitialisingCircleColor>-16776961</SpinnerInitialisingCircleColor>
<SpinnerNormalCircleColor>-16744423</SpinnerNormalCircleColor>
<SpinnerNormalTextColor>-16776961</SpinnerNormalTextColor>
<SpinnerFailingTestsTextColor>-65536</SpinnerFailingTestsTextColor>
<SpinnerRecentlyFailingTestsTextColor>1509949440</SpinnerRecentlyFailingTestsTextColor>
<SpinnerFailingBuildTextColor>-16777216</SpinnerFailingBuildTextColor>
<SpinnerRecentlyFailingBuildTextColor>1509949440</SpinnerRecentlyFailingBuildTextColor>
<SpinnerTestsQueuedTextColor>-16756736</SpinnerTestsQueuedTextColor>
<SpinnerBuildTaskSpinnerColor>-1</SpinnerBuildTaskSpinnerColor>
<SpinnerAnalysisTaskSpinnerColor>-1</SpinnerAnalysisTaskSpinnerColor>
<BackgroundColor>-1</BackgroundColor>
<ForegroundColor>-16777216</ForegroundColor>
<SecondaryBackgroundColor>-2039584</SecondaryBackgroundColor>
<SecondaryForegroundColor>-16777216</SecondaryForegroundColor>
<SpinnerTestTaskSpinnerColor>-1</SpinnerTestTaskSpinnerColor>
<UseVSColors>true</UseVSColors>
<TextOutputFontName>Courier New</TextOutputFontName>
<TextOutputFontStyle>Regular</TextOutputFontStyle>
<TextOutputFontSize>8.25</TextOutputFontSize>
<GridReconnectionDelayInSeconds>10</GridReconnectionDelayInSeconds>
<FastLaneThreads>2</FastLaneThreads>
<FastLaneThresholdInMilliseconds>10000</FastLaneThresholdInMilliseconds>
<SlidingBuildDelayInMilliseconds>0</SlidingBuildDelayInMilliseconds>
<LogToOutputWindow>false</LogToOutputWindow>
<CheckForUpdates>true</CheckForUpdates>
<CPUCoresAssignedToVisualStudio>7</CPUCoresAssignedToVisualStudio>
<CPUCoresAssignedToNCrunch>0,1,2,3,4,5,6</CPUCoresAssignedToNCrunch>
<MarkerColorNoCoverage>-16777216</MarkerColorNoCoverage>
<MarkerColorPoorPerformance>-256</MarkerColorPoorPerformance>
<MarkerColorPassingCoverage>-16744448</MarkerColorPassingCoverage>
<MarkerColorFailingCoverage>-65536</MarkerColorFailingCoverage>
<FileVersion>2</FileVersion>
<WorkspaceBasePath>e:\ncrunch\</WorkspaceBasePath>
<BuildProcessMemoryLimit>125000000</BuildProcessMemoryLimit>
<TestProcessMemoryLimit>0</TestProcessMemoryLimit>
<MaxNumberOfProcessingThreads>8</MaxNumberOfProcessingThreads>
<LogVerbosity>Summary</LogVerbosity>
<TerminateTestRunnerTasksOnExecutionComplete>true</TerminateTestRunnerTasksOnExecutionComplete>
<MaxTestRunnerProcessesToPool>4</MaxTestRunnerProcessesToPool>
<TestSelectorExpression></TestSelectorExpression>
</GlobalConfiguration>

Are there anything wrong with the config? or is there some way to determine why the tests are not running in parallel?

Cheers

Sam

I don't have a solution configuration checked in to source control, but adding one has allowed the tests to run in parallel. Thanks for the tip. I'd expected the global setting to be used if none existed, but I assume that it has defaulted to 'false' as none was available