exception running tests on the grid

When running tests we sometimes get all tests pass but the run fails with this exception

[14:17:49][Step 8/12] [?-14] ERROR (Internal): System.IO.IOException: The process cannot access the file 'C:\Users\username\AppData\Local\NCrunch\hashcache' because it is being used by another process.
[14:17:49][Step 8/12] [?-14] at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
[14:17:49][Step 8/12] [?-14] at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
[14:17:49][Step 8/12] [?-14] at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
[14:17:49][Step 8/12] [?-14] at nCrunch.Core.Hashing.HashCacheIOManager. . ()
[14:17:49][Step 8/12] [?-14] at nCrunch.Core.Threading.PooledWorkItem.Start()
[14:17:49][Step 8/12] [?-14] at nCrunch.Core.Threading.ThreadFactory. (Object )

ANy way we can prevent this from failing the test run?

This looks to be an internal concurrency issue caused by a race condition.

It'll take a deep code review to figure out exactly what is happening here.

How is this error being reported for you? Does it appear on one of the test tasks or in the Tests Window? Is it happening locally or on a grid node?

Sorry, missed the notification about the response.

This is happening on a grid node, so we see it in the build log on teamcity.

Would you mind trying the build below? We've made some speculative fixes for this, at the very least we should have a little more information in the log:

NCrunch_Console_4.4.0.1.msi
NCrunch_Console_4.4.0.1.zip
NCrunch_GridNodeServer_4.4.0.1.msi
NCrunch_GridNodeServer_4.4.0.1.zip
NCrunch_LicenseServer_4.4.0.1.zip
NCrunch_VS2008_4.4.0.1.msi
NCrunch_VS2010_4.4.0.1.msi
NCrunch_VS2010_4.4.0.1.zip
NCrunch_VS2012_4.4.0.1.msi
NCrunch_VS2012_4.4.0.1.zip
NCrunch_VS2013_4.4.0.1.msi
NCrunch_VS2013_4.4.0.1.zip
NCrunch_VS2015_4.4.0.1.msi
NCrunch_VS2015_4.4.0.1.msi.7z
NCrunch_VS2015_4.4.0.1.zip
NCrunch_VS2017_4.4.0.1.msi
NCrunch_VS2017_4.4.0.1.msi.7z
NCrunch_VS2017_4.4.0.1.zip
NCrunch_VS2019_4.4.0.1.msi
NCrunch_VS2019_4.4.0.1.msi.7z
NCrunch_VS2019_4.4.0.1.zip

I'll try and do this but not super easy as lots of machines to update, and happens infrequently. Will this speculative fix be included in the main NCrunch release? It might be easier to schedule the update at some later date with a released version

samholder wrote:I'll try and do this but not super easy as lots of machines to update, and happens infrequently. Will this speculative fix be included in the main NCrunch release? It might be easier to schedule the update at some later date with a released version

Yes. We have some level of confidence behind this fix. The reason it's considered speculative is because the nature of the problem (race condition) makes it hard to reproduce reliably, so we don't have a way to completely confirm that the fix works. Given that it's probably not a critical issue, I'd say it's fine for you to just wait until the next version to update your NCrunch.