ConsoleTool: System.AccessViolationException: Attempted to read or write protected memory.

[NCrunch Console Tool v3.9.0.1]

Got this error (NUnit tests) on TeamCity today (am about to upgrade to 3.10.0.6 in order to keep versions in sync with my vstudio2017 version).

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at nCrunch.TestRuntime.SharedExecutionMap.MarkTestLineExecutionTime(Int32 lineMarkerIndex, UInt32 tickCount)
   at nCrunch.TestRuntime.SharedMemoryExecutionDataRecorder.MarkTestLineExecutionTime(Int32 componentMappingId, Int32 lineMarkerIndex, UInt32 tickCount)
   at nCrunch.TestRuntime.TestCoverageEventListener.NCrunchExitMethod(Int32 componentId, Int32& existingCoverageMarkerIndex, UInt32 existingTickCount)
   at MYPROJ.Application.Tests.NUnitTests.DeadlineApplicationServiceFixtures.StartFixtures.WhenRestartedFixture.<>c__DisplayClass4.<>c__DisplayClass6.<ItShouldInvokeDoWorkAgain>b__1() in D:\TeamCity\buildAgent\work\xxx\Tests\MYPROJ.Application.Tests.NUnitTests\DeadlineApplicationServiceFixtures\StartFixtures\WhenRestartedFixture.cs:line 22
   at Utils.UnitTest.Threading.ThreadSleep.ThreadSleepByCriteria.Sleep() in D:\TeamCity\buildAgent\work\xxx\Utils\UnitTest\Threading\ThreadSleep.cs:line 158
   at Utils.UnitTest.Threading.ThreadSleep.ThreadSleepByCriteria..ctor(Func`1 sleepCriteriaCb, Int32 msCriteriaCheckInterval, Boolean sleepImmediately, Int32 msTimeout) in D:\TeamCity\buildAgent\work\xxx\Utils\UnitTest\Threading\ThreadSleep.cs:line 187
   at Utils.UnitTest.Threading.ThreadSleep.WhileCriteriaIsTrue(Func`1 criteriaCb, Int32 msCriteriaCheckInterval, Boolean sleepImmediately, Int32 msTimeout) in D:\TeamCity\buildAgent\work\xxx\Utils\UnitTest\Threading\ThreadSleep.cs:line 93
   at Utils.UnitTest.Threading.ThreadSleep.WhileCriteriaIsTrue(Func`1 criteriaCb, Int32 msCriteriaCheckInterval, Boolean sleepImmediately) in D:\TeamCity\buildAgent\work\xxx\Utils\UnitTest\Threading\ThreadSleep.cs:line 81
   at Utils.UnitTest.Threading.ThreadSleep.WhileCriteriaIsTrue(Func`1 criteriaCb, Int32 msCriteriaCheckInterval) in D:\TeamCity\buildAgent\work\xxx\Utils\UnitTest\Threading\ThreadSleep.cs:line 61
   at MYPROJ.Application.Tests.NUnitTests.DeadlineApplicationServiceFixtures.StartFixtures.WhenRestartedFixture.<>c__DisplayClass4.<ItShouldInvokeDoWorkAgain>b__0(CancellationToken cancellationToken, TimeSpan checkFrequency) in D:\TeamCity\buildAgent\work\xxx\Tests\MYPROJ.Application.Tests.NUnitTests\DeadlineApplicationServiceFixtures\StartFixtures\WhenRestartedFixture.cs:line 21
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object() arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object() parameters, Object() arguments)
   at System.Delegate.DynamicInvokeImpl(Object() args)
   at Moq.Extensions.InvokePreserveStack(Delegate del, Object() args)
   at Moq.MethodCall.<>c__DisplayClass8.<SetCallbackWithArguments>b__7(Object() args)
   at Moq.MethodCall.Execute(ICallContext call)
   at Moq.Interceptor.Intercept(ICallContext invocation)
   at Moq.Proxy.CastleProxyFactory.Interceptor.Intercept(IInvocation invocation)
   at Castle.DynamicProxy.AbstractInvocation.Proceed()
   at Castle.Proxies.IDeadlineProcessorServiceHostProxy.DoWork(CancellationToken cancellationToken, TimeSpan processingFrequencyParam)
   at MYPROJ.Application.Services.DeadlineApplicationService.<>c__DisplayClass1.<get_ServiceHostDoWorkAction>b__0() in D:\TeamCity\buildAgent\work\xxx\Application\MYPROJ.Application\Services\DeadlineApplicationService.cs:line 29
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
   at System.Threading.Tasks.Task.ExecutionContextCallback(Object obj)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
   at System.Threading.Tasks.Task.ExecuteEntry(Boolean bPreventDoubleExecution)
   at System.Threading.Tasks.ThreadPoolTaskScheduler.LongRunningThreadWork(Object obj)
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart(Object obj)

I've seen this also, rarely, on a heavyweight grid node.

It may take me time to track it down. I had hoped I was the only one seeing it. I guess you just crushed that delusion :(

A reset of the client or grid node always gets rid of it.

Ok thanks, this case should not have been involving any grid nodes though, and it didn't re-appear after upgrading (usage of NCrunch with this heavy project on TeamCity is still in the "experimental" phase so it isn't executed very often).

Hrm I take that back, now the build reported successful but with 0 tests run, looking into build log I see

15:53:50][Step 2/2] [15:55:29.4888-?-14] Ceasing to send messages because of an error (was the connection closed?): System.ObjectDisposedException: Cannot access a disposed object.
[15:53:50][Step 2/2] Object name: 'System.Net.Sockets.NetworkStream'.
[15:53:50][Step 2/2] 
[15:53:50][Step 2/2] Server stack trace: 
[15:53:50][Step 2/2]    at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.EncryptorStream. (Byte[]  , Int32  , Int32  )
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.EncryptorStream.Write(Byte[] buffer, Int32 offset, Int32 count)
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.BidirectionalStream.Write(Byte[] buffer, Int32 offset, Int32 count)
[15:53:50][Step 2/2]    at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
[15:53:50][Step 2/2]    at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)
[15:53:50][Step 2/2] 
[15:53:50][Step 2/2] Exception rethrown at [0]: 
[15:53:50][Step 2/2]    at System.Runtime.Remoting.Proxies.RealProxy.EndInvokeHelper(Message reqMsg, Boolean bProxyCase)
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.BidirectionalStream. .EndInvoke(IAsyncResult  )
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.BidirectionalStream.EndWrite(IAsyncResult asyncResult)
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.GridMessageSender. (IAsyncResult  )
[15:53:50][Step 2/2] [15:55:29.4947-?-23] Ceasing to send messages because of an error (was the connection closed?): System.ObjectDisposedException: Cannot access a disposed object.
[15:53:50][Step 2/2] Object name: 'System.Net.Sockets.NetworkStream'.
[15:53:50][Step 2/2] 
[15:53:50][Step 2/2] Server stack trace: 
[15:53:50][Step 2/2]    at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.EncryptorStream. (Byte[]  , Int32  , Int32  )
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.EncryptorStream.Write(Byte[] buffer, Int32 offset, Int32 count)
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.BidirectionalStream.Write(Byte[] buffer, Int32 offset, Int32 count)
[15:53:50][Step 2/2]    at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
[15:53:50][Step 2/2]    at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)
[15:53:50][Step 2/2] 
[15:53:50][Step 2/2] Exception rethrown at [0]: 
[15:53:50][Step 2/2]    at System.Runtime.Remoting.Proxies.RealProxy.EndInvokeHelper(Message reqMsg, Boolean bProxyCase)
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.BidirectionalStream. .EndInvoke(IAsyncResult  )
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.BidirectionalStream.EndWrite(IAsyncResult asyncResult)
[15:53:50][Step 2/2]    at nCrunch.Core.Grid.Connectivity.GridMessageSender. (IAsyncResult  )

And after that it seem to spend 15 minutes to just call LocalAnalysisTask and LocalBuildTask to grid nodes :/

The socket related issue above is essentially a noise exception that can happen if the connection is suddenly closed without warning. This won't be related to the earlier problem you've reported, but it does make sense that this could slow things down for your test run. A disconnection means that the grid client needs to resynchronise with remote grid nodes, which on a large solution can sometimes take a little while. There are some improvements here coming as part of the 3.10 release that haven't been included in any publicly available builds (yet).

NCrunch v3.10 has now been released.