System.ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.

After a relatively small update I started getting the error below. I've confirmed that I can update to the prior commit and the error goes away. The code builds and runs just fine at the commit that causes the NCrunch error. I'm running version 4.9.0.8.

[PID:31152 18:16:23.8724 LocalBuildTask-56] ERROR (Internal): System.ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.
Parameter name: sourceIndex
at System.Array.Copy(Array sourceArray, Int64 sourceIndex, Array destinationArray, Int64 destinationIndex, Int64 length)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory.(CilAddress , CilAddress , UInt32 , UInt32 )
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory..ctor(CilAddress rawPdbBase, CilLegacyPdbHeader header)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacySymbolsDataSource..ctor(Byte[] pdbFileContent, UInt32 entryPointToken)
at nCrunch.Compiler.Cil.CilAssembly.Load(FilePath assemblyFile, CilContext context)
at nCrunch.Compiler.Cil.CachedCilAssembly..ctor(FilePath assemblyFilePath, CilContext context, Boolean isInstrumentationTarget)
at nCrunch.Compiler.Cil.CilSession.LoadAssembly(FilePath assemblyFilePath, Boolean isInstrumentationTarget)
at nCrunch.Compiler.StaticManipulation.BuiltCilAssembly..ctor(FilePath builtAssemblyFilePath, IList`1 adjustedAssemblyReferences, Boolean instrumentForPerformanceMeasurement, ComponentInstrumentationParameters instrumentationParameters, FilePath expectedForegroundBuildOutputPath)
at nCrunch.Compiler.RemoteBuildRunner..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentInstrumentationParameters , BuildOutput , IBackgroundTaskProcessor )
at nCrunch.Compiler.RemoteBuildRunner.PerformPostProcessingOfBuiltAssembly(ComponentInstrumentationParameters instrumentationParameters, BuildOutput output, IBackgroundTaskProcessor backgroundTaskProcessor)
at nCrunch.Core.BuildManagement.BuildEnvironment..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.BuildManagement.BuildEnvironment.Build(SnapshotComponent snapshotComponentToBuild, IList`1 referencedComponents, GridClientId gridClientId, IList`1 customEnvironmentVariables, IPlatformBuildExtender extender, Guid taskId, GridAddress clientAddress, Boolean extractCoverageReportStructure)
at nCrunch.Core.Processing.BuildTaskLogic.DoProcessTaskAndReturnSuccessFlag()
at nCrunch.Core.Processing.TaskLogic.ProcessTaskAndReturnSuccessFlag()
at nCrunch.Client.Processing.LocalProcessingTask.ProcessTaskAndReturnSuccessFlag()
at nCrunch.Client.Processing.ProcessingQueue..()

robinz wrote:After a relatively small update I started getting the error below. I've confirmed that I can update to the prior commit and the error goes away. The code builds and runs just fine at the commit that causes the NCrunch error. I'm running version 4.9.0.8.

Thanks for sharing this issue. Can you confirm whether the update you're describing was to your own codebase, or was it to NCrunch itself?

Is the project involved of an unusual size?

Can you confirm if the build referenced in this thread solves the issue for you? https://forum.ncrunch.net/yaf_postst3145_Problem-with-new-instrumentation-since-4-9.aspx.

The update was to our codebase.

We're working in VS 2019, in Debug x64 mode.

On the update that works we have this size:
dll: 11.0 MB (11,601,920 bytes)
pdb: 26.1 MB (27,383,296 bytes)

The commit that fails has this size:
dll: 11.0 MB (11,602,432 bytes)
pdb: 26.1 MB (27,385,344 bytes)

I updated to NCrunch 4.10.0.2 and am receiving a similar error:
[PID:25896 09:52:18.7988 LocalBuildTask-53] ERROR (Internal): System.ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.
Parameter name: sourceIndex
at System.Array.Copy(Array sourceArray, Int64 sourceIndex, Array destinationArray, Int64 destinationIndex, Int64 length)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory.(CilAddress , CilAddress , UInt32 , UInt32 )
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory..ctor(CilAddress rawPdbBase, CilLegacyPdbHeader header)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacySymbolsDataSource..ctor(Byte[] pdbFileContent, UInt32 entryPointToken)
at nCrunch.Compiler.Cil.CilAssembly.Load(FilePath assemblyFile, CilContext context)
at nCrunch.Compiler.Cil.CachedCilAssembly..ctor(FilePath assemblyFilePath, CilContext context, Boolean isInstrumentationTarget)
at nCrunch.Compiler.Cil.CilSession.LoadAssembly(FilePath assemblyFilePath, Boolean isInstrumentationTarget)
at nCrunch.Compiler.StaticManipulation.BuiltCilAssembly..ctor(FilePath builtAssemblyFilePath, IList`1 adjustedAssemblyReferences, Boolean instrumentForPerformanceMeasurement, ComponentInstrumentationParameters instrumentationParameters, FilePath expectedForegroundBuildOutputPath)
at nCrunch.Compiler.RemoteBuildRunner..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentInstrumentationParameters , BuildOutput , IBackgroundTaskProcessor )
at nCrunch.Compiler.RemoteBuildRunner.PerformPostProcessingOfBuiltAssembly(ComponentInstrumentationParameters instrumentationParameters, BuildOutput output, IBackgroundTaskProcessor backgroundTaskProcessor)
at nCrunch.Core.BuildManagement.BuildEnvironment..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.BuildManagement.BuildEnvironment.Build(SnapshotComponent snapshotComponentToBuild, IList`1 referencedComponents, GridClientId gridClientId, IList`1 customEnvironmentVariables, IPlatformBuildExtender extender, Guid taskId, GridAddress clientAddress, Boolean extractCoverageReportStructure)
at nCrunch.Core.Processing.BuildTaskLogic.DoProcessTaskAndReturnSuccessFlag()
at nCrunch.Core.Processing.TaskLogic.ProcessTaskAndReturnSuccessFlag()
at nCrunch.Client.Processing.LocalProcessingTask.ProcessTa

Thanks for these extra details.

I haven't seen this issue before. Somehow there has been a change in the way the O/S is structuring the PDB.

Do you have the option to switch to using a portable PDB instead? This would give an effective workaround to the problem.

I was able to rebase the branch on a newer version of our software that had some changes to how the project was configured and the problem went away.

robinz wrote:I was able to rebase the branch on a newer version of our software that had some changes to how the project was configured and the problem went away.

Thanks for letting me know. If you find a way to reproduce this problem with a self-contained sample you can share with me, please send it through the contact form. NCrunch is supposed to be able to handle any PDB produced by the compiler. It makes me nervous knowing it's possible for something to exist that we can't handle.

I got the same error today and found a workaround. Similarly, it was an innocuous change to the code that compiled fine in Visual Studio but created this error when nCrunch compiled it. I don't know what the root cause was, but restarting my PC worked around the issue.

Got this error again after a small code change using NCrunch 4.11. I updated to 4.18.0.3 and received a similar error:

4.11

[PID:37408 10:39:22.244 LocalBuildTask-66] ERROR (Internal): System.ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.
Parameter name: sourceIndex
at System.Array.Copy(Array sourceArray, Int64 sourceIndex, Array destinationArray, Int64 destinationIndex, Int64 length)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory.(CilAddress , CilAddress , UInt32 , UInt32 )
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory..ctor(CilAddress rawPdbBase, CilLegacyPdbHeader header)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacySymbolsDataSource..ctor(Byte[] pdbFileContent, UInt32 entryPointToken)
at nCrunch.Compiler.Cil.CilAssembly.Load(FilePath assemblyFile, CilContext context)
at nCrunch.Compiler.Cil.CachedCilAssembly..ctor(FilePath assemblyFilePath, CilContext context, Boolean isInstrumentationTarget)
at nCrunch.Compiler.Cil.CilSession.LoadAssembly(FilePath assemblyFilePath, Boolean isInstrumentationTarget)
at nCrunch.Compiler.StaticManipulation.BuiltCilAssembly..ctor(FilePath builtAssemblyFilePath, IList`1 adjustedAssemblyReferences, Boolean instrumentForPerformanceMeasurement, ComponentInstrumentationParameters instrumentationParameters, FilePath expectedForegroundBuildOutputPath)
at nCrunch.Compiler.RemoteBuildRunner..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentInstrumentationParameters , BuildOutput , IBackgroundTaskProcessor )
at nCrunch.Compiler.RemoteBuildRunner.PerformPostProcessingOfBuiltAssembly(ComponentInstrumentationParameters instrumentationParameters, BuildOutput output, IBackgroundTaskProcessor backgroundTaskProcessor)
at nCrunch.Core.BuildManagement.BuildEnvironment..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.BuildManagement.BuildEnvironment.Build(SnapshotComponent snapshotComponentToBuild, IList`1 referencedComponents, GridClientId gridClientId, IList`1 customEnvironmentVariables, IPlatformBuildExtender extender, Guid taskId, GridAddress clientAddress, Boolean extractCoverageReportStructure)
at nCrunch.Core.Processing.BuildTaskLogic.DoProcessTaskAndReturnSuccessFlag()
at nCrunch.Core.Processing.TaskLogic.ProcessTaskAndReturnSuccessFlag()
at nCrunch.Client.Processing.LocalProcessingTask.ProcessTaskAndReturnSuccessFlag()
at nCrunch.Client.Processing.ProcessingQueue..()

4.18.0.3
NCrunch: If you are experiencing problems in getting this project to build, have a look at http://www.ncrunch.net/documentation/troubleshooting_project-build-issues
NCrunch has encountered an internal error: System.ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.
Parameter name: sourceIndex
at System.Array.Copy(Array sourceArray, Int64 sourceIndex, Array destinationArray, Int64 destinationIndex, Int64 length)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory.(CilAddress , CilAddress , UInt32 , UInt32 )
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory..ctor(CilAddress rawPdbBase, CilLegacyPdbHeader header)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacySymbolsDataSource..ctor(Byte[] pdbFileContent, UInt32 entryPointToken)
at nCrunch.Compiler.Cil.CilAssembly.Load(FilePath assemblyFile, CilContext context)
at nCrunch.Compiler.Cil.CachedCilAssembly..ctor(FilePath assemblyFilePath, CilContext context, Boolean isInstrumentationTarget)
at nCrunch.Compiler.Cil.CilSession.LoadAssembly(FilePath assemblyFilePath, Boolean isInstrumentationTarget)
at nCrunch.Compiler.StaticManipulation.BuiltCilAssembly..ctor(FilePath builtAssemblyFilePath, IList`1 adjustedAssemblyReferences, Boolean instrumentForPerformanceMeasurement, ComponentInstrumentationParameters instrumentationParameters, FilePath expectedForegroundBuildOutputPath)
at nCrunch.Compiler.RemoteBuildRunner..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentInstrumentationParameters , BuildOutput , IBackgroundTaskProcessor )
at nCrunch.Compiler.RemoteBuildRunner.PerformPostProcessingOfBuiltAssembly(ComponentInstrumentationParameters instrumentationParameters, BuildOutput output, IBackgroundTaskProcessor backgroundTaskProcessor)
at nCrunch.Core.BuildManagement.BuildEnvironment..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.BuildManagement.BuildEnvironment.Build(SnapshotComponent snapshotComponentToBuild, IList`1 referencedComponents, GridClientId gridClientId, IList`1 customEnvironmentVariables, IPlatformBuildExtender extender, Guid taskId, GridClientDescription clientDescription, Boolean extractCoverageReportStructure)

I was able to get around this by removing some obsolete files in the offending project.

Hi, thanks for sharing this issue.

I believe this to be a compatibility problem in NCrunch's assembly parsing system. Somehow, the size/structure of the assembly PDB it's trying to load is different to what it can handle.

I've yet to manage to produce such a situation with a test assembly in order to fix the problem. You might be able to work around this by using the new portable PDB format for this project instead of the old legacy one. The legacy PDBs are slowing being phased out now anyway.

Remco,

Wanted to let you know that it has been a while since I've seen this error, but it just popped up again in 5.20.0.2:
NCrunch has encountered an internal error: System.ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.
Parameter name: sourceIndex
at System.Array.Copy(Array sourceArray, Int64 sourceIndex, Array destinationArray, Int64 destinationIndex, Int64 length)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory.readStream(CilAddress rawPdbBase, CilAddress streamPagesArrayAddress, UInt32 totalStreamLengthInBytes, UInt32 pageCount)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory..ctor(CilAddress rawPdbBase, CilLegacyPdbHeader header)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacySymbolsDataSource..ctor(Byte[] pdbFileContent, UInt32 entryPointToken)
at nCrunch.Compiler.Cil.CilAssembly.Load(FilePath assemblyFile, CilContext context)
at nCrunch.Compiler.Cil.CachedCilAssembly..ctor(FilePath assemblyFilePath, CilContext context, Boolean isInstrumentationTarget)
at nCrunch.Compiler.Cil.CilSession.LoadAssembly(FilePath assemblyFilePath, Boolean isInstrumentationTarget)
at nCrunch.Compiler.StaticManipulation.BuiltCilAssembly..ctor(FilePath builtAssemblyFilePath, IList`1 adjustedAssemblyReferences, Boolean instrumentForPerformanceMeasurement, ComponentInstrumentationParameters instrumentationParameters, FilePath expectedForegroundBuildOutputPath)
at nCrunch.Compiler.RemoteBuildRunner.<>c__DisplayClass27_0.<processBuiltAssembly>b__0()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Compiler.RemoteBuildRunner.processBuiltAssembly(ComponentInstrumentationParameters instrumentationParameters, BuildOutput output, IBackgroundTaskProcessor backgroundTaskProcessor)
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.BuildManagement.BuildEnvironment.Build(SnapshotComponent snapshotComponentToBuild, IList`1 referencedComponents, GridClientId gridClientId, IList`1 customEnvironmentVariables, IPlatformBuildExtender extender, Guid taskId, GridClientDescription clientDescription, Boolean extractCoverageReportStructure)

Thanks for letting me know. I have also had another recent report that suggests this is still happening in niche scenarios. I have yet to pin it down in a way that allows me to fix it.

You might find you're able to work around the problem by adding a dummy class/method in project, as this shuffles the element counts involved.

FYI, I switched to another working directory, and it worked just fine on the same code.

robinz wrote:FYI, I switched to another working directory, and it worked just fine on the same code.

That's actually quite alarming. It suggests the problem is intermittent or not specific to the structure of the assembly. When you switched directory, what actually changed? Is there anything that might have affected the compiled state of the assembly?

I figured you would love that ;)

Nothing changed that I'm aware of. My working directories "should" be identical, but they obviously aren't.

When I get this error, I can ask NCrunch to Reload and Rebuild as many times as I want, but it always fails. The only fix is to add or remove enough code (and I have no idea what "enough" is) to get it past the error. I believe I've tried the trick of moving to another working directory in the past and had that continue to fail, but it has been long enough that I might be remembering incorrectly.

For the project that's failing like this, what do you currently have your debug/PDB set to? Is it embedded? Does changing it to portable resolve the issue?

I'm currently set to full. I moved on to another project in that workspace, and I can't reproduce the error now. I'll change that setting the next time this shows up.

robinz wrote:I'm currently set to full. I moved on to another project in that workspace, and I can't reproduce the error now. I'll change that setting the next time this shows up.

Thanks, I'm glad you're up and running again. Right now my suspicion is that something is wrong or unusual with the legacy PDB. If anyone else encounters this issue, I expect that setting the DebugType to Portable will solve it. If your application needs legacy PDBs for its normal runtime environment, you can set them to portable for NCrunch only by using the 'Custom Build Properties' setting.