Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
System.ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.
Options
Previous Topic Next Topic
LatinProgrammer
#1 Posted : Friday, July 17, 2020 5:49:29 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 11/26/2019(UTC)
Posts: 7
Hello,

My NCrunch setup started failing today with this error: "System.ArgumentOutOfRangeException: Arrays larger than 2GB are not supported."

We have a pretty large project and I'm using NCrunch to get performance indicators. It was going slow but was giving useful results. Is there any way I can get it to work again? I have 24 GB of RAM in my machine and an i7-4770 processor so resources haven't been an issue.

Kind regards,

-Esteban

P.S.: The stack trace is below:
[12:45:25.7797-LocalBuildTask-29] ERROR (Internal): System.ArgumentOutOfRangeException: Arrays larger than 2GB are not supported.
Parameter name: sourceIndex
at System.Array.Copy(Array sourceArray, Int64 sourceIndex, Array destinationArray, Int64 destinationIndex, Int64 length)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory.(CilAddress , CilAddress , UInt32 , UInt32 )
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacyPdbStreamDirectory..ctor(CilAddress rawPdbBase, CilLegacyPdbHeader header)
at nCrunch.Compiler.Cil.Symbols.Legacy.CilLegacySymbolsDataSource..ctor(Byte[] pdbFileContent, UInt32 entryPointToken)
at nCrunch.Compiler.Cil.CilAssembly.Load(FilePath assemblyFile, CilContext context)
at nCrunch.Compiler.Cil.CachedCilAssembly..ctor(FilePath assemblyFilePath, CilContext context, Boolean isInstrumentationTarget)
at nCrunch.Compiler.Cil.CilSession.LoadAssembly(FilePath assemblyFilePath, Boolean isInstrumentationTarget)
at nCrunch.Compiler.NewStaticManipulation.BuiltCilAssembly..ctor(FilePath builtAssemblyFilePath, IList`1 adjustedAssemblyReferences, Boolean instrumentForPerformanceMeasurement, ComponentInstrumentationParameters instrumentationParameters, ILogger logger)
at nCrunch.Compiler.RemoteBuildRunner..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Compiler.RemoteBuildRunner.(ComponentInstrumentationParameters , BuildOutput , IBackgroundTaskProcessor )
at nCrunch.Compiler.RemoteBuildRunner.PerformPostProcessingOfBuiltAssembly(ComponentInstrumentationParameters instrumentationParameters, BuildOutput output, IBackgroundTaskProcessor backgroundTaskProcessor)
at nCrunch.Core.BuildManagement.BuildEnvironment..()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.Core.BuildManagement.BuildEnvironment.Build(SnapshotComponent snapshotComponentToBuild, IList`1 referencedComponents, GridClientId gridClientId, IList`1 customEnvironmentVariables, IPlatformBuildExtender extender, Guid taskId, GridAddress clientAddress)
at nCrunch.Core.Processing.BuildTaskLogic.DoProcessTaskAndReturnSuccessFlag()
at nCrunch.Core.Processing.TaskLogic.ProcessTaskAndReturnSuccessFlag()
at nCrunch.Client.Processing.LocalProcessingTask.ProcessTaskAndReturnSuccessFlag()
at nCrunch.Client.Processing.ProcessingQueue..()
Back to top
Remco
#2 Posted : Saturday, July 18, 2020 12:11:13 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Back to top
LatinProgrammer
#3 Posted : Monday, July 20, 2020 5:50:23 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 11/26/2019(UTC)
Posts: 7
Remco,

Thanks for the reply. I installed version 4.4.0.10 and opened the branch that was failing on me and no cigar, I still get the error.

Now, here is a very odd thing. I merged that developer branch down to our main developer branch and NCrunch was able to still move forward with the build (even after adding the new tests coming from the developer branch).

Below is my NCrunch windows run with the new version as of a few minutes ago:
https://ibb.co/gDhJsBG

It is good that the issue it is not in the main developer branch (if it was, I wouldn't be able to use NCrunch anymore). It would still be good to find out why the dev branch is failing and I'd be happy to try anything else to get it working so that I can have a way to resolve if it shows up elsewhere.

-Esteban
P.S.: I had to immediately roll back to version 4.3.0.13 because some async tests made NCrunch 4.4 hang indefinitely as long as they had the following signature:

[TestMethod]
public async Task [...]
Back to top
 |  Edit by user
Remco
#4 Posted : Tuesday, July 21, 2020 12:08:48 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for the extra detail here.

The problem is caused by a PDB file with an unexpected structure. Basically, the compiler on your machine is somehow producing a PDB file with a structure that we haven't seen before. Most probably it's something trivial in the PDB that we simply haven't been made aware of. The trick here is going to be finding a way to reproduce the exception so that we can analyse the PDB structure and figure out how to work with it. Are you able to build a sample project that can produce this error, or are you perhaps able to share the DLL and PDB files with me? You can submit code in ZIP form through the NCrunch contact form.

I'm not aware of any changes in the v4.4 release that would affect the execution of async MSTest tests. However, the installation of v4.4 would most likely have changed the execution sequence of your tests, which could cause existing race conditions or intermittent issues in the code to manifest differently. It would be great if you could closely examine the behaviour of these tests (i.e. using a debugger or trace logs) to try and understand why they are hanging. If you're able to find a way to produce this problem for us, I'd very much like to try and get it fixed before we push v4.4 up to the download page.

Edit: I just noticed how long your build times are for this assembly. How large is the PDB in terms of filesize? I wonder if this issue is size related.
Back to top
 |  Edit by user
LatinProgrammer
#5 Posted : Tuesday, July 21, 2020 1:37:26 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 11/26/2019(UTC)
Posts: 7
Remco;14886 wrote:
Hi, thanks for the extra detail here.

The problem is caused by a PDB file with an unexpected structure. Basically, the compiler on your machine is somehow producing a PDB file with a structure that we haven't seen before. Most probably it's something trivial in the PDB that we simply haven't been made aware of. The trick here is going to be finding a way to reproduce the exception so that we can analyse the PDB structure and figure out how to work with it. Are you able to build a sample project that can produce this error, or are you perhaps able to share the DLL and PDB files with me? You can submit code in ZIP form through the NCrunch contact form.

I'm not aware of any changes in the v4.4 release that would affect the execution of async MSTest tests. However, the installation of v4.4 would most likely have changed the execution sequence of your tests, which could cause existing race conditions or intermittent issues in the code to manifest differently. It would be great if you could closely examine the behaviour of these tests (i.e. using a debugger or trace logs) to try and understand why they are hanging. If you're able to find a way to produce this problem for us, I'd very much like to try and get it fixed before we push v4.4 up to the download page.

Edit: I just noticed how long your build times are for this assembly. How large is the PDB in terms of filesize? I wonder if this issue is size related.


I used the contact form to attempt to send you my files. I'm not sure if this worked because the two times I tried I get a 404 after submitting (perhaps the "success" page does not exist?).

The PDB is 33.5 MB in size. As mentioned before this is a large project and I'm not surprised that NCrunch is that slow with it. For a while, I excluded that project from NCrunch since I did not have any unit or integration tests running against it. However, we are working on some smoke tests and evaluating the performance of the assembly now, so I finally ended up needing to at this to NCrunch.

I re-installed 4.4 and I will let you know if I'm able to bypass the issues with the async tests. When I re-installed 4.3 I did have some issues with those as well but I was able to resolve pretty quickly (partially because I set those tests as ignored in 4.4 and 4.4 did not still want to move forward but 4.3 picked up the fact that the tests are ignored).

-Esteban
Back to top
Remco
#6 Posted : Wednesday, July 22, 2020 12:02:39 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
LatinProgrammer;14888 wrote:

I used the contact form to attempt to send you my files. I'm not sure if this worked because the two times I tried I get a 404 after submitting (perhaps the "success" page does not exist?).


Sorry about this. The submission form has a file size limit, and it's a safe bet that we've gone over it. Do you have another way to share the files with me? You can send a download link through the contact form (it will work if there are no large attachments).

If you are running one of the newer versions of .NET, a potential workaround for this problem is to change your build settings for this project to output a portable PDB instead of a 'full' one. The portable PDBs are much smaller and more efficient. That said, I still really need to fix this issue as NCrunch should work regardless.

Keep me posted on those async tests. If you can find a pattern to reproduce any bad behaviour by NCrunch when running these, I'll be happy to look as this one more deeply. We haven't made changes to any of the async execution code in MSTest or ignore flagging code for quite a while now, but that doesn't mean we can't have a long standing issue hiding in there somewhere.
Back to top
 |  Edit by user
LatinProgrammer
#7 Posted : Wednesday, July 22, 2020 1:39:45 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 11/26/2019(UTC)
Posts: 7
Hi Remco,

I sent you a contact form submission with a secure link to the files. Please do let me know if you got them.

The project is not on .NET core (and will be very unlikely to be ported). We do plan to use latest version of the "full" Framework whenever possible although it sometimes takes us a while to get there. I see that portable PDB files are exclusive to .NET core in the quick search that I did.

-Esteban
Back to top
LatinProgrammer
#8 Posted : Wednesday, July 22, 2020 5:42:16 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 11/26/2019(UTC)
Posts: 7
I have some additional information on those async tests as well. It looks like the main issue is that I have a very large timeout for this particular VS solution (the default test timeout I have for NCrunch is 300000). Because of this, since the web API is hanging on me, then it looks like NCrunch is hanging, whereas the issue might be with the web call itself. My current workaround is to ignore all the web API tests and then let the rest of NCrunch tasks complete. Then I unignore the tests and the API call works (the whole process of calling the web API and processing the API takes a maximum of 266 ms to complete and an average of 96 ms.

My current thinking is that the API call may be flagging the NCrunch parallel calls done in quick succession as a DOS attack and then temporarily holding the responses. I would need to decrease the NCrunch timeout of whole solution to confirm after it happens again.

Behavior is the same between 4.4 and 4.3 now that I looked at it in detail so I no longer think this is an issue with the new version.
Back to top
 |  Edit by user
Remco
#9 Posted : Thursday, July 23, 2020 12:42:46 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for sending through the files and for the additional information.

Unexpectedly, when I put the files through the instrumentor on this side, they worked correctly without any problems. From what I can establish, it looks to me like the PDB file is being corrupted somehow prior to NCrunch being able to load it in the workspace, perhaps by something in the build system or by a background process of some kind.

Could you try setting your 'Instrumentation mode' configuration setting to 'Legacy' to see if the legacy system still works with the file? (note, with a file of this size, the legacy system will be horribly slow).

Also, if you have a virus scanner running on the machine, try disabling it temporarily to see this has any effect (unlikely, but I've seen crazy things happen like this).

Regarding the async tests, I recommend marking these with ExclusivelyUsesAttribute so that they don't get run at the same time. This should prevent the need for you to constantly adjust timeout values.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.101 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download