I'm aware that an ExecutionEngineException usually means something is terribly wrong and since in this particular case I only run proper .NET MSIL tests, this may appear to be a bug in .NET's Core CLR or JIT itself. Nevertheless, I thought you'd want to know that if this happens, the NCrunch engine crashes badly, with a typical windows popup asking to debug or close the application.

Because NCrunch runs out of process, it does not seem to hamper my ability to continue working within Visual Studio, or to subsequently run the tests. I have not investigated whether it makes NCrunch unstable overall (so far it seems ok).

Since NCrunch is capable of catching that other uncatchable exception. StackOverflowException, I figured perhaps it is possible to add this one to that list. I'm not quite sure how you fix the SO exception (do you use a hosted CLR?), so I don't know if this lies in the range of easy to catch and fix, or hard. Besides, I realize this is probably a rare thing to happen.

Note, the NCrunch test window shows this in the log:

Sounds like a viable way, I assume you use the intrumentation API that you also use for measuring method's execution time?



Yes, I know. In fact, the CLR uses something similar to Environment.FailFast, which skips all exception handling, except in the CLR host process. The only way I have managed to catch an overflow *and* to recover (to the extend as to allow a managed message and error report) is by writing your own CLR host implementation in C++ and the appropriate allocation hooks (while not trivial, it isn't that hard either). This is the same approach Microsoft uses for hosted processes in SQL Server, for instance.

While there are some ways of recovering from an SO exception, the same is not true of an ExecutionEngineException. Though you can catch this exception in the CLR host process as well, there is no way to recover from it (though the CLR host process could opt a way to report this error back to the user by providing a hook through a dynamically loaded error reporting dll, or something similar). Even more than an SO exception, after this exception the process is in an indeterminate state. I think that SEHException and FatalExecutionError are in the same ballpark. There are some reports (and here) that .NET 4.5 and RyuJIT throw these errors on otherwise verifiable code.

I'm not 100% sure, but I think NCrunch uses a host process and child processes for running the tests. Perhaps the host process could check and catch the error, though I think you need C++ SEH injected in the child process to actually do so, I think.

Earlier versions of NCrunch did actually implement a CLR host in C++, with the main intention of this being to be able to detect and report stack overflow exceptions via an SEH block, prior to tearing the process down and continuing the test run in a new process.

I couldn't find a way to implement it reliably. It seemed to work well on some systems, then not on others. Eventually, the whole hosting infrastructure was removed because there always seemed to be an endless list of problems that were intermittent, unreproduceable and critically annoying for users. My understanding is that the CLR hosting infrastructure was originally implemented for hosting the CLR in SQL Server, and the API itself was simply exposed and documented for other developers a bit of a bonus point. I found many things in this API to be difficult to work with - much of the behaviour was inconsistent and didn't always work as documented.

So the current stack overflow handling just uses the static instrumentation written by NCrunch post-build. This means that disabling instrumentation on an assembly will turn off the stack overflow handling, but in almost all cases instrumentation is left on anyway :)

Thanks :) It did take a few iterations to get it right. Stack overflows are quite common when running tests concurrently, so it was important to handle them cleanly.



Thanks - I'll let you know if I find a way to handle them. My current plan is to hope that whatever underlying problems are triggering them will eventually be fixed by MS (where this is possible) :)