Hi, thanks for posting!

Effective cleanup of resources at the end of a test run is difficult to do, because of the nature of automated testing.

The key underlying problem is that there is no way to be certain that the cleanup code will have a chance to run. For example, if the test process experiences a critical failure (i.e. stack overflow, out of memory, unmanaged exception, etc), it will be immediately terminated by the O/S and the cleanup code won't have a chance to run. When using a tool like NCrunch to run your tests all the time, the chances of something like this happening are an order of magnitude higher than with an end-to-end manual runner. The nature of this problem is such that no additional feature or modification would allow NCrunch to work reliably in the way you're hoping for (regardless of the complexities of by-passing the NUnit integration, which are also problematic).

The best way to approach this is with a re-think. Instead of making tests clean up out-of-process resources when they finish, look at cleaning up these resources when the test starts its execution. So when the test starts its execution, it checks for orphaned Solr instances that aren't tied to any resident test processes and terminates them. In this way, you never encounter a scenario where the resources can leak. A side-effect of this is that you'll probably still some of them hanging around after you close down VS, though there are probably other mechanisms you could implement to handle this situation if it's important to you.

This depends a bit on how the processes are launched by your own code. The key piece of information would be the Process ID. If you can find a way to identify the process ID of the Solr instance being used by your test run, then you can store this somewhere in a globally accessible location (i.e. file system, memory mapped file, etc).

Basically you'd get the PID of the Solr instance, tie this up with the PID of your test runner process (Process.GetCurrentProcess()), then put these values together as an entry in your global state. At the start of every test run, the test goes and reads from this state and enumerates all entries, comparing these with the output from Process.GetProcesses(). Test processes that are listed in the global state but aren't showing up in Process.GetProcesses() must be from tests that have finished, and you can safely kill their related Solr process instances.

The hard part may well be getting the PID of the Solr process instance. If the API doesn't give you this, there is still a messy way to get it. You can infer the value from the timing of the Solr process being launched. For example:

1. Check the list of running processes, remember all the Solr ones
2. Initialise Solr process
3. (In a loop) Repeatedly query the list of running processes, comparing against the list from 1. until a new process shows up that isn't known
4. Take note of the PID of the new process

The above gets more complicated when running tests in parallel, because they can interfere with each other. So you'd need to wrap this up using System.Threading.Mutex to make sure only one test is launching Solr at any one time.

Of course, if Solr just has a .GetProcessId(), then it's much easier :)