Hi Matthew,
This is an idea that has received some attention in the past. It was actually a business model I considered pursuing when designing the distributed processing. I eventually rejected it for several reasons. A few people have since approached me with the idea asking my opinion about it. So far, as far as I know, no one has yet implemented it.
My reasons for not taking the distributed processing in this direction are as follows:
- Trust issues. As you've identified, there needs to be considerable trust between the provider and the customer for such a service to work. Or at least, very sophisticated systems to prevent a customer from damaging the service or performing illegal activities.
- Investment. Such a service would require quite a bit of investment to make it into a tidy package that could easily be used. This may or may not involve further development of NCrunch itself. Basically, it would need to be a serious project handled with controlled costing etc.
- Liability. The legal risk of setting up such a service would need to be explored in a great deal of detail. If the service were to operate internationally, this would be very complex as you may be exposed to many different regulatory environments.
- Appliance. Unfortunately, most of NCrunch's users never touch distributed processing. I can only assume that multi-core for them is fast enough, or they use it only for unit testing. Distributed processing will likely continue to be a niche but high value feature of the product.
- Complexity:
This is the big one. As developers we tend to underestimate the environmental complexity of projects that we aren't directly involved in. It's natural to expect that on our first day on the job, we should be able to check out source code from a repository and be able to build the solution with minimal effort. If NCrunch has taught me one thing, it's that development environments can be infinitely complex in ways that can never be accurately guessed or assumed up-front. When NCrunch was first released, it was free of defects from every angle I'd thought to test it. It had been thrashed for over a year in real world environments, and I thought it was ready. As soon as it hit the real world, I then spent the next 18 months picking up on every crazy unexpected detail that I had never known about or even imagined could exist. The size of the .NET tool stack is immense, and it is moving at breakneck speed. There is no telling just what people are doing with their tests or what sort of dependencies their code has.
When you provide developers with a well tested product that behaves in a (mostly) predictable fashion, they are usually very effective at finding ways to use this product and configuring it for their own environment. As soon as you stop treating it as a product and start treating it as a service, you take on much more responsibility for that environmental complexity. For such a service to be worth paying for, you would need to commoditise these environments and try to find configurations that will work with minimal effort from your users. I expect that the support overhead of such a service would be very high.
Finally, it's worth being aware that the tests most likely in need of grid distribution are complex tests with external dependencies. Most external dependencies would be hard to provide with commodity environments.
My personal belief is that it just isn't worth it. When you weigh up the costs, the risk, and the payoff, I don't think such a service would be profitable.
