MS Fakes cause slow tests

Hey all,

I am wondering if anyone else is having issues with the speed of ncrunch when using MS Fakes. I am using it to shim out SharePoint for TDD but the tests take a very long time to run. I think this might have something to do with the faked assembly being generated each time by ncrunch?

Any ideas?

Thanks

Aaron

Hi Aaron -

Thanks for posting!

As far as I'm aware, the main performance bottleneck in MS Fakes is in the generation of the fakes assemblies. For large assemblies, this build task does not perform well at all. I have little experience with SharePoint but I would expect that SharePoint reference libraries are not small. A tiny test library shimming out mscorlib.dll and system.dll will take upwards of 10 seconds to generate fakes assemblies even on a very high end machine.

The good news is that usually the generation of Fakes assemblies only needs to be performed once per workspace. You'll also notice that if you build your project using Visual Studio, the fakes assemblies don't get generated with every build.

Because NCrunch creates multiple workspaces per test project in order to perform parallel testing, it will probably need to build the fakes assemblies multiple times before the workspaces are saturated enough to allow for good performance. If you have your max number of processing threads set to 5, NCrunch will need to build at least 5 workspaces before the performance improves. This means it'll need to build your project at least 5 times with sluggish speed.

Try keeping an eye on your project build times. Make sure you have the 'Processing Time' column showing in your tests window, and turn on the 'Show passing tests' option so that all your projects are showing. What kind of build times do you see next to the test projects using fakes? Do the build times improve after you've been using the engine for a while?


Cheers,

Remco

Thanks Remco,

I will keep an eye on the build times and let you know. I will have to do a bit more investigation if this is a good idea, but would it be possible to have fake assemblies compiled once and coped around to the workspaces?

Aaron

Hi Aaron -

Yes - this is possible as a feature. In fact, you can probably rig NCrunch up to do this for you already. Assuming that you've already built the fakes assemblies using Visual Studio, you can include them using the 'additional files to include' configuration setting at project-level. In this way, NCrunch will copy the pre-built assemblies to the workspace, and hopefully MSFakes will decide it doesn't need to rebuild.

That is a very good idea, I will give that a go and let you know.

That is so much better, I included all the files in the 'FakesAssemblies' directory(this is a sub direct of the project where fake assemblies have been added). As far as I can tell this is where VS puts the generated assemblies.

Thanks

Aaron

Great! Thanks for letting me know :)

Also Remco, thanks you so much for ncrunch. It has helped me to use Red Green Refactor all the time.

Aaron

You're welcome :)

Finally got around to Blogging about this and thought I would share it here.

http://blog.gravypower.net/ncrunch-sharepoint-and-microsoft-fakes/

Hope that helps someone :D

Aaron