Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

MS Fakes cause slow tests
gravypower
#1 Posted : Monday, February 10, 2014 4:59:09 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/14/2013(UTC)
Posts: 13
Location: Australia

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
Hey all,

I am wondering if anyone else is having issues with the speed of ncrunch when using MS Fakes. I am using it to shim out SharePoint for TDD but the tests take a very long time to run. I think this might have something to do with the faked assembly being generated each time by ncrunch?

Any ideas?

Thanks

Aaron
Remco
#2 Posted : Tuesday, February 11, 2014 12:15:45 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi Aaron -

Thanks for posting!

As far as I'm aware, the main performance bottleneck in MS Fakes is in the generation of the fakes assemblies. For large assemblies, this build task does not perform well at all. I have little experience with SharePoint but I would expect that SharePoint reference libraries are not small. A tiny test library shimming out mscorlib.dll and system.dll will take upwards of 10 seconds to generate fakes assemblies even on a very high end machine.

The good news is that usually the generation of Fakes assemblies only needs to be performed once per workspace. You'll also notice that if you build your project using Visual Studio, the fakes assemblies don't get generated with every build.

Because NCrunch creates multiple workspaces per test project in order to perform parallel testing, it will probably need to build the fakes assemblies multiple times before the workspaces are saturated enough to allow for good performance. If you have your max number of processing threads set to 5, NCrunch will need to build at least 5 workspaces before the performance improves. This means it'll need to build your project at least 5 times with sluggish speed.

Try keeping an eye on your project build times. Make sure you have the 'Processing Time' column showing in your tests window, and turn on the 'Show passing tests' option so that all your projects are showing. What kind of build times do you see next to the test projects using fakes? Do the build times improve after you've been using the engine for a while?


Cheers,

Remco
1 user thanked Remco for this useful post.
gravypower on 2/11/2014(UTC)
gravypower
#3 Posted : Tuesday, February 11, 2014 8:50:25 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/14/2013(UTC)
Posts: 13
Location: Australia

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
Thanks Remco,

I will keep an eye on the build times and let you know. I will have to do a bit more investigation if this is a good idea, but would it be possible to have fake assemblies compiled once and coped around to the workspaces?

Aaron
Remco
#4 Posted : Tuesday, February 11, 2014 9:02:37 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi Aaron -

Yes - this is possible as a feature. In fact, you can probably rig NCrunch up to do this for you already. Assuming that you've already built the fakes assemblies using Visual Studio, you can include them using the 'additional files to include' configuration setting at project-level. In this way, NCrunch will copy the pre-built assemblies to the workspace, and hopefully MSFakes will decide it doesn't need to rebuild.
1 user thanked Remco for this useful post.
gravypower on 2/11/2014(UTC)
gravypower
#5 Posted : Tuesday, February 11, 2014 9:04:04 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/14/2013(UTC)
Posts: 13
Location: Australia

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
That is a very good idea, I will give that a go and let you know.
gravypower
#6 Posted : Tuesday, February 11, 2014 9:11:53 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/14/2013(UTC)
Posts: 13
Location: Australia

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
That is so much better, I included all the files in the 'FakesAssemblies' directory(this is a sub direct of the project where fake assemblies have been added). As far as I can tell this is where VS puts the generated assemblies.

Thanks

Aaron
Remco
#7 Posted : Tuesday, February 11, 2014 9:14:46 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Great! Thanks for letting me know :)
gravypower
#8 Posted : Tuesday, February 11, 2014 9:15:03 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/14/2013(UTC)
Posts: 13
Location: Australia

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
Also Remco, thanks you so much for ncrunch. It has helped me to use Red Green Refactor all the time.

Aaron
Remco
#9 Posted : Tuesday, February 11, 2014 9:16:08 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
You're welcome :)
gravypower
#10 Posted : Tuesday, February 25, 2014 7:36:54 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/14/2013(UTC)
Posts: 13
Location: Australia

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
Finally got around to Blogging about this and thought I would share it here.

http://blog.gravypower.n...nt-and-microsoft-fakes/

Hope that helps someone :D

Aaron
1 user thanked gravypower for this useful post.
Remco on 2/25/2014(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.050 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download