Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Performance Issues
Options
Previous Topic Next Topic
DHirani
#1 Posted : Friday, December 17, 2021 2:56:32 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 12/17/2021(UTC)
Posts: 5
Location: United Kingdom

Thanks: 1 times
I'm testing NCrunch with ReSharper and noticed that NCrunch is 20 times slower.

If you run this code using Resharper Unit Test I'm getting the rates per second close to 56million whereas NCrunch its around 2.4million.

Code:
    [TestFixture]
    public class Class1
    {
        [Test]
        public void PerformanceTest()
        {
            var items = new List<int>();
            var count = 1000000;
            var sw = Stopwatch.StartNew();
            for (var i = 0; i < count; i++)
            {
                items.Add(i);
            }
            sw.Stop();
            var ratePerSecond = count / sw.Elapsed.TotalSeconds;
            Console.Write($"{ratePerSecond}/s");
            Assert.IsTrue(ratePerSecond >= 50000000);
        }
    }


Result from NCrunch
2,492,096.3165321187/s Expected: True
But was: False

Result from ReSharper
56,679,060.48789335/s

Any idea why that much of a difference and how do I get it close to ReSharper's number?

I did try turning almost everything off and still no joy.
Back to top
 |  Edit by user
michaelkroes
#2 Posted : Friday, December 17, 2021 3:07:04 PM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 280
Location: Netherlands

Thanks: 124 times
Was thanked: 63 time(s) in 60 post(s)
Hi thanks for posting.

NCrunch instruments the code while running the tests. There are certain areas where this can cause problems. The instrumentation can be suppressed in these areas by using comments.

The block can be started with the comment //ncrunch: no coverage start and ended with //ncrunch: no coverage end.

See here for further information: https://www.ncrunch.net/documentation/concepts_code-coverage-suppression

This works for me:

Code:

        public void TestMethod1()
        {
            var items = new List<int>();
            var count = 1000000;
            var sw = Stopwatch.StartNew();
            //ncrunch: no coverage start
            for (var i = 0; i < count; i++)
            {
                items.Add(i);
            }
            //ncrunch: no coverage end
            sw.Stop();
            var ratePerSecond = count / sw.Elapsed.TotalSeconds;
            Console.Write($"{ratePerSecond}/s");
            Assert.IsTrue(ratePerSecond >= 50000000);
        }


If it doesn't work for you, don't hesitate to let me know!
Back to top
DHirani
#3 Posted : Friday, December 17, 2021 3:36:12 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 12/17/2021(UTC)
Posts: 5
Location: United Kingdom

Thanks: 1 times
I see that made a massive improvement, is there away to have this as a global config or project config as I don't really want to keep adding this all over my unit test code if possible.
Back to top
 |  Edit by user
michaelkroes
#4 Posted : Friday, December 17, 2021 3:43:04 PM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 280
Location: Netherlands

Thanks: 124 times
Was thanked: 63 time(s) in 60 post(s)
You can do that via the Instrument Output Assembly configuration.

See: https://www.ncrunch.net/...strument-output-assembly
Back to top
DHirani
#5 Posted : Friday, December 17, 2021 4:15:38 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 12/17/2021(UTC)
Posts: 5
Location: United Kingdom

Thanks: 1 times
Thank you for the above information which does resolve the issue. Just one question I have is when you switch InstrumentOutputAssembly to false then I lose all the code coverage which sort of defeats the object of using NCrunch for me, is there any plans to improve the performance when InstrumentOutputAssembly is set to true?

For the time being I'm thinking to just ignore a handful of unit test that are failing due to performance as my workaround.
Back to top
 |  Edit by user
Remco
#6 Posted : Monday, December 20, 2021 9:07:53 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
DHirani;15892 wrote:
Thank you for the above information which does resolve the issue. Just one question I have is when you switch InstrumentOutputAssembly to false then I lose all the code coverage which sort of defeats the object of using NCrunch for me, is there any plans to improve the performance when InstrumentOutputAssembly is set to true?


Unfortunately, this is the trade-off. It isn't possible to obtain code coverage data without instrumenting the code, and it isn't possible to instrument the code without impacting its performance.

A possible middle-ground is to turn off the 'Analyse line execution times' setting, which will simplify the instrumentation so that it only marks the covered lines of code without tracking their execution times.

Generally the best approach is to use the coverage suppression comments to deactivate the instrumentation in areas of the code that are performance critical. In this way, you can usually resolve the performance problems while still having the value of the coverage tracking on the rest of the codebase.
Back to top
DHirani
#7 Posted : Monday, December 20, 2021 9:29:46 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 12/17/2021(UTC)
Posts: 5
Location: United Kingdom

Thanks: 1 times
Thank you for your response. Understand looks like comments is the best approach. Again thank you for your help.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.050 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download