Performance Issues

I'm testing NCrunch with ReSharper and noticed that NCrunch is 20 times slower.

If you run this code using Resharper Unit Test I'm getting the rates per second close to 56million whereas NCrunch its around 2.4million.

    [TestFixture]
    public class Class1
    {
        [Test]
        public void PerformanceTest()
        {
            var items = new List<int>();
            var count = 1000000;
            var sw = Stopwatch.StartNew();
            for (var i = 0; i < count; i++)
            {
                items.Add(i);
            }
            sw.Stop();
            var ratePerSecond = count / sw.Elapsed.TotalSeconds;
            Console.Write($"{ratePerSecond}/s");
            Assert.IsTrue(ratePerSecond >= 50000000);
        }
    }

Result from NCrunch
2,492,096.3165321187/s Expected: True
But was: False

Result from ReSharper
56,679,060.48789335/s

Any idea why that much of a difference and how do I get it close to ReSharper's number?

I did try turning almost everything off and still no joy.

Hi thanks for posting.

NCrunch instruments the code while running the tests. There are certain areas where this can cause problems. The instrumentation can be suppressed in these areas by using comments.

The block can be started with the comment //ncrunch: no coverage start and ended with //ncrunch: no coverage end.

See here for further information: https://www.ncrunch.net/documentation/concepts_code-coverage-suppression

This works for me:

        public void TestMethod1()
        {
            var items = new List<int>();
            var count = 1000000;
            var sw = Stopwatch.StartNew();
            //ncrunch: no coverage start
            for (var i = 0; i < count; i++)
            {
                items.Add(i);
            }
            //ncrunch: no coverage end
            sw.Stop();
            var ratePerSecond = count / sw.Elapsed.TotalSeconds;
            Console.Write($"{ratePerSecond}/s");
            Assert.IsTrue(ratePerSecond >= 50000000);
        }

If it doesn't work for you, don't hesitate to let me know!

I see that made a massive improvement, is there away to have this as a global config or project config as I don't really want to keep adding this all over my unit test code if possible.

You can do that via the Instrument Output Assembly configuration.

See: https://www.ncrunch.net/documentation/reference_project-configuration_instrument-output-assembly

Thank you for the above information which does resolve the issue. Just one question I have is when you switch InstrumentOutputAssembly to false then I lose all the code coverage which sort of defeats the object of using NCrunch for me, is there any plans to improve the performance when InstrumentOutputAssembly is set to true?

For the time being I'm thinking to just ignore a handful of unit test that are failing due to performance as my workaround.

DHirani wrote:Thank you for the above information which does resolve the issue. Just one question I have is when you switch InstrumentOutputAssembly to false then I lose all the code coverage which sort of defeats the object of using NCrunch for me, is there any plans to improve the performance when InstrumentOutputAssembly is set to true?

Unfortunately, this is the trade-off. It isn't possible to obtain code coverage data without instrumenting the code, and it isn't possible to instrument the code without impacting its performance.

A possible middle-ground is to turn off the 'Analyse line execution times' setting, which will simplify the instrumentation so that it only marks the covered lines of code without tracking their execution times.

Generally the best approach is to use the coverage suppression comments to deactivate the instrumentation in areas of the code that are performance critical. In this way, you can usually resolve the performance problems while still having the value of the coverage tracking on the rest of the codebase.

Thank you for your response. Understand looks like comments is the best approach. Again thank you for your help.