Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Processing Time column in Tests window aggregate.
Micah71381
#1 Posted : Wednesday, January 1, 2014 9:45:05 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/18/2013(UTC)
Posts: 27
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
In the NCrunch Tests window there is a Processing Time column. It is my understanding that the column shows the time it took to execute the thing on that line. However, the view is a tree view, meaning tests are nested inside of their classes. In order to find the tests that took the longest, I have to expand every class (which takes a very long time when you have 100s of test classes) and scroll through the list to find the long-running tests.

If the "Processing Time" column for a class displayed the total time it took to process the entire class (meaning the overhead of the class plus the time taken for every nested class and test method inside of it then when I sorted by Processing time slow tests would cause their class to bubble up to the top when sorted and the method would then bubble up to the top of that class, making it easy to find my slow tests.

As an example, imagine you have two classes, each with one test method inside of it. Class A has a long overhead and takes 1 second to startup. Class B has very little overhead and starts up in 1 millisecond. Inside of class A you have method Foo (A.Foo) which is really fast. Inside of class B you have method Bar (B.Bar) which takes 10 seconds to execute. In this scenario, if you run both tests (A.Foo and B.Bar) and tell NCrunch to sort by Processing Time A will show up above B and both will have their methods collapsed (hidden). In order to find my slow (10 second) test I have to expand every class and manually look through all of the processing times to find the slow one.

The above example is simplified, but you can imagine how difficult this can become when you have 100s of test classes (some nested) and 1000s of tests.
Remco
#2 Posted : Wednesday, January 1, 2014 11:09:02 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for sharing this. I can see the annoying limitation here.

The Processing Time column is actually a massive over-simplification of several important data elements. It simultaneously represents the 'expected processing time' and the 'actual processing time'. On fixtures, it will report only the processing time related to the fixture as a whole (i.e. TestFixtureSetup, TestFixtureTearDown), while as you've mentioned, the processing time of child tests are reported separately.

Reporting the fixture setup times as a whole becomes problematic if the child test execution times are included. The only real answer is to split this column up, so we have a 'Total Processing Time', 'Expected Processing Time', and 'Last Processing Time'. Sooner or later this will need to happen.

If you're running V2, something that may help you here is to change the grouping setting on the Tests Window to 'Test only'. This will allow you to sort all tests in order of their processing time, making it easier to find the ones that take a long time to run. You could combine this with the list of fixtures sorted by processing time to help you find fixtures with a long setup/teardown time.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.027 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download