Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Processing Queue Times: What do they mean?
Options
Previous Topic Next Topic
Grendil
#1 Posted : Saturday, April 15, 2017 6:59:13 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2017(UTC)
Posts: 54
Location: United States of America

Thanks: 22 times
Was thanked: 11 time(s) in 10 post(s)
I'm troubleshooting some test performance issues. I'm using the NCrunch Processing Queue to help measure progress. I'm occasionally confused by what it's telling me. For example, I just had an entry that read:
Task: "Run 1 Test"
Status: "Complete"
Actual Processing Time 7.381
Expected Processing Time 0.605

It contains one test as expected, and that one reads:
Status: Passed
Actual Processing Time: 0.646
Expected Processing Time: 0.646

So this raises two questions:
(1) Is it odd that the task processing time was so long compared to the processing time for the single test it contained?
(2) Why was the expected processing time for the task actually less than the expected processing time for the test it contained?

(Just FYI: I did try to RTFM first, but got no hits on "actual processing time".)
Back to top
Remco
#2 Posted : Monday, April 17, 2017 11:06:07 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Hi, thanks for posting.

The processing queue isn't really designed for this kind of performance analysis. Instead, I suggest you use this feature: http://www.ncrunch.net/documentation/reference_global-configuration_track-engine-performance.

The expected processing time for a test task in the processing queue doesn't consider additional overhead, such as time taken to initialise the test host process and test framework. The above engine performance tracking should provide more information about what is taking this time.
Back to top
1 user thanked Remco for this useful post.
Grendil on 4/18/2017(UTC)
Grendil
#3 Posted : Monday, April 17, 2017 11:44:52 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2017(UTC)
Posts: 54
Location: United States of America

Thanks: 22 times
Was thanked: 11 time(s) in 10 post(s)
Remco;10277 wrote:
Hi, thanks for posting.

The processing queue isn't really designed for this kind of performance analysis. Instead, I suggest you use this feature: http://www.ncrunch.net/documentation/reference_global-configuration_track-engine-performance.

The expected processing time for a test task in the processing queue doesn't consider additional overhead, such as time taken to initialize the test host process and test framework. The above engine performance tracking should provide more information about what is taking this time.


I've used that too, however I've appreciated the tabular view in the Processing Queue for reviewing a lot of NCrunch behavior (both tests and build/analyse steps) at once. Are the test execution times listed in the Tests window "valid" for performance analysis? (It is a bit less convenient to study the engine tab for each test individually, but it's certainly helpful.)
Back to top
Remco
#4 Posted : Monday, April 17, 2017 11:50:09 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Grendil;10281 wrote:

I've used that too, however I've appreciated the tabular view in the Processing Queue for reviewing a lot of NCrunch behavior (both tests and build/analyse steps) at once. Are the test execution times listed in the Tests window "valid" for performance analysis? (It is a bit less convenient to study the engine tab for each test individually, but it's certainly helpful.)


Yes. Examining the performance of individual tests should definitely be done from the Tests Window. This is the most accurate way to determine the true running time of any specific test.

Something else you can do is to set your Log Verbosity to 'Detailed', then examine the time-marked logs in the Processing Queue trace output. These logs aren't intended for performance analysis but the timestamps are still useful and they can also reveal quite a bit of internal information about what the engine is doing. Note that the logs themselves are considered 'internal' and are not intended to be user friendly.
Back to top
1 user thanked Remco for this useful post.
Grendil on 4/18/2017(UTC)
Grendil
#5 Posted : Tuesday, April 18, 2017 12:16:04 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2017(UTC)
Posts: 54
Location: United States of America

Thanks: 22 times
Was thanked: 11 time(s) in 10 post(s)
Ah, great idea. No worries about "unfriendly" logs, I've found NCrunch extremely kind to new users overall. Thanks!
Back to top
1 user thanked Grendil for this useful post.
Remco on 4/18/2017(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.041 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download