Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Automatically run impacted tests without automatically running all tests
Options
Previous Topic Next Topic
distilled
#1 Posted : Saturday, February 6, 2021 6:02:27 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/3/2012(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 6 time(s) in 6 post(s)
I use NCrunch in a solution with a large number of tests (~20k). I prefer to leave NCrunch in the "Automatically run impacted tests" mode. However, when I select this mode NCrunch automatically runs all unignored tests right away, before only running tests that are impacted by my changes. Is there a way to disable this initial run without just ignoring all of my tests and then selectively unignoring the ones that I think I might impact?
Back to top
Remco
#2 Posted : Saturday, February 6, 2021 10:56:10 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Hi, thanks for posting.

If you're using the 'Run impacted tests automatically, others manually' mode, NCrunch should only run tests when it detects that they are impacted by IL-level changes in your code.

This includes after the engine is started. NCrunch does this by storing the hash state of your codebase in its cache file and comparing this to the state of the codebase after the engine has switched on. Tests that cover the changes areas are considered impacted and are executed.

The same process is used when the mode is switched to this one.

So I guess what I'm saying is that if things are working properly, then this is the sort of behaviour you should already be seeing.

Can you confirm exactly which engine mode you are switching to and what its settings are?
Back to top
distilled
#3 Posted : Sunday, February 7, 2021 11:56:29 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/3/2012(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 6 time(s) in 6 post(s)
Quote:
So I guess what I'm saying is that if things are working properly, then this is the sort of behaviour you should already be seeing.


Yep, I can confirm that that's what I'm seeing. I guess in my experience the file hash very rarely matches what NCrunch has cached, and most of the time when I launch VS I need to quickly disable NCrunch or tell it to ignore everything so that it doesn't try to start running all of the tests.
Back to top
Remco
#4 Posted : Monday, February 8, 2021 3:59:54 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
distilled;15311 wrote:

Yep, I can confirm that that's what I'm seeing. I guess in my experience the file hash very rarely matches what NCrunch has cached, and most of the time when I launch VS I need to quickly disable NCrunch or tell it to ignore everything so that it doesn't try to start running all of the tests.


The hashing system is very sensitive to even the smallest of IL changes in the code. If you regularly update from a VCS before opening your codebase, you'll likely see tests get triggered every time a changeset comes down. If you have a lot of long running tests that NCrunch is running a bit too eagerly, you might want to consider setting up your own engine mode that separates them out to be executed manually.
Back to top
1 user thanked Remco for this useful post.
distilled on 2/8/2021(UTC)
distilled
#5 Posted : Monday, February 8, 2021 3:16:56 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/3/2012(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 6 time(s) in 6 post(s)
I'll give customizing the engine a shot. Thanks!
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.032 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download