Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Add Pin/Unpin tests on assembly level
Marqus
#1 Posted : Thursday, February 23, 2012 8:21:53 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/22/2012(UTC)
Posts: 38
Location: Poland

Thanks: 2 times
Was thanked: 6 time(s) in 6 post(s)
I've got solution with many projects, so I pin only those test that I need at this moment. If I want to pin all tests from assembly, i have to expand this assembly, select all test and pin them. I would like to pin assembly - all tests in this assembly should be pinned/unpinned automatically.
1 user thanked Marqus for this useful post.
jomtois on 1/6/2014(UTC)
Remco
#2 Posted : Thursday, February 23, 2012 8:29:37 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
This seems like a reasonable request - and it would work well with the tests window structure. I'll see what I can do.
Marqus
#3 Posted : Thursday, February 23, 2012 11:39:30 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/22/2012(UTC)
Posts: 38
Location: Poland

Thanks: 2 times
Was thanked: 6 time(s) in 6 post(s)
And one more: "unpin all tests"
Remco
#4 Posted : Thursday, February 23, 2012 8:09:58 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Thanks - yes, this is one that I've also been looking at! It's good to see the pinned tests getting some use. I've seen limited feedback on them up until today, and I was wondering if most people didn't know about them :)
Marqus
#5 Posted : Friday, February 24, 2012 7:33:56 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/22/2012(UTC)
Posts: 38
Location: Poland

Thanks: 2 times
Was thanked: 6 time(s) in 6 post(s)
I've started using pinning because of the problem described in: http://forum.ncrunch.net...r-extended-periods.aspx - pinning only interesting tests makes this problem less frequent. But this is using pinning as a workaround, and pinning should be used for something else.
As I understand, ignoring tests should be used to 'switch off' tests that aren't compatible with nCrunch, and this is a global setting (saved in *.ncrunchproject).
Pinning test is used for showing nCrunch, which tests are important now, for a concrete developer, so pinning should be saved locally for user (I didn't found this in *.ncrunchsolution.user, so I don't know exacly how this works). In this scenario, I think that there is some place for improvment:
- another Engine mode: "Run pinned tests at high priority, others in background"
- in "Run pinned tests automatically, others manually" way to run all tests (pinned and not pinned) - tests tree doesn't have common root, so you can't run them in one click. I work with 'NCrunch Test' window showing only failing tests (in this mode it shows pinned passing tests too - very handy), so selecting all tests with Ctrl+A doesn't do the trick, because I don't see unpined tests.
Remco
#6 Posted : Friday, February 24, 2012 9:43:51 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Thanks for the feedback :)

The pinned test status is stored in the .cache file, which should be local to every user. The upcoming 1.38b release will add a few feature where engine modes can be customised (i.e. you can set criteria for which tests you want run automatically), and I have a feeling this will open up some new options for you for running partially continuous.

Right now the engine does actually give a higher priority to pinned tests, so they'll generally be run earlier than the others regardless of which mode you're in.

If you use the 'Run all tests' option, NCrunch will queue ALL the tests in your solution (regardless of pinned status). Does this help you at all?

I've updated the message thread in http://forum.ncrunch.net...r-extended-periods.aspx with some more information about how you can reduce the impact of the problem described. It's a particularly painful problem but hopefully I'll have a fix out soon.


Cheers,

Remco
Marqus
#7 Posted : Friday, February 24, 2012 9:48:55 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/22/2012(UTC)
Posts: 38
Location: Poland

Thanks: 2 times
Was thanked: 6 time(s) in 6 post(s)
Thanks for reply

"Run all tests" will work :)
Customised engine modes sounds interesting, I will wait for them.
Remco
#8 Posted : Wednesday, March 7, 2012 12:30:11 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
For anyone interested, the ability to customise engine modes has just been released with NCrunch 1.38b.
jomtois
#9 Posted : Monday, January 6, 2014 7:32:09 PM(UTC)
Rank: Member

Groups: Registered
Joined: 9/24/2012(UTC)
Posts: 10
Location: Newport, VT

Thanks: 5 times
Was thanked: 2 time(s) in 2 post(s)
I would still like to see the ability to pin all from the assembly level. As it stands now, when I expand the assembly, each class is expanded with its tests, so I have to roll them all up then select them and scroll. On perhaps a tangential note, I use git for source control so I switch branches frequently. I suspect this causes the NCrunch cache to become invalid and I have to resync it, which loses the pin statuses of my tests. Therefore, it would be a good feature for me to easily restore the pin statuses. Especially if I just want to pick an assembly or two germane to what I'm working on.

Thanks
Remco
#10 Posted : Tuesday, January 7, 2014 12:28:13 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi, thanks for this feedback.

Do you might if I ask how/why you're using the pinned test status? The original idea behind pinned tests was that they would be used fairly transiently. I'm wondering if there may be a larger feature gap here.


Cheers,

Remco
jomtois
#11 Posted : Tuesday, January 7, 2014 2:13:27 PM(UTC)
Rank: Member

Groups: Registered
Joined: 9/24/2012(UTC)
Posts: 10
Location: Newport, VT

Thanks: 5 times
Was thanked: 2 time(s) in 2 post(s)
No problem, I will explain how I use the pinned tests, maybe there is better way to accomplish the same thing that I have not tried yet. Usually I leave my engine mode to "Pinned tests automatically/others manually". So what I will do is manually run all tests (minus ignored of course) when I start and end working on a feature. I will then pin the tests in the namespaces I know will be impacted, so just those will fire off. If I run all my tests all the time it can lag my system, and I when I have used the "Impacted tests automatically" mode, it doesn't always run all the right tests and I end up not trusting the red/green indicators as I'm coding along. So I found the pinning to work well since it gives a focused subset to constantly run rather than all tests. If I used Ignore/Unignore to achieve the same thing then it is then harder to manually run everything without having to go unignore. I tend to use Ignore for my integration tests which I only have run on my CI server.

I hope that is clear. If you have suggestions I'm definitely open to them. I really like NCrunch and this pinning/unpinning from the namespace is just a very minor annoyance which I thought would be a fairly simple feature to implement (resurrecting year old post I know, but when I google searched it that is what I found).

Thanks!
Remco
#12 Posted : Tuesday, January 7, 2014 10:16:37 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Thanks! This is helpful to know. What you've described does fit the intention of the design... I guess the only thing is that your situation does involve grouping of similar tests at higher level than fixture.

When the namespace groupings were introduced in V2, the intention was to make these work with the pinned tests. Unfortunately, this was found to be surprisingly very complex. This is because of the derived lifespan of these objects and their attachment to the different grouping dimensions in the Tests Window (i.e. you'll notice the behaviour changes depending upon which grouping mode is selected in the UI).

Implementing pinned tests at project/namespace level is much easier if the flag is set for all tests/fixtures that are visible under the project/namespace, as opposed to setting at the flag at the project/namespace level itself. Unfortunately, this does also mean that newly created tests under pinned projects or namespaces won't automatically inherit the pinned status. Still, this is probably the best I can immediately offer for this feature without doing some replanning. Do you think such an approach would work for you?


Cheers,

Remco
jomtois
#13 Posted : Wednesday, January 8, 2014 1:54:41 PM(UTC)
Rank: Member

Groups: Registered
Joined: 9/24/2012(UTC)
Posts: 10
Location: Newport, VT

Thanks: 5 times
Was thanked: 2 time(s) in 2 post(s)
I guess I don't care so much about the namespace groupings, the test fixtures within each project/namespace is fine. I definitely like how it works now with having the fixture pinned it automatically pins and runs new tests as they are added. I would probably not care whether it auto-added new testfixtures under the parent project if the project was able to have a pin status attached to it.

I'm curious to know what you mean by changing the grouping mode in the UI? Or does that just mean the menu items are different depending on which level is selected?

I would be interested to know if there was a way to turn off the namespace groupings.

Actually for me, the most annoying thing is that the tree is auto-expanded out to the individual test level by default when it loads tests initially (assuming NCrunch not using cache due to switching branches in version control or something, so it loses prior pin statuses). Thus I open the tests window, navigate to the project, and have to click the roll-up button of every test fixture to fit them on my screen. From that point, I can just highlight the whole lot of testfixtures under a single project, right click, and pin them. Maybe a solution would be just to implement some "Collapse children" function to the context menu, and even a "Select All / Deselect All" functionality then not having to mess with namespace groupings issues.

Maybe this is very specific to my situation. I can provide screenshots or a more detailed description of my projects layout if needed.

Thanks for your responses,
Jon

Remco
#14 Posted : Wednesday, January 8, 2014 11:51:01 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi Jon -

The grouping I'm referring to is the dropdown option at the top of the Tests Window that lets you choose how you want the tree to be structured. This is a new feature in V2, so I'm sorry for the confusion if you're not running on this version yet.

The problem you're experiencing with the loss of test data after a branch switch is probably because of differences in the test names between the branches. For example, if you were to rename the root namespace containing all your tests in one branch only, the test names would be different between the branches. This would be seriously painful, because all code coverage information and remembered times of executed tests would be lost every time you switched branches. In this case, I would suggest changing the name of your solution (.sln) file in one of the branches. NCrunch always names its cache directory according to the name of the solution, so by doing this, you'll have separated cache data for each of your branches. This means you won't need to deal with ever expanding tree nodes and lost data every time you switch branches.

There are also plans for an "expand/collapse all " button on the Tests Window :)


Cheers,

Remco
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.090 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download