Reset status on treeview node with children does not work

I've got a a few tests which, sadly, are unstable on my machine. Fixing that is in techdebt and will eventually gotten round to, but because they are stable enough in the CI (strangely), this has no high priority.

So every once in a while they or a few of them fail. They are all contained in the same test project, in different nested namespaces.

When I click on the treeview node for the test project and select "Advanced | Reset selected tests status", this does have no effect at all.
If I "reset" the tests individually, though, it works just fine.

While not a big issue, it would be nice if that could be fixed.

Thanks,
MR

Hi, thanks for sharing this. The reset feature was designed to work on tests, and projects weren't considered .. but there is no reason why they couldn't be. I'll take a look to see if we can improve this.

I think the problem you're describing with intermittently failing tests is very common. We introduced Churn Mode to try and help with solving this, and we have more features on the way that I hope will make it easier.

Churn mode has already helped a lot and often!

The tests where these things happen are usually integration tests that use stuff like MassTransit; or tests that try to ensure a certain performance threshold is met.

In my particular case, it's weird though, because they don't fail on the CI - only on developer machines. While this is of course the lesser of the two evils, it's the very first time I've seen this happening - usually, it's either both or only the CI.

As for the "Reset selected tests" feature, I should also mention that if I select multiple tests and then click "Reset", only the one where my mouse cursor was when opening the context menu is actually reset.

Again, this whole topic is not super urgent/important, but if you could adjust the behavior in one of your future releases, it would be nice.

Thanks once again for the most important VS plugin in my development workflow (seriously, it beats even R# because 80% of what R# does, I could replicate with a mix of builtin functionality and other extensions, while NCrunch is really unique in its feature set).

MR

MaxRonin wrote:In my particular case, it's weird though, because they don't fail on the CI - only on developer machines.

FWIW, in my experience it is often due to incorrect setups/teardowns of a test (e.g. test2 relies on test1's setup), or they rely on static test data that is not reset properly.

One thing that may be helpful in the meantime could be to utilize [Isolated] NCrunch attribute to force the flaky tests run in a new process (thus ensuring any static initialization is done before test is executed for instance).

I wasn't aware of the [Isolated] attribute - this is very interesting, thank you!