I have some long-running stability/performance tests, that are excluded in my default engine mode via a TestCategory.
I usually disable 'show passing tests' and enable the other three options (failing, unexecuted, ignored).
When i manually execute the stability tests successfully for the first time, they disappears from the Tests window.
After making some changes, that also impact the stability tests, all other tests are executed, but the stability tests remain in state succeded.
In my opinion, the state for these tests should change to unexecuted, and they should be displayed in the test window as unexecuted.
Maybe this can be changed
Regards
Wilhelm
Remco NCrunch Developer
#11159
07 Sep 2017 08:12 UTC
Hi Wilhelm,
There is a special state that the NCrunch engine uses for this. You'll notice that the status of these tests is set to Passing but Out Of Date, which is symbolised by a green question mark. Code coverage for the tests is then faded somewhat to show that this data is outdated.
To put the tests back into an unexecuted state wouldn't fit with the other attributes that tests have when they are in this state. Because the tests still have execution history, their data is still relevant. The tests still did pass when they were last run, they have code coverage, performance data, trace results, and a result that is still relevant even if it is outdated.
If I understand the way you are using the tool, do you want the tests to revert to an unexecuted state so that they show up in your Tests Window filter? It seems that to solve this, the better thing to do would be to simply make the Tests Window filter more configurable so that you can show tests that are passing but out of date, while hiding the tests that are passing but are current. What do you think?
There is a special state that the NCrunch engine uses for this. You'll notice that the status of these tests is set to Passing but Out Of Date, which is symbolised by a green question mark. Code coverage for the tests is then faded somewhat to show that this data is outdated.
To put the tests back into an unexecuted state wouldn't fit with the other attributes that tests have when they are in this state. Because the tests still have execution history, their data is still relevant. The tests still did pass when they were last run, they have code coverage, performance data, trace results, and a result that is still relevant even if it is outdated.
If I understand the way you are using the tool, do you want the tests to revert to an unexecuted state so that they show up in your Tests Window filter? It seems that to solve this, the better thing to do would be to simply make the Tests Window filter more configurable so that you can show tests that are passing but out of date, while hiding the tests that are passing but are current. What do you think?
Hi Remco,
'make the Tests Window filter more configurable so that you can show tests that are passing but out of date, while hiding the tests that are passing but are current.' seems to be exactly what I want.
'make the Tests Window filter more configurable so that you can show tests that are passing but out of date, while hiding the tests that are passing but are current.' seems to be exactly what I want.
Remco NCrunch Developer
#11162
07 Sep 2017 08:38 UTC
wilhelmmedetz wrote:Hi Remco,
'make the Tests Window filter more configurable so that you can show tests that are passing but out of date, while hiding the tests that are passing but are current.' seems to be exactly what I want.
Great :) Would you like to write up a feature request for this in uservoice that captures the requirement as it fits your needs?
Post a reply
Log in to reply.