Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Explicit tests are getting run
jamezor
#1 Posted : Tuesday, June 9, 2015 8:13:05 PM(UTC)
Rank: Member

Groups: Registered
Joined: 5/9/2012(UTC)
Posts: 13
Location: Auckland

Was thanked: 1 time(s) in 1 post(s)
Tests that are marked as [Explicit] are being run by NCrunch. Version 2.15.0.9 in Visual Studio 2013 on Windows 8.1.

Repro:
1. Disable NCrunch.
1. Make new solution with new CSharp DLL project.
2. Add NUnit and write a new explicit test.
3. Enable NCrunch. Select "Yes - run my tests alongside each other!", "Let me run my tests manually", and "Let my tests run - I will ignore them as I need to" when asked by the wizard.
4. Wait while NCrunch loads up. The test should be visible with status "Pending, Impacted". The top bar would read "No tests are queued for execution. Monitoring 1 test, with no tests ignored."
5. Press "Run all tests visible here". Boom, the explicit test has been executed.

The same behaviour occurs when disabling/reenabling NCrunch on existing projects, or writing a new explicit test on a project that already has NCrunch enabled.

I would expect that explicit tests should basically never be run by NCrunch, even if the user clicks "Run all tests visible here" or "Run all tests".
Remco
#2 Posted : Tuesday, June 9, 2015 11:31:20 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for posting.

Explicit tests have a special status with NCrunch. Unlike other tests, the following rules will apply to them:

1. Explicit tests will never be run automatically due to changes made to a solution
2. Explicit tests will never be run when the 'Run all tests' option is selected
3. Explicit tests WILL be run when they are explicitly selected for execution (i.e. right click the test and choose to run)
4. Explicit tests WILL be run if they are selected for execution via the 'Run all tests visible here' option, as this option will explicitly select all tests visible in the tests window for execution in the same manner as though they were selected and set to run.

There is an edge case where an explicit test can also be run incidently by the engine due to status changes. This is because of the timing in which the test is marked as explicit. If you create a new unexplicit test, NCrunch will place it in the pipeline as soon as the test is detected. If you then add an [Explicit] flag to the test, NCrunch will still execute the originally placed test in the pipeline because the analysis step has not yet reported the test as explicit. From the next time any changes are made to the solution, the test will be considered explicit and won't be run automatically.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.027 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download