Tests that are marked as [Explicit] are being run by NCrunch. Version 2.15.0.9 in Visual Studio 2013 on Windows 8.1.
Repro:
1. Disable NCrunch.
1. Make new solution with new CSharp DLL project.
2. Add NUnit and write a new explicit test.
3. Enable NCrunch. Select "Yes - run my tests alongside each other!", "Let me run my tests manually", and "Let my tests run - I will ignore them as I need to" when asked by the wizard.
4. Wait while NCrunch loads up. The test should be visible with status "Pending, Impacted". The top bar would read "No tests are queued for execution. Monitoring 1 test, with no tests ignored."
5. Press "Run all tests visible here". Boom, the explicit test has been executed.
The same behaviour occurs when disabling/reenabling NCrunch on existing projects, or writing a new explicit test on a project that already has NCrunch enabled.
I would expect that explicit tests should basically never be run by NCrunch, even if the user clicks "Run all tests visible here" or "Run all tests".
Remco NCrunch Developer
#7411
09 Jun 2015 23:31 UTC
Hi, thanks for posting.
Explicit tests have a special status with NCrunch. Unlike other tests, the following rules will apply to them:
1. Explicit tests will never be run automatically due to changes made to a solution
2. Explicit tests will never be run when the 'Run all tests' option is selected
3. Explicit tests WILL be run when they are explicitly selected for execution (i.e. right click the test and choose to run)
4. Explicit tests WILL be run if they are selected for execution via the 'Run all tests visible here' option, as this option will explicitly select all tests visible in the tests window for execution in the same manner as though they were selected and set to run.
There is an edge case where an explicit test can also be run incidently by the engine due to status changes. This is because of the timing in which the test is marked as explicit. If you create a new unexplicit test, NCrunch will place it in the pipeline as soon as the test is detected. If you then add an [Explicit] flag to the test, NCrunch will still execute the originally placed test in the pipeline because the analysis step has not yet reported the test as explicit. From the next time any changes are made to the solution, the test will be considered explicit and won't be run automatically.
Explicit tests have a special status with NCrunch. Unlike other tests, the following rules will apply to them:
1. Explicit tests will never be run automatically due to changes made to a solution
2. Explicit tests will never be run when the 'Run all tests' option is selected
3. Explicit tests WILL be run when they are explicitly selected for execution (i.e. right click the test and choose to run)
4. Explicit tests WILL be run if they are selected for execution via the 'Run all tests visible here' option, as this option will explicitly select all tests visible in the tests window for execution in the same manner as though they were selected and set to run.
There is an edge case where an explicit test can also be run incidently by the engine due to status changes. This is because of the timing in which the test is marked as explicit. If you create a new unexplicit test, NCrunch will place it in the pipeline as soon as the test is detected. If you then add an [Explicit] flag to the test, NCrunch will still execute the originally placed test in the pipeline because the analysis step has not yet reported the test as explicit. From the next time any changes are made to the solution, the test will be considered explicit and won't be run automatically.
Post a reply
Log in to reply.