Ncrunch is exhibiting a number of odd behaviours for me since upgrading to VS2022 preview and Ncrunch 4.10...
1) A test marked as Explicit is being run. (I'm not editing this test file so it's not a matter of Ncrunch seeing the test before the Explicit atttribute is added)
2) I'm not convinced Ncrunch is obeying the "IsEnabled" flag in distributed processing. I turned off the local server and yet a lot of the builds steps still seem to be run on it (though perhaps it's just that this flag is only checked at startup?)
3) Possibly related to #2, I'm finding that Ncrunch is sometimes hanging on rebuilds... For example Ncrunch is currently stuck on "build in progress" and the top 4 tasks in the processing queue are "Build Assembly" on the local server with status "Reprocessing".
If I get a bit of time I'll try reverting to VS2019 to see if the issues is with the Visual Studio upgrade.
Remco NCrunch Developer
#15745
27 Oct 2021 00:02 UTC
Hi, thanks for sharing these issues.
In the cases of 1 and 2, the changes for VS2022 haven't included any changes to these areas of NCrunch. I think there might have been a change in usage here that is surfacing something unexpected.
Explicit tests will be run by NCrunch when they are specifically targeted in the NCrunch Tests Window. They won't run automatically. It should be possible to check whether NCrunch has correctly detected the 'IsExplicit' tag by adding the 'Is Explicit' column to the Tests Window and checking the status of the child test. If there's a problem with test discovery, it should show up here.
The 'Is Enabled' flag can be disrespected by NCrunch when a test is targeted for priority execution and no grid node is able to run it. This will happen if you try to debug any test, and doing so will cause NCrunch to build any related projects locally so that it can build an environment for the test run.
Issue 3 is a cause for concern and could be related to the change in underlying platform. Does NCrunch give you any error information for these tasks under the Processing Queue Window?
In the cases of 1 and 2, the changes for VS2022 haven't included any changes to these areas of NCrunch. I think there might have been a change in usage here that is surfacing something unexpected.
Explicit tests will be run by NCrunch when they are specifically targeted in the NCrunch Tests Window. They won't run automatically. It should be possible to check whether NCrunch has correctly detected the 'IsExplicit' tag by adding the 'Is Explicit' column to the Tests Window and checking the status of the child test. If there's a problem with test discovery, it should show up here.
The 'Is Enabled' flag can be disrespected by NCrunch when a test is targeted for priority execution and no grid node is able to run it. This will happen if you try to debug any test, and doing so will cause NCrunch to build any related projects locally so that it can build an environment for the test run.
Issue 3 is a cause for concern and could be related to the change in underlying platform. Does NCrunch give you any error information for these tasks under the Processing Queue Window?
Thanks Remco, I saw this again this morning (after upgrading to 2022 pv7).
I can't come up with an explanation of why the original explicit test was run (it's *possible* I accidentally right-clicked and chose "Run") but your explanation does at least satisfy item 2.
As far as the "hanging" (#3) is concerned.
I'm not sure if this a "reliable" recipe and some steps may be red herrings. but I seem to be able to reproduce the issue sometimes by:
- Unclicking "Is Enabled" for the local server in the distributed processing tab.
- Restarting ncrunch (disable then enable)
- Let all tests run (and successfully pass)
- Right click on a single test and Choose "Debug/Run selected test as high priority task"
At that point the processing queue shows 3 x "Build Assembly" tasks with "Processing" status. All using the same process id. If it's relevant, the declared task-capacity of "local" is 2.
> Does NCrunch give you any error information for these tasks under the Processing Queue Window?
If I click on the tasks, no additional information is shown in the split panel below the list. (Maybe there's another way to get some insight into the task status?)
Note that if I _enable_ local and restart ncrunch, the same projects can be built fine and run/debugged.
So the workaround _seems_ to be to keep the local server enabled but I'll post back if that turns out to be insufficient.
I can't come up with an explanation of why the original explicit test was run (it's *possible* I accidentally right-clicked and chose "Run") but your explanation does at least satisfy item 2.
As far as the "hanging" (#3) is concerned.
I'm not sure if this a "reliable" recipe and some steps may be red herrings. but I seem to be able to reproduce the issue sometimes by:
- Unclicking "Is Enabled" for the local server in the distributed processing tab.
- Restarting ncrunch (disable then enable)
- Let all tests run (and successfully pass)
- Right click on a single test and Choose "Debug/Run selected test as high priority task"
At that point the processing queue shows 3 x "Build Assembly" tasks with "Processing" status. All using the same process id. If it's relevant, the declared task-capacity of "local" is 2.
> Does NCrunch give you any error information for these tasks under the Processing Queue Window?
If I click on the tasks, no additional information is shown in the split panel below the list. (Maybe there's another way to get some insight into the task status?)
Note that if I _enable_ local and restart ncrunch, the same projects can be built fine and run/debugged.
So the workaround _seems_ to be to keep the local server enabled but I'll post back if that turns out to be insufficient.
Edited 27 Oct 2021 10:48 UTC
Remco NCrunch Developer
#15753
28 Oct 2021 01:04 UTC
Thanks for the extra detail here. Based on your description, I've successfully reproduced this problem and implemented a fix for it. The issue was caused by an oversight in how threading resources are managed when working in grid only mode. The build tasks were being starved of threads when trying to instrument, and they were hanging. I'll be deploying this fix with the next build we manage to get through the system.
It should be possible to work around this problem by increasing the 'Max number of processing threads' setting on your client machine when you're working in grid only mode. This will raise the threshold for the threads involved and will prevent the starvation.
It should be possible to work around this problem by increasing the 'Max number of processing threads' setting on your client machine when you're working in grid only mode. This will raise the threshold for the threads involved and will prevent the starvation.
Super - that's great to hear. Thanks for such a quick fix :-)
Remco NCrunch Developer
#15758
29 Oct 2021 04:26 UTC
Could you try the build below and let me know if this solves the problem for you?
NCrunch_Console_4.11.0.1.msi
NCrunch_Console_4.11.0.1.zip
NCrunch_GridNodeServer_4.11.0.1.msi
NCrunch_GridNodeServer_4.11.0.1.zip
NCrunch_LicenseServer_4.11.0.1.zip
NCrunch_VS2010_4.11.0.1.msi
NCrunch_VS2010_4.11.0.1.zip
NCrunch_VS2012_4.11.0.1.msi
NCrunch_VS2012_4.11.0.1.zip
NCrunch_VS2013_4.11.0.1.msi
NCrunch_VS2013_4.11.0.1.zip
NCrunch_VS2015_4.11.0.1.msi
NCrunch_VS2015_4.11.0.1.msi.7z
NCrunch_VS2015_4.11.0.1.zip
NCrunch_VS2017_4.11.0.1.msi
NCrunch_VS2017_4.11.0.1.msi.7z
NCrunch_VS2017_4.11.0.1.zip
NCrunch_VS2019_4.11.0.1.msi
NCrunch_VS2019_4.11.0.1.msi.7z
NCrunch_VS2019_4.11.0.1.zip
NCrunch_VS2022_4.11.0.1.msi
NCrunch_VS2022_4.11.0.1.msi.7z
NCrunch_VS2022_4.11.0.1.zip
NCrunch_Console_4.11.0.1.msi
NCrunch_Console_4.11.0.1.zip
NCrunch_GridNodeServer_4.11.0.1.msi
NCrunch_GridNodeServer_4.11.0.1.zip
NCrunch_LicenseServer_4.11.0.1.zip
NCrunch_VS2010_4.11.0.1.msi
NCrunch_VS2010_4.11.0.1.zip
NCrunch_VS2012_4.11.0.1.msi
NCrunch_VS2012_4.11.0.1.zip
NCrunch_VS2013_4.11.0.1.msi
NCrunch_VS2013_4.11.0.1.zip
NCrunch_VS2015_4.11.0.1.msi
NCrunch_VS2015_4.11.0.1.msi.7z
NCrunch_VS2015_4.11.0.1.zip
NCrunch_VS2017_4.11.0.1.msi
NCrunch_VS2017_4.11.0.1.msi.7z
NCrunch_VS2017_4.11.0.1.zip
NCrunch_VS2019_4.11.0.1.msi
NCrunch_VS2019_4.11.0.1.msi.7z
NCrunch_VS2019_4.11.0.1.zip
NCrunch_VS2022_4.11.0.1.msi
NCrunch_VS2022_4.11.0.1.msi.7z
NCrunch_VS2022_4.11.0.1.zip
Thanks Remco. First impressions are encouraging - I haven't been able to reproduce the issue with 4.11.01 :thumb:
Thanks Remco, I confirm that this is resolved in 4.11.0.2.
PS: It's always a pleasure to pay for an NCrunch license !
PS: It's always a pleasure to pay for an NCrunch license !
Post a reply
Log in to reply.