abelb;11013 wrote:
Change in the code of a used attribute, directly or indirectly, does not always (never?) trigger an Impact
Impact detection doesn't currently work over attributes, because these are universally accessed using reflection, which is isn't possible to track using NCrunch's current impact detection methods.
abelb;11013 wrote:- Change in an inlined method that is used by a test does not always trigger Impact
- Change in a used constant (as opposed to readonly) does not trigger an Impact
Both of these situations should be handled by the engine, as there is code specifically intended to target them. Have you observed this in C#/VB at all? If you can provide a set of steps to reproduce this I should be able to investigate further.
abelb;11013 wrote:
Change code directly in the test itself, when these are themselves nested functions does not always trigger an Impact (for sure F# only)
Change inside Quotations (i.e. with the test harness provided by UnQuote) never trigger an Impact (whether the quotation is directly inside the test, or not, does not matter, also F# only). However, changes in code used by quotations do trigger an impact
F#, unfortunately, won't ever get the same level of impact detection support as C#/VB under NCrunch. The reason for this is that impact detection requires very carefully targeted handling to certain compiler IL nuances. An example of this is generated code, of which usually has random or sequence-dependent naming and makes cross-assembly code comparison particularly challenging. When compared with C# and VB, F# code has a far larger amount of compiler generated code, making it much more challenging to implement any reliable level of impact detection without sending performance or maintainability through the floor. Considering the difference in user base size, it is also much harder to justify the disproportional amount of effort needed to get F# to the same level of handling.
If you're able to give very detailed step-by-step instructions for reproducing these scenarios under F#, I will do my best to investigate, but please understand that it may be an uphill battle, especially as my own F# skills are significantly lacking at the moment.
abelb;11013 wrote:
Occasionally it seems that rebuild + run does not pick up a change, that is, a failing test that is supposed to report success won't report success and issues the same error. This seems to happen with used constants and with quotations, but each time I try to create a small reproducible example I fail to repeat the behavior. Even the actual behavior itself does not always seem repeatable (but undoing/redoing the same changes in VS with Ctrl-Z/Ctrl-Y), so it is hard to pinpoint this to a particular case. The workaround (basically to remedy the feeling "I can't believe what I am seeing in this test, I am sure the test is correct, I'm going nuts!") is usually to change something in the NCrunch configuration, after which the rebuild + run does pick up the change and report success.
Sorry I can understand your frustration here. The impact detection system was not enjoyable to use in its pre-release state. It may be best to switch back to an impact detection mode of 'WatchText' for a better experience if you're exclusively using F#.
