Thanks for your quick response.
Quote:I also wonder if there is a way to make these sorts of problems more obvious. Perhaps placing a timestamp at the top of the test output indicating when the test was last run?
The primary focus of someone is usually at the icons, they are clearest: red means bad, green means good. My suggestion would be to do some or all of these:
- Create a hover-help text on each text, this can contain info on the last test run datetime, last compile datetime and the like
- Such balloontext can also indicate a warning, like "NCrunch detected that this test was not impacted by your latest change, click X to rerun"
- An added benefit of balloons is that you can put more info in them, and accumulated text can be put in parent items, like "not all tests have been rerun since the last change"
- But more than balloons, icons would be great. Change clear-red into troubled red and change clear green into troubled green to indicate that they are "stale", i.e. not run again
- In addition to that, I certainly would welcome some more info in the logwindow of the NCrunch test screen, think about a caption bar with full path to the test method (before it's extended with testsource), the time it was build/run, repetition of success/fail icons. Or something like that. Just make it collapsible and people will be content with it.
For me, most help would be given from knowing that a test is "stale", or "out of sync" with its parent. Even if source doesn't change, I often find it hard to keep track. If I recompile a project, and rerun two or three individual tests, I can't see which one was newly run and which one is old (stale / out of sync / from before last build).
So, my
vision ;) on this is that on each rebuild, no rerun tests keep the same icon, same color but a slightly troubled (darker / greyish) color.
Another thing you can do from within the NCrunch engine is to accept some exceptions as vital. That is, if an exception is vital, than any change may impact the whole project. Such exceptions are the ones that cannot be recovered from: BadImageFormatException (can be fixed by compile policy of file system rights change), OutOfMemoryException (can be fixed by environment change), StackOverflowException (can be fixed by debug into release change), but this can be a slippery slope: a SecurityException can mean you are not running in elevated mode, and a FileNotFoundException can be fixed by the environment.
Hmm, thinking out load: what if you recognize the exceptions that are not test exceptions (like NUnit / xUnit exceptions)? These are usually assertions anyway, but the rest often has to do with the environment.
Anyway, whatever you decide, improvements here are welcome. Still, making sure that Rerun All means exactly that, a Clean Everything is added and perhaps the greyish icons would already help many times over (and all have wider applicability than just this issue).
