The error you are encountering actually originates from NCrunch rather than NUnit. It's being caused by your generation returning a different set of results even though your test project has seen no changes.
Previously, this would cause your test results to become skewed and unpredictable due to the positions of tests in the result set not matching their positions in the initial discovery run.
NCrunch is designed with the assumption that the same unchanged project will always return the same set of tests every time it is asked. Where this is not the case, we now detect it and give an error rather than give malformed results.
Looking at your description of your design, I think that what you are trying to do is too complex given the constraints of the runner. Perhaps consider avoiding dynamic test generation and place the test cases inside a static test instead.
