Believe it or not, this is actually normal behaviour :)

NCrunch will not remove tests from the queue while they are still executing, so if your tests were still running at the time your project was changed, then a new (often identical) task would be queued next to the old one.

It's important to consider this as this behaviour means that a test can be run side-by-side with itself ... so if you have any tests that are using specific resources in a database or the file system, make sure you attribute them with the ExclusivelyUsesAttribute (if you're running tests in parallel that is).