Changing a 'const' value makes test fail

When changing a string declared as const in my assembly under test (but also used in the tests), the impacted tests started to fail.
Restarting VS made them pass again.

[TestMethod]
public void CanUserModifyIT132_UserWithBrusselsRoleAndDifferentPostAsFilePost_UserCannotModifyIT132()
{
// Arange
var target = new Xm25SearchIndexViewModel();
var searchResultMock = new Mock<IXm25SearchResult>();
var document = GetDocumentMock();
document.FileInfo.IT022ResidencesAbroad[0].ResidenceAbroad.DiplomaticPost.Code = "4321";
searchResultMock.Setup(x => x.Document).Returns(document);
target.SearchResult = searchResultMock.Object;
var userMock = GetUserMock();
userMock.Setup(x => x.IsInRole(Constants.RoleBrusselsUser)).Returns(true); // <-- const used here
userMock.Setup(x => x.Post).Returns(new Post { InsCode = "1234" });
// Act
var canModifyIT132 = target.GetCanModifyIT132(userMock.Object);
// Assert
Assert.IsFalse(canModifyIT132);
}

Is this a known issue?

Hi, thanks for posting!

I'm not aware of any known issue that can cause this. Can you share any more details about the failure pattern and behaviour of the code? Is it an issue you can recreate?


Cheers,

Remco

I've put together a example solution to reproduce the problem: (- BROKEN LINK -)(- BROKEN LINK -)[/url]

Once the solution is built and the test passes in NCrunch, open the 'Constants.cs' file and change the value of the constant (e.g. from 'Updater' to 'Updaters') --> the test fails.

When you rebuild the test project, then make a useless change (e.g. adding a space) to the test itself, the test passes again.

It's not a big problem in itself, but it's worth knowing that the the solution needs to be rebuilt to make the test(s) pass again...

Regards,

Julien

Thanks Julien! The sample solution you've provided was enormously helpful in identifying this problem.

You've managed to find a hole in NCrunch's optimised build. Basically, because the constant is evaluated at compile time and not run time, it's value is being hard-baked into the references (one in the tests project, one in the production code). By default, NCrunch will avoid rebuilding projects that depend upon changed projects unless the surface of the changed project has changed (i.e. if you've added a parameter to a public method, then referencing assemblies must be rebuilt). This has many advantages, as when changing the contents of a method at the bottom of a huge dependency tree of assemblies, the entire dependency tree doesn't need to be rebuilt. In this case, NCrunch isn't recognising that the new constant value requires a rebuild of the dependent Tests assembly, so it builds the test domain with an older version of the assembly and the string constants clash.

I've noted this down to be fixed in the next maintenance release. If you're making frequent changes to string constants that are referenced across assemblies, it may be worth turning on the Copy referenced assemblies to workspace setting for all projects involved - as this will turn off NCrunch's optimised build and resolve the issue entirely. However, because the optimised build adds considerable value, I would probably recommend just resetting the engine if you change one of these string constants (assuming you don't do this very often).

As far as bugs go, this was a good catch. This issue has been in place over the life of the software, and no one has bothered to report it before .. Most likely they just reset the engine.

Cheers,

Remco