*.ncrunchproject and *.ncrunchsolution encoding change?

Hi Remco and NCrunch users,

Am I right in believing that NCrunch has recently changed the encoding used in the .ncrunchproject and .ncrunchsolution files? These files appear to have been UTF-8 w/o BOM however now they are UTF-16/UCS-2. I had a look through some changelogs but could not find anything regarding this.

The reason I am asking is because UTF-16/UCS-2 encoding really does not work well with Git, git flags these files as binary unless various work arounds are employed making diffs much more difficult to perform. Is there any chance of UTF-8 encoding being used once again?

I am currently using version 2.15.0.9 for VS2013.

Thanks for your help,

Luke

Hi Luke,

You are correct - this was changed .. though it was a while ago (I think maybe a year or so).

The encoding needed to be changed because these files can contain user entered strings that sometimes require complex character sets that just don't work in UTF8. This creates some very strange bugs when using NCrunch in certain localities.

I'm afraid there isn't much I can do about this. The correct format is UTF16, as otherwise the application won't behave correctly.

Hi Remco,

Thanks for getting back to me so quickly.

I have no doubt there was a valid reason for the change, however UTF-8, UTF-16 and UTF-32 all support the full set of Unicode code points; UTF-8 and UTF-32 could potentially support more if it weren't limited by UTF-16.

I didn't think there could exist a string which can be encoded in one Unicode encoding but not another, if I am wrong on this, feel free to correct me, I would be very interested.

lhorsley wrote:Hi Remco,

Thanks for getting back to me so quickly.

I have no doubt there was a valid reason for the change, however UTF-8, UTF-16 and UTF-32 all support the full set of Unicode code points; UTF-8 and UTF-32 could potentially support more if it weren't limited by UTF-16.

I didn't think there could exist a string which can be encoded in one Unicode encoding but not another, if I am wrong on this, feel free to correct me, I would be very interested.

This may well be true, though when the product was rigged up to store the data as UTF8, there were serious problems in storing and retrieving some data from the files.

I am unsure whether this was due to the way the encoding itself works, or perhaps some issue with the components involved. However, the impact of this was very serious for some users. It resulted in a complete inability to store and retrieve any settings data, which is far more serious than simply being unable to easily merge a file using a tool that hasn't implemented UTF16 support.

I am sorry but I will not be changing the encoding of these files again.

Remco wrote:I am unsure whether this was due to the way the encoding itself works, or perhaps some issue with the components involved. However, the impact of this was very serious for some users. It resulted in a complete inability to store and retrieve any settings data, which is far more serious than simply being unable to easily merge a file using a tool that hasn't implemented UTF16 support.

I am sorry but I will not be changing the encoding of these files again.

I agree that is a much more serious issue, but your fix for that has caused a new issue for users of NCrunch with Git.

Considering your documentation states:

NCrunch has been designed with consideration for teams that want to share settings via source control systems.

This doesn't seem very considerate for users of Git.

As it is, we know UTF-16 is causing issue for Git users and may be solving a problem for other users, is it not even worth some investigation into whether UTF-8 could be used instead?

Just a footnote, I have been a user of NCrunch since it was free and have convinced two companies to purchase licenses for their developers, I really believe in this product but feel that a refusal to investigate is not a good attitude to have when you know a user has a problem.

It looks like there may be some options in Git that can be used to improve its handling of merging UTF-16 files. It might be worth a look - https://stackoverflow.com/questions/777949/can-i-make-git-recognize-a-utf-16-file-as-text.