So I was having problems backporting some changes to a fork of our platform today. WinMerge kept reporting files as different in the summary view, but displayed no differences in the file comparison view. I looked closer and noticed the character sets were “different”.

A developer on the team had started saving files in the repository with BOM (Byte Order Mark) information. I don’t know if it was intentional or an effect of their IDE. Either way, it had to go, so … perl to the rescue.

for i in $(find -regex ".*\.jsp$"); do perl -pi.old -e 'BEGIN { $/ = undef; } s/^\xEF\xBB\xBF//' $i; done