File converted to UTF-8 BOM

Sławomir Marcjanski's Avatar

Sławomir Marcjanski

24 Sep, 2015 09:12 AM

Hello,
Some strange issue happened to our build

As part of our build process we are preparing executable with embedded json files

Everything was working fine until 15'th of September

But things started to go wrong after that date. Files embeded in our executable changed their encoding from UTF-8 to UTF-8 BOM
There was no change in files in git.

When we take artefacts from before 15'th of September everything works fine
I took one of old builds (artefacts are correct) and made rebuild and new artefacts was wrong.
So it seems that there was some change in build environment about 10 days ago.

Has anything chnged recently?

  1. Support Staff 1 Posted by Feodor Fitsner on 24 Sep, 2015 01:49 PM

    Feodor Fitsner's Avatar

    Hi Slawomir,

    When exactly was the last "good" build? What are those embedded files, how do you make them?

    - Feodor

  2. 2 Posted by Sławomir Marcja... on 24 Sep, 2015 02:32 PM

    Sławomir Marcjanski's Avatar

    Hi thanx for reply,

    So last good one was at 11'th of September 13:43

    There are two projects in Solution
    One contains files (which we need to deploy to some device), and another project is WPF application that builds executable used to deploy this files (binary stream transfer over SCP server so files are only saved in destination filesysem, and nowhere in between).
    Files are added to executable using Resources (as binary) and linked at compile time (this is where I think problem exists because they must be saved first on build server).
    When I compile same project on my local machine it works fine

  3. Support Staff 3 Posted by Feodor Fitsner on 24 Sep, 2015 02:42 PM

    Feodor Fitsner's Avatar

    This the the update occurred on September 13: http://www.appveyor.com/updates/2015/09/13

    Are you using VS 2013 or 2015? I'm not sure what in that update could affected the way files are saved. And the only thing affecting BOM is assembly patcher but it updates only patched files, not all of them.

    - Feodor

  4. 4 Posted by Sławomir Marcja... on 24 Sep, 2015 03:16 PM

    Sławomir Marcjanski's Avatar

    I am using VS 2013
    Hmm yes change description sounds harmless but it is quite big coincidence :)
    In fact I have numer of files there (not only json files but problem with parsing them on destination OS was most visible) and they all seem to be modified in same way - when comparing with version from before 11'th of September

    Resoureces are binary so I think they are quite good snapshot of build server files
    Maybe it is bug in fix :) - hope not

  5. Support Staff 5 Posted by Feodor Fitsner on 24 Sep, 2015 03:25 PM

    Feodor Fitsner's Avatar

    What fix do you mean?

    If you check files right after repo clone - do they have BOM already?

    - Feodor

  6. 6 Posted by Sławomir Marcja... on 24 Sep, 2015 05:07 PM

    Sławomir Marcjanski's Avatar

    After repo clone to my local PC I have UTF-8 files with no BOM and that is correct
    What I ment was that maybe fix introduced new bug

  7. Support Staff 7 Posted by Feodor Fitsner on 24 Sep, 2015 06:18 PM

    Feodor Fitsner's Avatar

    OK, you can try switching AssemblyInfo patching off to see if that's the problem (if you ever use it at all).

    Just a crazy idea - what if when VS detects there are few files in the project with BOM it starts "thinking" all of them should be processes with BOM? :)

  8. 8 Posted by Sławomir Marcja... on 25 Sep, 2015 09:39 AM

    Sławomir Marcjanski's Avatar

    Ok so after testing seems that assembly patching causes trouble
    We are using this feature on almost all files (not only assembly metadata)
    So - is it by-design behaviour to change encoding?
    If not will it be fixed at some point?

  9. Support Staff 9 Posted by Feodor Fitsner on 25 Sep, 2015 05:24 PM

    Feodor Fitsner's Avatar

    You can use this simple PowerShell to re-save AssemblyInfo.cs files without BOM. Add it before build:

    Get-ChildItem -Path .\ -Filter AssemblyInfo.cs -Recurse | % { "Updating $($_.FullName)"; $c = [IO.File]::ReadAllText($_.FullName); [IO.File]::WriteAllText($_.FullName, $c); }
    
  10. 10 Posted by Sławomir Marcja... on 28 Sep, 2015 11:14 AM

    Sławomir Marcjanski's Avatar

    Can you tell me if you plan to fix this in near future?
    My CTO is not very happy that we must do this kind of tricks for something that was already working...

    Can't assembly patching just preserve existing encoding?

    Do you consider this as a bug at all?

  11. Support Staff 11 Posted by Feodor Fitsner on 28 Sep, 2015 03:10 PM

    Feodor Fitsner's Avatar

    Yes, this is a bug and will be fixed shortly.

  12. 12 Posted by Sławomir Marcja... on 28 Sep, 2015 03:32 PM

    Sławomir Marcjanski's Avatar

    This is great news. Thank you :)

  13. Support Staff 13 Posted by Feodor Fitsner on 29 Sep, 2015 04:48 AM

    Feodor Fitsner's Avatar

    It's been fixed and update deployed.

  14. 14 Posted by Sławomir Marcja... on 29 Sep, 2015 08:02 AM

    Sławomir Marcjanski's Avatar

    I can confirm that It fixed my issue. Thank you.

  15. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:00 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac