MSYS build on VS 2019 image is twice as slow as on VS 2017 one

belegdol's Avatar


17 Oct, 2019 05:37 AM

as stated in topic, the MSYS builds on VS2019 image are almost twice as slow as VS2017:
VS2017: 35 min, including huge pacman update
VS2019: 57 min
VS2019 builds are in fact so slow that they regularly go over the default 60 minutes. While the timeout for my fork was bumped to 90 minutes [1], this issue still affects upstream (startaq/mame).
Is such a huge performance downgrade expected here?


  1. Support Staff 1 Posted by Feodor Fitsner on 17 Oct, 2019 04:32 PM

    Feodor Fitsner's Avatar

    My guess would be MSYS version difference between VS 2017 and VS 2019 images. Currently, VS 2019 image is being built from scratch with always the latest MSYS pulled at the moment of building. MSYS on VS 2017 though wasn't updated for some time now. Could it be the case?

  2. 2 Posted by belegdol on 17 Oct, 2019 04:52 PM

    belegdol's Avatar

    I don't think it is down to MSYS version difference. While VS 2017 starts with much older MSYS, I have two
    bash -lc "pacman -Syu --noconfirm"
    calls in my .appveyor.yml, which effectively updates everything to the latest status.

  3. Support Staff 3 Posted by Feodor Fitsner on 17 Oct, 2019 04:54 PM

    Feodor Fitsner's Avatar

    OK, could base OS be a problem? VS 2017 is on Windows Server 2016 while VS 2019 image is on Windows Server 2019.

  4. 4 Posted by belegdol on 17 Oct, 2019 04:58 PM

    belegdol's Avatar

    The OS version difference could very well be the reason. Is this something you might check from appveyor end? Nearly 50% performance drop is quite something.

  5. Support Staff 5 Posted by Feodor Fitsner on 17 Oct, 2019 05:01 PM

    Feodor Fitsner's Avatar

    Would be great having some sort of an easy repro, kind of something doing in a loop X times, if that possible at all? I don't think VS 2019 VMs are generally slower than VS 2016 as it's the same hardware, but there could be some particular operation that is slower.

  6. 6 Posted by belegdol on 17 Oct, 2019 05:07 PM

    belegdol's Avatar

    You can repro it by building mame (MINGW variant) from belegdol/mame appveyor-debug branch, but this does take a while. The builds are consistently slower. If you look at
    almost none of the builds over the last few days succeeded due to hitting 60 minutes.
    On the other hand, the few builds from the branch still carrying the old .appveyor.yml all finished within 35 minutes or so:

  7. Support Staff 7 Posted by Feodor Fitsner on 18 Oct, 2019 05:24 PM

    Feodor Fitsner's Avatar

    We've updated Visual Studio 2019 image with some additional optimizations. It's faster now, at least we match the performance of VS 2017 image for now.

    You can give it another try and let me know how it worked.

  8. 8 Posted by belegdol on 18 Oct, 2019 09:01 PM

    belegdol's Avatar

    Thank you for looking into this! It doesn't appear that much faster unfortunately, only 3 minutes:
    2017 build was 10 minutes slower than last time but still almost 10 minutes faster than 2019.
    Are the fixes you mentioned rolled out already? The latest update to 2019 image mentioned on was on 13 October.

  9. Support Staff 9 Posted by Feodor Fitsner on 18 Oct, 2019 09:07 PM

    Feodor Fitsner's Avatar

    There is no update record for that changes (yet). The change was mostly around disabling a bunch of services.

    We will continue investigating what is slowing down your project on Visual Studio 2019 image. It's interesting that we are getting an approximately the same time when running this image on Hyper-V or Azure. That could mean it's not the problem in hypervisor.

  10. Feodor Fitsner closed this discussion on 03 Apr, 2020 06:45 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts


? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac