Major downgrade of C++ compiler(mingw64) on all windows images
Hello!
I'm debugging a build issue in my projects for several days and seen to find reason why mingw64 build not compiling in CI.
In last known green job: https://ci.appveyor.com/project/Try/tempest/builds/52628098/job/wlbnko1d2qm13q5h
> -- The CXX compiler identification is GNU 14.2.0
and since this week:
> -- The CXX compiler identification is GNU 8.1.0
Version 8.1 is so old, it practically unusable, as it has not C++20 support.
Can you please revert back to 14.2 or upgrade to newer? Thanks!
Comments are currently closed for this discussion. You can start a new one.
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
Support Staff 1 Posted by Owen McDonnell on 18 Sep, 2025 08:16 PM
The minGW and MSYS folders have not been altered from last images. Could this have to do with some cmake breaking change? It was recently updated to 4.1.0.
2 Posted by Александр Весел... on 18 Sep, 2025 08:21 PM
I've checked old build jobs: path to C++ compiler was "
C:\msys64\mingw64\bin\g++.exe" and version 14.2. And after examining the
same path via RDP, I can note that now the compiler version is 8.1.
If we assume that the path to g++ was rerouted, then it's not clear g++
14.2 is supposed to be. I've checked common paths to mingw/msys/cygwin as
well as windows search - compiler appears to be absent.
Support Staff 3 Posted by Owen McDonnell on 18 Sep, 2025 10:30 PM
Here is a build with current
Visual Studio 2019image and here is a build withPrevious Visual Studio 2019image. 15.2 on former and 14.2 on latter.4 Posted by Александр Весел... on 18 Sep, 2025 10:54 PM
Hi,
I've double checked - VS2019 has something. But now there is a new problem
- g++ apparently cannot run:
```
cmake -B../../build -G Ninja -DCMAKE_BUILD_TYPE:STRING=RelWithDebInfo
-- The CXX compiler identification is unknown
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - failed
-- Check for working CXX compiler: C:/msys64/mingw64/bin/g++.exe
-- Check for working CXX compiler: C:/msys64/mingw64/bin/g++.exe - broken
CMake Error at C:/Program
Files/CMake/share/cmake-4.1/Modules/CMakeTestCXXCompiler.cmake:73 (message):
The C++ compiler
"C:/msys64/mingw64/bin/g++.exe"
is not able to compile a simple test program.
It fails with the following output:
Change Dir:
'C:/projects/tempest/build/CMakeFiles/CMakeScratch/TryCompile-nj3foa'
Run Build Command(s): C:/PROGRA~1/Meson/ninja.exe -v cmTC_81cdc
[1/2] C:\msys64\mingw64\bin\g++.exe -o
CMakeFiles\cmTC_81cdc.dir\testCXXCompiler.cxx.obj -c
C:\projects\tempest\build\CMakeFiles\CMakeScratch\TryCompile-nj3foa\testCXXCompiler.cxx
FAILED: [code=1] CMakeFiles/cmTC_81cdc.dir/testCXXCompiler.cxx.obj
C:\msys64\mingw64\bin\g++.exe -o
CMakeFiles\cmTC_81cdc.dir\testCXXCompiler.cxx.obj -c
C:\projects\tempest\build\CMakeFiles\CMakeScratch\TryCompile-nj3foa\testCXXCompiler.cxx
ninja: build stopped: subcommand failed.
```
Support Staff 5 Posted by Owen McDonnell on 18 Sep, 2025 10:59 PM
Looks like a problem with cmake 4 which is new to this image.
6 Posted by Александр Весел... on 21 Sep, 2025 11:21 PM
Would it be possible in this case to revert change on VS2019 image or
upgrade msys on VS2017 image?
Support Staff 7 Posted by Owen McDonnell on 22 Sep, 2025 05:21 AM
Somehow I don't think all users will agree that downgrading cmake is a great idea. You can use
Previous Visual Studio 2019in the short term and run some version of this script to install an older cmake in theinstall:phase of your build in the medium term.8 Posted by Александр Весел... on 23 Sep, 2025 10:16 PM
Hello, Owen!
I've attempted as you suggested: downgrade cmake and build normally.
Unfortunately, on VS2019 image cmake complaining about g++ regardless of
the cmake version.
I'm starting to assume that some patch is broken and g++.exe cannot run as
a subprocess, or something similar is going on.
Here is recent buildlog:
https://ci.appveyor.com/project/Try/tempest/builds/52795380/job/rkftuwwrk8asadpc
Support Staff 9 Posted by Owen McDonnell on 24 Sep, 2025 04:17 AM
You may want to explore whether some update to ninja is causing this as that was also updated from 1.11.1 to 1.13.1.
10 Posted by Александр Весел... on 24 Sep, 2025 09:25 PM
I've tested with downgraded cmake&ninja today - no improvement, C++ still
broken:
https://ci.appveyor.com/project/Try/tempest/builds/52801785/job/x6fsl2ds0b4f1s92
Support Staff 11 Posted by Owen McDonnell on 24 Sep, 2025 09:58 PM
Just to clarify, you think there is something broken on our build images? What could it be? Can you explain more what you mean by "some patch is broken and g++.exe cannot run as a subprocess"?
Support Staff 12 Posted by Owen McDonnell on 24 Sep, 2025 09:58 PM
Can you also test with downgraded gnu compilers?
13 Posted by Александр Весел... on 25 Sep, 2025 05:14 PM
HI,
Using older version of gcc/g++ from existing
installation ("C:\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin\g++.exe"),
allows to configure cmake successfully.
Unfortunately, 8.1 is super old and can't compile the project.
> Just to clarify, you think there is something broken in our build
images? What could it be?
My working theory(only theory!), that cmake makes no difference between
failed compilation and crash. gcc may fail to launch, if it missing it's
own runtime *.dll's, such as libstdc++-6.dll. Maybe something preventing it
from finding them or from finding the correct ones .
14 Posted by Александр Весел... on 29 Sep, 2025 05:52 PM
Hi, Owen!
Are there any updates on this issue?
чт, 25 сент. 2025 г. в 19:13, Александр Веселов <[email blocked]>:
Support Staff 15 Posted by Owen McDonnell on 30 Sep, 2025 01:36 AM
Not really. Here are two builds of current and previous Visual Studio 2019 image. Looks like they have pretty much the exact same contents. What do you want me to check next? I'm not a C++ developer. Can you RDP into the machine and try to figure out the problem with the msys/mingw installation and then send me a repro of the error?
https://ci.appveyor.com/project/appveyor-tests/test-appveyor/builds...
https://ci.appveyor.com/project/appveyor-tests/test-appveyor/builds...
16 Posted by Александр Весел... on 30 Sep, 2025 06:46 PM
Would it be possible on your end to bisect, what image update start to
cause the issue? Generally it's not a normal situation, if cmake can't
recognize compiler - I'm also not sure what exactly can cause it.
> I'm not a C++ developer.
I've created a minimal reproducer here:
https://github.com/OwenMcDonnell/test-appveyor/pull/1
May you kick-off CI, to check, if mingw issue is reproducible? Thanks!
Support Staff 17 Posted by Owen McDonnell on 30 Sep, 2025 10:31 PM
Your reproduction fails on the newer image and also on the previous image, which doesn't tell me anything about what the "major downgrade" is on our images.
18 Posted by Александр Весел... on 01 Oct, 2025 05:24 PM
Here is build where cmake+mingw use to work last time:
https://ci.appveyor.com/project/Try/tempest/builds/52628098/job/wlbnko1d2qm13q5h
cmake version is 3.29.3 and mingw - 14.2.0.
In any case, the build system failure, in compiling "Hello World", is
quite a problem :)
ср, 1 окт. 2025 г. в 00:32, Owen McDonnell <[email blocked]
>:
19 Posted by Александр Весел... on 04 Oct, 2025 08:07 PM
Hello, Owen!
Any luck reproducing green build on older image? And are there any dates on
when this issue might be fixed? Thanks!
ср, 1 окт. 2025 г. в 19:23, Александр Веселов <[email blocked]>:
Support Staff 20 Posted by Owen McDonnell on 06 Oct, 2025 06:06 AM
Sure here is the build succeeding on previous image.
There are no dates on when the issue will be fixed because we don't know it's an issue with our image. Take me to school. Explain exactly how the very simple repo you provided fails to build on the previous image which you were using successfully for some time?
Have you checked/raised issue on msys and/or cygwin repos? Those are all updated versions on the newer image but the install process was exactly the same.
21 Posted by Александр Весел... on 06 Oct, 2025 08:57 PM
Hi,
It seems in build script for minimal reproducer missing initialization of
PATH variable; here is pr:
https://github.com/OwenMcDonnell/test-appveyor/pull/2
Generally, you can also work backwards, if needed, from your green run:
replace all code to single, file, replace cmake to trivial one (Hello world
- like), remove Vulkan instalation and so on.
I've also noticed that you are using "Previous Visual Studio 2019". Is it
something that any user can add to their build script? Can it be used as a
temporary workaround?
> Have you checked/raised an issue on msys and/or cygwin repos?
I'm not using Cygwin and Msys work on my windows laptop locally - doesn't
look like a compiler issue of any sort, at this point.
пн, 6 окт. 2025 г. в 08:06, Owen McDonnell <[email blocked]
>:
22 Posted by Александр Весел... on 06 Oct, 2025 11:02 PM
After a few more tweaks green job:
https://ci.appveyor.com/project/appveyor-tests/test-appveyor/builds/52864330
пн, 6 окт. 2025 г. в 22:57, Александр Веселов <[email blocked]>:
Support Staff 23 Posted by Owen McDonnell on 08 Oct, 2025 03:58 AM
Here are 2 jobs from
Previous Visual Studio 2019image. The first one is successful while the second one fails. The only difference between the two is that the failing build updates msys before running the build. That strongly suggests to me the problem is with msys and not our image.24 Posted by Александр Весел... on 08 Oct, 2025 09:20 PM
> That strongly suggests to me the problem is with msys and not our image.
Log shows that both are 14.0.2
What about use of older image? Is there a way for me to specify the exact
one to use in my build-script? Asking since it's now a major bottleneck in
my work.
ср, 8 окт. 2025 г. в 05:58, Owen McDonnell <[email blocked]
>:
Support Staff 25 Posted by Owen McDonnell on 08 Oct, 2025 10:24 PM
What is 14.0.2?
Yes you can use
Previous Visual Studio 2019but eventually it will be set to current image, i.e. be updated.26 Posted by Александр Весел... on 09 Oct, 2025 11:40 PM
> What is 14.0.2?
Do you know why, then on newer image, if mingw is downgraded back to
14.0.2, it still leads to error in cmake configuration?
чт, 9 окт. 2025 г. в 00:24, Owen McDonnell <[email blocked]
>:
Support Staff 27 Posted by Owen McDonnell on 10 Oct, 2025 03:40 AM
When you say "mingw is downgraded back to 14.0.2" are you talking about downgrading GNU compilers to 14.2.0?
I have never seen that done on newer images. Do you have a build link?
28 Posted by Александр Весел... on 12 Oct, 2025 10:39 PM
Actually yes - we tested cmake and ninja, not mingw.
Tested today: manual install of 14.2 works.
To summarise:
* issue seem to be unreproducible with mingw14
* reproducible with mingw15 (part of msys)
* locally non-reproducible with llvm-mingw17 (I'm using one that comes with
QtCreator)
Maybe you can upgrade to mingw14 or add it to the setup? At the moment
folder " C:\mingw-w64" still at version 8.x
Owen McDonnell closed this discussion on 05 Nov, 2025 07:09 AM.