C++ build with Coverity Scan terminated because of timeout

Marat Abrarov's Avatar

Marat Abrarov

29 Jul, 2019 04:24 PM

Hi

First of all I'd like to thank AppVeyor team for this helpful (and free for open source) project. I find AppVeyor the best option for Windows builds and I find AppVeyor free plan great because it provides ability to store built artifacts - most of CI platforms do not provide this option for free and there are not so much CI platforms which provide Windows builds.

I have a small C++ project located at https://github.com/mabrarov/asio_samples. I configured AppVeyor for Windows builds. My configuration uses matrix (multiple jobs) and one of the jobs performs scan with Coverity Scan tool. I have to use Windows builds for this scan because of Coverity Scan detects % of scanned sources and fail my scans on Linux because some Windows-only code is not scanned when building on Linux.

I configured my build with Coverity Scan in the same way as it is described in Using Coverity Scan with AppVeyor article. Scan is performed only for builds performed on coverity_scan branch.

My scans worked fine for a year (approx.), but recently (~4-6 months ago) started failing - refer to this one for example. It looks like build job which performs scan (using cov-build tool) runs out of time (1 hr, it looks like this timeout is not configurable for free plan I use) and build fails because of that.

According to detailed log with timestamps:

[00:07:35]          C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\CL.exe /c /I"C:\projects\asio-samples\examples\ma_asio_performance_test_client\include" /I"C:\projects\dependencies\boost-1.70.0-x64-vs2015\include\boost-1_70" /I"C:\projects\asio-samples\libs\ma_config\include" /I"C:\projects\asio-samples\libs\ma_compat\include" /I"C:\projects\asio-samples\libs\ma_cyclic_buffer\include" /I"C:\projects\asio-samples\libs\ma_limited_int\include" /I"C:\projects\asio-samples\libs\ma_custom_alloc_handler\include" /I"C:\projects\asio-samples\libs\ma_helpers\include" /I"C:\projects\asio-samples\libs\ma_steady_deadline_timer\include" /I"C:\projects\asio-samples\libs\ma_strand\include" /I"C:\projects\asio-samples\libs\ma_context_wrapped_handler\include" /I"C:\projects\asio-samples\libs\ma_thread_group\include" /I"C:\projects\asio-samples\libs\ma_async_connect\include" /I"C:\projects\asio-samples\libs\ma_bind_handler\include" /I"C:\Tools\vcpkg\installed\x64-windows\include" /nologo /W4 /WX- /O2 /Ob2 /D WIN32 /D _WINDOWS /D NDEBUG /D WIN32_LEAN_AND_MEAN /D WINVER=0x0501 /D _WIN32_WINNT=0x0501 /D _WIN32_WINDOWS=0x0501 /D _WIN32_IE=0x0600 /D _UNICODE /D UNICODE /D _WINSOCK_DEPRECATED_NO_WARNINGS /D "CMAKE_INTDIR=\"Release\"" /D _UNICODE /D UNICODE /Gm- /EHsc /MD /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR /Fo"ma_asio_performance_test_client.dir\Release\\" /Fd"ma_asio_performance_test_client.dir\Release\vc140.pdb" /Gd /TP /errorReport:queue "C:\projects\asio-samples\examples\ma_asio_performance_test_client\src\main.cpp"
[00:07:43]          main.cpp
[00:13:20]     46>Lib:
[00:13:20]          C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\Lib.exe /OUT:"C:\projects\asio-samples\build\libs\ma_io_service_pool\Release\ma_io_service_pool.lib" /NOLOGO /MACHINE:X64  /machine:x64 ma_io_service_pool.dir\Release\io_service_pool.obj
[00:13:42]          ma_io_service_pool.vcxproj -> C:\projects\asio-samples\build\libs\ma_io_service_pool\Release\ma_io_service_pool.lib

compilation of C++ sources (*.cpp files) is the most time consuming operation and - when building with cov-build tool - compilation of one C++ source file may take almost the same time as building the whole project without cov-build tool (duration of 1st job in the same build was 5 min 3 sec).

I remembered that there was some notification sent by AppVeyor about changes in performance of VMs provisioned for free plan. So my questions are:

  1. Could smbd from AppVeyor team take a look at my issue and tell me if it's smth wrong with cov-build tool (global configuration, my mistake in build, issue with cov-build tool installation / setup)?
  2. If my finding regarding possible root cause - performance of VM for free plan - is correct then could AppVeyor team confirm that?
  3. If performance of VM is the root cause of my failed scans then is there a way to fix that staying on free plan (maybe it's smth with IO and not CPU?)
  1. 1 Posted by Owen McDonnell on 29 Jul, 2019 08:32 PM

    Owen McDonnell's Avatar

    A couple questions.
    When you compile using cov-build.exe locally, does it take noticeably longer than without?
    Have you done any investigation by connecting to the build machine via RDP while the build is running?

  2. 2 Posted by Marat Abrarov on 29 Jul, 2019 09:03 PM

    Marat Abrarov's Avatar

    When you compile using cov-build.exe locally, does it take noticeably longer than without?

    I didn't try that yet. I can try it on Travis CI (Linux) and compare. I (yet) have no local Windows environment to test it. Let me try...

    Have you done any investigation by connecting to the build machine via RDP while the build is running?

    No. Does it really matter?

  3. 3 Posted by Owen McDonnell on 29 Jul, 2019 10:30 PM

    Owen McDonnell's Avatar

    It doesn't matter to me, since i know nothing about coverity build tool, but maybe if you do, you could use process monitor while its running and gain some insight.
    Just an investigative tool available to you.

  4. 4 Posted by Marat Abrarov on 13 Aug, 2019 10:29 PM

    Marat Abrarov's Avatar

    After some time passed, without changes in code (scanned code and AppVeyor configuration / scripts) I restarted failed build with Coverity scan and related AppVeyor job completed successfully within 13 minutes:

    [STATUS] Running security dynamic analysis...
    Looking for translation units
    |0----------25-----------50----------75---------100|
    ****************************************************
    [STATUS] Preparing for bytecode analysis
    |0----------25-----------50----------75---------100|
    ****************************************************
    [STATUS] Running security dynamic analysis
    |0----------25-----------50----------75---------100|
    ****************************************************
    Emitted 2 C# compilation units (100%) successfully
    Emitted 73 C/C++ compilation units (100%) successfully
    2 C# compilation units (100%) are ready for analysis
    73 C/C++ compilation units (100%) are ready for analysis
    The cov-build utility completed successfully.
    Compressing Coverity Scan results...
    Uploading Coverity Scan results (version: 79d9c90)...
    Build successfully submitted.
    

    I guess it was issue with AppVeyor because in another project built by AppVeyor - mabrarov/build-scripts - I had failed builds of master branch where build was interrupted after 32 minutes (26 minutes were spent for building Docker images and running tests, the rest 6 minutes were spent on pushing Docker images to Docker Hub) as timed out while the last build of master branch was successful and its duration (including deployment) took 45 minutes (build and tests took 31 minutes).

    I plan to retest build with Coverity Scan a couple of times before closing this issue (if it won't happen again).

  5. 5 Posted by Marat Abrarov on 19 Aug, 2019 05:37 PM

    Marat Abrarov's Avatar

    Retested on 8/19/19 - https://ci.appveyor.com/project/mabrarov/asio-samples/builds/268014... - and build completed successfully in 11.5 minutes:

    Looking for translation units
    |0----------25-----------50----------75---------100|
    ****************************************************
    [STATUS] Preparing for bytecode analysis
    |0----------25-----------50----------75---------100|
    ****************************************************
    [STATUS] Running security dynamic analysis
    |0----------25-----------50----------75---------100|
    ****************************************************
    Emitted 2 C# compilation units (100%) successfully
    Emitted 73 C/C++ compilation units (100%) successfully
    2 C# compilation units (100%) are ready for analysis
    73 C/C++ compilation units (100%) are ready for analysis
    The cov-build utility completed successfully.
    Compressing Coverity Scan results...
    Uploading Coverity Scan results (version: 98e8fc2)...
    Build successfully submitted.
    

    Closing this ticket as resolved.

  6. Marat Abrarov closed this discussion on 19 Aug, 2019 05:37 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac