Problem with docker-compose build on Windows Server 2019

mikejolley's Avatar

mikejolley

19 Feb, 2019 02:05 AM

Link to build: https://ci.appveyor.com/project/AdvancedSystemsUnlimited/kykenkee/b...

It seems that it gets 90% through the build and errors with:

terminate called after throwing an instance of 'Exception*'
Service '*****.ui.iotapp' failed to build: The command '/bin/sh -c dotnet publish KyKenKee.UI.IOTApp.csproj -c Release -o /app' returned a non-zero code: 134

Command executed with exception: Service '*****.ui.iotapp' failed to build: The command '/bin/sh -c dotnet publish KyKenKee.UI.IOTApp.csproj -c Release -o /app' returned a non-zero code: 134

Running this build locally using docker-compose build runs fine and generates my two images.

  1. Support Staff 1 Posted by Ilya Finkelshte... on 19 Feb, 2019 08:19 PM

    Ilya Finkelshteyn's Avatar

    From what I found about this error, it can be related to the resource limits on the VMs. We temporary upgraded you to "Quad" VMs, which will create Standard_D4s_v3 Azure VMs (instead of Standard_D2s_v3) for Windows Server 2019 image. Can you please try and let us know if this helps?

  2. 2 Posted by mikejolley on 19 Feb, 2019 09:57 PM

    mikejolley's Avatar

    Ilya, I'm now getting:

    Unhandled Exception: OutOfMemoryException.
    Service '*****.ui.iotapp' failed to build: The command '/bin/sh -c dotnet publish KyKenKee.UI.IOTApp.csproj -c Release -o /app' returned a non-zero code: 134
    Command executed with exception: Service '*****.ui.iotapp' failed to build: The command '/bin/sh -c dotnet publish KyKenKee.UI.IOTApp.csproj -c Release -o /app' returned a non-zero code: 134
    
  3. Support Staff 3 Posted by Ilya Finkelshte... on 19 Feb, 2019 10:44 PM

    Ilya Finkelshteyn's Avatar

    Can you please run the following in the beginning if your build. install or init stages are good candidate:

    - ps: gcim Win32_OperatingSystem | % { "$([int]($_.TotalVisibleMemorySize/1mb)) Gb" }
    

    Just to ensure it shows 16Gb. And what are specs of your local computer which endures this build?

    Ilya.

  4. 4 Posted by mikejolley on 19 Feb, 2019 11:31 PM

    mikejolley's Avatar

    https://ci.appveyor.com/project/AdvancedSystemsUnlimited/kykenkee/b...

    It does show 16Gb but now I'm bombing on docker-compose:

    docker-compose : Windows named pipe error: The system cannot find the file specified. (code: 2)
    At line:1 char:1
    + docker-compose build
    + ~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (Windows named p...fied. (code: 2):String) [], RemoteException
        + FullyQualifiedErrorId : NativeCommandError
     
    Command executed with exception: Windows named pipe error: The system cannot find the file specified. (code: 2)
    
  5. Support Staff 5 Posted by Ilya Finkelshte... on 20 Feb, 2019 12:34 AM

    Ilya Finkelshteyn's Avatar

    Can you please try one more time now and let us know?

  6. 6 Posted by mikejolley on 20 Feb, 2019 01:31 AM

    mikejolley's Avatar

    Ilya,

    Back to the memory error now:

    MSBUILD : error MSB1025: An internal failure occurred while running MSBuild.
    System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
       at Microsoft.Build.CommandLine.MSBuildApp.BuildProject(String projectFile, String[] targets, String toolsVersion, Dictionary`2 globalProperties, Dictionary`2 restoreProperties, ILogger[] loggers, LoggerVerbosity verbosity, DistributedLoggerRecord[] distributedLoggerRecords, Int32 cpuCount, Boolean enableNodeReuse, TextWriter preprocessWriter, Boolean detailedSummary, ISet`1 warningsAsErrors, ISet`1 warningsAsMessages, Boolean enableRestore, ProfilerLogger profilerLogger, Boolean enableProfiler)
       at Microsoft.Build.CommandLine.MSBuildApp.Execute(String[] commandLine)
    Unhandled Exception: OutOfMemoryException.
    Service '*****.ui.iotapp' failed to build: The command '/bin/sh -c dotnet publish KyKenKee.UI.IOTApp.csproj -c Release -o /app' returned a non-zero code: 134
    Command executed with exception: Service '*****.ui.iotapp' failed to build: The command '/bin/sh -c dotnet publish KyKenKee.UI.IOTApp.csproj -c Release -o /app' returned a non-zero code: 134
    

    It's really odd because these aren't large applications. Unless it's keeping the docker images in memory.

    https://ci.appveyor.com/project/AdvancedSystemsUnlimited/kykenkee/b...

  7. Support Staff 7 Posted by Ilya Finkelshte... on 20 Feb, 2019 03:56 AM

    Ilya Finkelshteyn's Avatar

    What are your dev machine (where it works OK) specs? We can enable beefier VMs for you temporary to ensure it is just a memory size problem.

  8. 8 Posted by mikejolley on 20 Feb, 2019 03:05 PM

    mikejolley's Avatar

    48Gb memory on my dev box

  9. Support Staff 9 Posted by Ilya Finkelshte... on 20 Feb, 2019 06:47 PM

    Ilya Finkelshteyn's Avatar

    I set your account to use Standard_D8s_v3 VMs which has 32Gb of memory (no 48 at least in Dsv3 series, which is essential to use with docker). Let us know if this works.

  10. 10 Posted by mikejolley on 20 Feb, 2019 07:10 PM

    mikejolley's Avatar

    Still getting out of memory exception. Can't believe it has anything to do with memory since these are such tiny containers.

    Both containers combined are only about 250Mb.

    https://ci.appveyor.com/project/AdvancedSystemsUnlimited/kykenkee/b...

  11. Support Staff 11 Posted by Ilya Finkelshte... on 20 Feb, 2019 07:13 PM

    Ilya Finkelshteyn's Avatar

    OK, I reset it back to standard VM sizes. Do you believe you can help us with a repro?

  12. 12 Posted by mikejolley on 20 Feb, 2019 07:24 PM

    mikejolley's Avatar

    I can try. It's difficult based on that error message. Is it possible to rdp into the machine and run the build commands manually to see what exactly is happening?

  13. Support Staff 13 Posted by Ilya Finkelshte... on 20 Feb, 2019 07:53 PM

    Ilya Finkelshteyn's Avatar

    Sure. Please check https://www.appveyor.com/docs/how-to/rdp-to-build-worker/.

    Thanks a lot for cooperation.

  14. 14 Posted by mikejolley on 23 Feb, 2019 03:00 AM

    mikejolley's Avatar

    Ilya,

    The problem seems to be that I'm trying to create Linux containers on the Windows box. I've gotten the build working on the Ubuntu1804 image. There should be some way to tell the Windows Server to switch to Linux containers but I couldn't find it.

    Thanks!
    Mike

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac