Docker: Linux containers in Windows docker workers

randy's Avatar

randy

30 Jan, 2019 02:17 PM

Our produce is a *WIndows* product that uses *Linux* containers. We'd love to test with Appveyor.

I see in https://help.appveyor.com/discussions/questions/2652-windows-server-2016#comment_42637798 that linux containers were going to be offered, but I still don't see documentation on this. There are also references at https://help.appveyor.com/discussions/questions/8194-linux-docker-containers

Is this something Appveyor is going to be able to do? The last time I tried it was not possible, only WIndows containers were usable on Windows workers.

  1. 1 Posted by Ilya Finkelshte... on 30 Jan, 2019 08:06 PM

    Ilya Finkelshteyn's Avatar

    We actually provide the solution during last year, please check this self-descriptive build. It requires more expensive build VMs (check "Quad" VMs on the billing page).But right now we are forced to re-design it because of many issues with have with Docker for Windows and nested virtualization. We plan to provide an alternative based on Windows Server 2019 and Docker EE. If you are interested in trying this new approach, let us know, we will include you in pilot, which should start in coming weeks.

  2. 2 Posted by randy on 30 Jan, 2019 08:31 PM

    randy's Avatar

    I thank you very much for the answer but am a little confused. You say that you already provide the solution, but also that it's not yet available?

    Please include us in the pilot!
    We'd be happy to pay a reasonable monthly for this, it would be great. However, I don't see anything about Quad VM on the pricing page, https://www.appveyor.com/pricing/, can you be more specific?

  3. 3 Posted by Ilya Finkelshte... on 30 Jan, 2019 08:51 PM

    Ilya Finkelshteyn's Avatar

    It is available but we plan to discontinue it in favor of Windows Server 2019 based solution. Pricing page contains only very basic options, check account settings > billing in AppVeyor portal. E.g. https://ci.appveyor.com/account/<your_account>/billing. It has much more options.

  4. 4 Posted by randy on 30 Jan, 2019 09:23 PM

    randy's Avatar

    And thank you again for the quick response.

    Unfortunately, I seem to have opened this through github (it's open source, again we're happy to pay). So since I opened it through github there's no info on the billing page, see attached screenshot.

    Feel free to change the account, or tell me what to do, thanks.

  5. 5 Posted by Ilya Finkelshte... on 30 Jan, 2019 10:45 PM

    Ilya Finkelshteyn's Avatar

    I see, honestly we hesitate to provide this solution at the moment. We plan to move all Linux Container customers to new Windows Server 2019 VMs asap. We are right now preparing this. Is it OK for you to wait for couple of weeks?

  6. 6 Posted by randy on 30 Jan, 2019 10:58 PM

    randy's Avatar

    Yes, it's OK. We would dearly love to move our testing over to appveyor instead of managing machines on buildkite. You have me on the list for notification? Thanks!

  7. 7 Posted by Ilya Finkelshte... on 31 Jan, 2019 07:43 PM

    Ilya Finkelshteyn's Avatar

    Sure.

    Please subscribe to technical updates at https://ci.appveyor.com/notifications and follow us on Twitter.

    If we decide to not advertise trial of those VMs with those channels, we will send you information separately. But subscribing to technical updates is useful anyway.

    Ilya.

  8. 8 Posted by Ilya Finkelshte... on 15 Feb, 2019 01:22 AM

    Ilya Finkelshteyn's Avatar

    New solution is ready for testing and your enabled for your account.

    Please see sample build: https://ci.appveyor.com/project/appveyor-tests/win2k19-azure/builds...
    and respective YAML config: https://github.com/appveyor-tests/win2k19-azure/blob/master/appveyo...

    Note that build start is typically about 3 minutes (time to provision VM on Azure).

    Please let us know how it works for you.

  9. 9 Posted by Ilya Finkelshte... on 15 Feb, 2019 01:26 AM

    Ilya Finkelshteyn's Avatar

    Note that image: Windows Server 2019 is essential to run build on those VMs.

  10. 10 Posted by randy on 15 Feb, 2019 04:21 PM

    randy's Avatar

    Thanks so much! We're looking forward to trying it out.

  11. 11 Posted by mikejolley on 18 Feb, 2019 02:52 PM

    mikejolley's Avatar

    Could I get it added to our account as well?

  12. 12 Posted by Ilya Finkelshte... on 18 Feb, 2019 07:55 PM

    Ilya Finkelshteyn's Avatar

    @mikejolley: please provide your account name (or simple link to the build) -- I cannot figure it out using email you use in the forum.

  13. 13 Posted by mikejolley on 18 Feb, 2019 09:14 PM

    mikejolley's Avatar
  14. 14 Posted by Ilya Finkelshte... on 18 Feb, 2019 09:30 PM

    Ilya Finkelshteyn's Avatar

    Windows Server 2019 image is enabled for AdvancedSystemsUnlimited account. Couple of things to note:

    • This image runs on Azure VMs, so build start time is 2-6 minutes (overhead of creating and starting VM on Azure, though we are working to make it closer to 2)

    • For your plan it will be Standard_D2s_v3 VM (details here) with 2 logical CPUs. 4-CPUs Standard_D4s_v3 goes with "Quad VM" plan (available on the billing page).

    It is right now in preview, your feedback is greatly appreciated.

  15. 15 Posted by randy on 20 Feb, 2019 12:17 AM

    randy's Avatar

    Thanks so much for your work on this @ilya - Earlier today I was in fact able to run a simple docker command using your example (https://github.com/appveyor-tests/win2k19-azure/blob/master/appveyor.yml) but now I am no longer able to; it seems that the docker command is not found in the PATH. I don't think this is due to any changes I've made but maybe I got lost somewhere. Build link: https://ci.appveyor.com/project/rfay/build-tools/builds/22500893

    (If you'd rather have feedback somewhere else or an issue or whatever, just say the word. Thanks for being so responsive here.)

  16. 16 Posted by Ilya Finkelshte... on 20 Feb, 2019 12:33 AM

    Ilya Finkelshteyn's Avatar

    Thank you for pointing this out. Can you please try now?

  17. 17 Posted by randy on 20 Feb, 2019 12:52 AM

    randy's Avatar
  18. 18 Posted by randy on 05 Mar, 2019 03:17 PM

    randy's Avatar

    Thanks for the awesome help, I've mostly got things working.

    One of our test scenarios uses NFS and has to create a volume with options, but I see:

    docker volume create --driver local --opt type=nfs junker99

    Error response from daemon: create junker99: options are not supported on this platform

    I'm always a little confused about what docker behavior I'm seeing.
    Is it docker-ee?
    How does one find out about behaviors like this?
    Is there any workaround for this problem with volumes and options? That seems like a strange restriction, and so standard in the docker docs.

  19. 19 Posted by randy on 05 Mar, 2019 04:25 PM

    randy's Avatar

    Network creation also seems not to work?

    Tests get this trying to create a normal docker-compose network:

     error (404): could not find plugin bridge in v1 plugin registry: plugin not found
    
  20. 20 Posted by Ilya Finkelshte... on 06 Mar, 2019 10:30 PM

    Ilya Finkelshteyn's Avatar

    Docker is installed exactly as described this MS doc. As stated in Docker docs in Driver-specific options: "The built-in local driver on Windows does not support any options".
    Can you please describe the environment where it worked for you?

    Regarding network creation issue, can you please provide more detailed repro steps?

  21. 21 Posted by randy on 07 Mar, 2019 03:04 AM

    randy's Avatar

    Thanks so much for your support on this.

    This absolutely works on docker-ce (Docker for Windows):

    λ docker volume create --driver local --opt type=nfs junker99
    junker99
    

    An example of docker network creation is just docker network create junk

    Here's what happens when you do that here: https://ci.appveyor.com/project/rfay/ddev-rgym1

    Error response from daemon: could not find plugin bridge in v1 plugin registry: plugin not found

  22. 22 Posted by randy on 19 Mar, 2019 02:14 PM

    randy's Avatar

    I guess it may not be possible to do what needs to be done with Windows Server 2019. I'll give one more attempt by configuring a Windows Server 2019 with docker-ee myself to see if I can figure out what's going on here.

    Could you please let me know which edition of Windows Server 2019 you're using, and any particular characteristics that I'd need to duplicate? Versions are at https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-...

    Thanks!

  23. 23 Posted by Ilya Finkelshte... on 19 Mar, 2019 04:56 PM

    Ilya Finkelshteyn's Avatar

    We are working on the replacing it with Docker CE (backed by MobyLinuxVM) actually, which apparently have better functional for many scenarios, including your. Hope to release updated image this week.

    We use Windows Server 2019 Datacenter provided by Azure, if you are still interested.

  24. 24 Posted by randy on 19 Mar, 2019 05:00 PM

    randy's Avatar

    docker-ce would make me very happy. I'll look forward to the update.

  25. 25 Posted by Ilya Finkelshte... on 22 Apr, 2019 09:27 PM

    Ilya Finkelshteyn's Avatar

    It was not easy, but finally Windows Server 2019 image runs Docker CE. Please see usage sample here. Let us know how it goes.

  26. 26 Posted by randy on 23 Apr, 2019 01:01 AM

    randy's Avatar

    Yay, thanks for the hard work, look forward to trying it.

  27. 27 Posted by randy on 23 Apr, 2019 08:05 PM

    randy's Avatar

    I'm not having any more luck, the behavior is the same. docker network create fails, docker volume with drive fails.

    These work fine on docker-desktop (docker-ce) on Windows 10 Pro.

    Thanks for all your help!

  28. 28 Posted by Ilya Finkelshte... on 23 Apr, 2019 08:27 PM

    Ilya Finkelshteyn's Avatar

    Maybe I should have stress more that usage sample I sent in previous message is essential. Use Switch-DockerLinux in PowerShell or docker-switch-linux in CMD before running those commands.

    What you do now is the same as if you run those commands on your Windows 10 Pro without selecting Switch to Linux Containers... in the Docker UI.

  29. 29 Posted by randy on 23 Apr, 2019 11:06 PM

    randy's Avatar

    Great progress, and thanks for your help. Now I'm to the point where I need the drive to be shared so it can be mounted. Is that something I can do or a config change you have to make?

    https://ci.appveyor.com/project/rfay/build-tools-w323a/builds/24055637

  30. 30 Posted by Ilya Finkelshte... on 24 Apr, 2019 12:05 AM

    Ilya Finkelshteyn's Avatar

    Again I would refer to the usage example. Drive D is shared exactly for this task.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac