Docker: Linux containers in Windows docker workers
Our produce is a *WIndows* product that uses *Linux* containers. We'd love to test with Appveyor.
I see in https://help.appveyor.com/discussions/questions/2652-windows-server-2016#comment_42637798 that linux containers were going to be offered, but I still don't see documentation on this. There are also references at https://help.appveyor.com/discussions/questions/8194-linux-docker-containers
Is this something Appveyor is going to be able to do? The last time I tried it was not possible, only WIndows containers were usable on Windows workers.
Comments are currently closed for this discussion. You can start a new one.
Keyboard shortcuts
Generic
? | Show this help |
---|---|
ESC | Blurs the current field |
Comment Form
r | Focus the comment reply box |
---|---|
^ + ↩ | Submit the comment |
You can use Command ⌘
instead of Control ^
on Mac
1 Posted by Ilya Finkelshte... on 30 Jan, 2019 08:06 PM
We actually provide the solution during last year, please check this self-descriptive build. It requires more expensive build VMs (check "Quad" VMs on the billing page).But right now we are forced to re-design it because of many issues with have with Docker for Windows and nested virtualization. We plan to provide an alternative based on Windows Server 2019 and Docker EE. If you are interested in trying this new approach, let us know, we will include you in pilot, which should start in coming weeks.
2 Posted by randy on 30 Jan, 2019 08:31 PM
I thank you very much for the answer but am a little confused. You say that you already provide the solution, but also that it's not yet available?
Please include us in the pilot!
We'd be happy to pay a reasonable monthly for this, it would be great. However, I don't see anything about Quad VM on the pricing page, https://www.appveyor.com/pricing/, can you be more specific?
3 Posted by Ilya Finkelshte... on 30 Jan, 2019 08:51 PM
It is available but we plan to discontinue it in favor of Windows Server 2019 based solution. Pricing page contains only very basic options, check account settings > billing in AppVeyor portal. E.g.
https://ci.appveyor.com/account/<your_account>/billing
. It has much more options.4 Posted by randy on 30 Jan, 2019 09:23 PM
And thank you again for the quick response.
Unfortunately, I seem to have opened this through github (it's open source, again we're happy to pay). So since I opened it through github there's no info on the billing page, see attached screenshot.
Feel free to change the account, or tell me what to do, thanks.
5 Posted by Ilya Finkelshte... on 30 Jan, 2019 10:45 PM
I see, honestly we hesitate to provide this solution at the moment. We plan to move all Linux Container customers to new Windows Server 2019 VMs asap. We are right now preparing this. Is it OK for you to wait for couple of weeks?
6 Posted by randy on 30 Jan, 2019 10:58 PM
Yes, it's OK. We would dearly love to move our testing over to appveyor instead of managing machines on buildkite. You have me on the list for notification? Thanks!
7 Posted by Ilya Finkelshte... on 31 Jan, 2019 07:43 PM
Sure.
Please subscribe to technical updates at https://ci.appveyor.com/notifications and follow us on Twitter.
If we decide to not advertise trial of those VMs with those channels, we will send you information separately. But subscribing to technical updates is useful anyway.
Ilya.
8 Posted by Ilya Finkelshte... on 15 Feb, 2019 01:22 AM
New solution is ready for testing and your enabled for your account.
Please see sample build: https://ci.appveyor.com/project/appveyor-tests/win2k19-azure/builds...
and respective YAML config: https://github.com/appveyor-tests/win2k19-azure/blob/master/appveyo...
Note that build start is typically about 3 minutes (time to provision VM on Azure).
Please let us know how it works for you.
9 Posted by Ilya Finkelshte... on 15 Feb, 2019 01:26 AM
Note that
image: Windows Server 2019
is essential to run build on those VMs.10 Posted by randy on 15 Feb, 2019 04:21 PM
Thanks so much! We're looking forward to trying it out.
11 Posted by mikejolley on 18 Feb, 2019 02:52 PM
Could I get it added to our account as well?
12 Posted by Ilya Finkelshte... on 18 Feb, 2019 07:55 PM
@mikejolley: please provide your account name (or simple link to the build) -- I cannot figure it out using email you use in the forum.
13 Posted by mikejolley on 18 Feb, 2019 09:14 PM
https://ci.appveyor.com/project/AdvancedSystemsUnlimited/kykenkee/b...
14 Posted by Ilya Finkelshte... on 18 Feb, 2019 09:30 PM
Windows Server 2019
image is enabled forAdvancedSystemsUnlimited
account. Couple of things to note:This image runs on Azure VMs, so build start time is 2-6 minutes (overhead of creating and starting VM on Azure, though we are working to make it closer to 2)
For your plan it will be
Standard_D2s_v3
VM (details here) with 2 logical CPUs. 4-CPUsStandard_D4s_v3
goes with "Quad VM" plan (available on the billing page).It is right now in preview, your feedback is greatly appreciated.
15 Posted by randy on 20 Feb, 2019 12:17 AM
Thanks so much for your work on this @ilya - Earlier today I was in fact able to run a simple docker command using your example (https://github.com/appveyor-tests/win2k19-azure/blob/master/appveyor.yml) but now I am no longer able to; it seems that the docker command is not found in the PATH. I don't think this is due to any changes I've made but maybe I got lost somewhere. Build link: https://ci.appveyor.com/project/rfay/build-tools/builds/22500893
(If you'd rather have feedback somewhere else or an issue or whatever, just say the word. Thanks for being so responsive here.)
16 Posted by Ilya Finkelshte... on 20 Feb, 2019 12:33 AM
Thank you for pointing this out. Can you please try now?
17 Posted by randy on 20 Feb, 2019 12:52 AM
Thanks, it seems to be behaving again: https://ci.appveyor.com/project/rfay/build-tools/builds/22501376
18 Posted by randy on 05 Mar, 2019 03:17 PM
Thanks for the awesome help, I've mostly got things working.
One of our test scenarios uses NFS and has to create a volume with options, but I see:
I'm always a little confused about what docker behavior I'm seeing.
Is it docker-ee?
How does one find out about behaviors like this?
Is there any workaround for this problem with volumes and options? That seems like a strange restriction, and so standard in the docker docs.
19 Posted by randy on 05 Mar, 2019 04:25 PM
Network creation also seems not to work?
Tests get this trying to create a normal docker-compose network:
20 Posted by Ilya Finkelshte... on 06 Mar, 2019 10:30 PM
Docker is installed exactly as described this MS doc. As stated in Docker docs in Driver-specific options: "The built-in
local
driver on Windows does not support any options".Can you please describe the environment where it worked for you?
Regarding network creation issue, can you please provide more detailed repro steps?
21 Posted by randy on 07 Mar, 2019 03:04 AM
Thanks so much for your support on this.
This absolutely works on docker-ce (Docker for Windows):
An example of docker network creation is just
docker network create junk
Here's what happens when you do that here: https://ci.appveyor.com/project/rfay/ddev-rgym1
Error response from daemon: could not find plugin bridge in v1 plugin registry: plugin not found
22 Posted by randy on 19 Mar, 2019 02:14 PM
I guess it may not be possible to do what needs to be done with Windows Server 2019. I'll give one more attempt by configuring a Windows Server 2019 with docker-ee myself to see if I can figure out what's going on here.
Could you please let me know which edition of Windows Server 2019 you're using, and any particular characteristics that I'd need to duplicate? Versions are at https://www.microsoft.com/en-us/evalcenter/evaluate-windows-server-...
Thanks!
23 Posted by Ilya Finkelshte... on 19 Mar, 2019 04:56 PM
We are working on the replacing it with Docker CE (backed by MobyLinuxVM) actually, which apparently have better functional for many scenarios, including your. Hope to release updated image this week.
We use Windows Server 2019 Datacenter provided by Azure, if you are still interested.
24 Posted by randy on 19 Mar, 2019 05:00 PM
docker-ce would make me very happy. I'll look forward to the update.
25 Posted by Ilya Finkelshte... on 22 Apr, 2019 09:27 PM
It was not easy, but finally
Windows Server 2019
image runs Docker CE. Please see usage sample here. Let us know how it goes.26 Posted by randy on 23 Apr, 2019 01:01 AM
Yay, thanks for the hard work, look forward to trying it.
27 Posted by randy on 23 Apr, 2019 08:05 PM
I'm not having any more luck, the behavior is the same.
docker network create
fails,docker volume
with drive fails.These work fine on docker-desktop (docker-ce) on Windows 10 Pro.
Thanks for all your help!
28 Posted by Ilya Finkelshte... on 23 Apr, 2019 08:27 PM
Maybe I should have stress more that usage sample I sent in previous message is essential. Use
Switch-DockerLinux
in PowerShell ordocker-switch-linux
in CMD before running those commands.What you do now is the same as if you run those commands on your Windows 10 Pro without selecting
Switch to Linux Containers...
in the Docker UI.29 Posted by randy on 23 Apr, 2019 11:06 PM
Great progress, and thanks for your help. Now I'm to the point where I need the drive to be shared so it can be mounted. Is that something I can do or a config change you have to make?
https://ci.appveyor.com/project/rfay/build-tools-w323a/builds/24055637
30 Posted by Ilya Finkelshte... on 24 Apr, 2019 12:05 AM
Again I would refer to the usage example. Drive D is shared exactly for this task.