Home → Problems →

Vagrant not working inside AppVeyor

• Edit

ukio

20 Dec, 2014 11:05 PM

Hi everybody,

I'm trying to run some integration tests, which involves spinning up a VirtuaBox VM inside an AppVeyor Build Worker via Vagrant.

However, whenever it tries to bring up the VM (via `vagrant up`) it will never come up properly. It runs into a timeout while trying to connect to that VM. See here:
https://ci.appveyor.com/project/ukio/sample-toplevel-cookbook/build/34-master

I suspect this is because Vagrant wants to create an new network adapter which (at least on my Win7 laptop) causes the UAC prompt to come up. I'm afraid it stalls here becuase the prompt is never granted elevated permissions for creating the network adapters.

Any ideas how to confirm and / or fix this?

Thanks and kind regards,
Torben

1. Support Staff 1 Posted by Feodor Fitsner on 20 Dec, 2014 11:15 PM

   

   Well, I don't think it's UAC - in fact, it's disabled on build workers.

   Try using "unstable" OS (image) - it's running build agent interactive - maybe this could help. You can select it on Environment tab of project settings.

   • Edit

2. 2 Posted by ukio on 21 Dec, 2014 12:00 AM

   

   Hi Feodor, thanks for the quick response!

   Yes, I believe it's disabled. Also tried explicitly disabling UAC and / or the UAC prompt (from http://ss64.com/nt/syntax-uac.html), but it did not help though...

   Will try with the "unstable" OS (image) now...

   • Edit

3. 3 Posted by ukio on 21 Dec, 2014 01:26 AM

   

   FYI - I could verify that it works in general (via minimalistic vagrant file):
   https://ci.appveyor.com/project/ukio/sample-toplevel-cookbook/build/46-appveyor

   So it must be something else. I'll start with increasing the timeout a bit...

   • Edit

4. Support Staff 4 Posted by Feodor Fitsner on 21 Dec, 2014 01:30 AM

   

   That's great, keep us posted!

   -Feodor

   • Edit

5. 5 Posted by ukio on 27 Dec, 2014 10:11 PM

   

   Experiences so far:

   • increased the timeout for bringing up the vm to 20 minutes
   • running builds on x64 Windows Server 2012 only
   • the time for bringing up the VM varies quite a lot. I got the vm up and running
     • on hyperv: somewhere between 6 and 16 minutes
     • on azure: somewhere between 16 and 26 minutes
   • so far I only got 32-bit VMs running. The 64-bit VMs always timed out.
   • still randomly running into timeouts while bringing up the VMs (mostly on hyperv, azure feels more stable)
   • tried to cache the VirtualBox installation, but that doesn't work out -> you need to run the installer to get the additional network adapters and probably lots of other things

   So, current status: still trying to get a series of green stable builds reproducably and reliably

   • Edit

6. 6 Posted by ukio on 27 Dec, 2014 10:49 PM

   

   Results with the hyperv workers are getting better. Maybe it was a bad idea trying to cache the Vagrant installation as well, so I removed it from the cache, and promptly got a new build time record:

   • 5:09 for bringing up a VM on hyperv: https://ci.appveyor.com/project/ukio/vagrant-appveyor-testing/build...
   • Edit

7. 7 Posted by ukio on 27 Dec, 2014 11:16 PM

   

   ...and the next build on hyperv timed out again :-( https://ci.appveyor.com/project/ukio/vagrant-appveyor-testing/build...

   There was no code / appveyor.yml change involved. So what else could the differences be?

   • Reverese DNS resolution timing out?
   • Hyper-V enabled or not on the build worker
   • VT-x enabled via BIOS or not?

   Actually - the hyperv workers should be all exactly the same, shouldn't they?

   • Edit

8. 8 Posted by ukio on 28 Dec, 2014 12:00 AM

   

   New record:

   • 4:11 for bringing up a VM on hyperv (using Ubuntu precise32 basebox this time): https://ci.appveyor.com/project/ukio/vagrant-appveyor-testing/build...
   • Edit

9. Support Staff 9 Posted by Feodor Fitsner on 28 Dec, 2014 12:03 AM

   

   Not bad at all!

   • Edit

10. 10 Posted by ukio on 28 Dec, 2014 12:46 AM

    

    This one looks weird. I reverted back to the trusty32 box to check if it were the culprit. Got a failed build after 22 minutes again, but this time with a different error message:
    https://ci.appveyor.com/project/ukio/vagrant-appveyor-testing/build...

    ...
        default: Warning: Connection timeout. Retrying... 
    153    default: Warning: Connection timeout. Retrying... 
    154    default: Warning: Connection timeout. Retrying... 
    155Vagrant can't use the requested machine because it is locked! This
    156means that another Vagrant process is currently reading or modifying
    157the machine. Please wait for that Vagrant process to end and try
    158again. Details about the machine are shown below:
    159 
    160Name: default
    161Provider: virtualbox 
    162Command exited with code 1
    

    The error indicates some kind of shared state, but it's also pretty exactly at the configured boot timeout after 20 minutes.

    • Edit

11. 11 Posted by ukio on 29 Dec, 2014 11:41 PM

    

    Hi Feodor,

    I have no luck with the ubuntu 14.04 base boxes on hyper-v build workers. On the azure workers everything works well (it just takes a bit longer).

    Tried with the "ubuntu/trusty32" as well as the "chef/ubuntu-14.04-i386" base box. In both cases it randomly (looks like random at least) fails to connect via SSH within the given timeout of 15 minutes.

    Compare the latest build results from hyper-v (mostly instable):
    https://ci.appveyor.com/project/ukio/vagrant-appveyor-testing/history
    ...with the ones from azure (stable): https://ci.appveyor.com/project/tknerr/vagrant-appveyor-testing/his...

    What could be the differences between the azure and hyper-v workers? Anything ssh related maybe?

    I added some client-side ssh debugging if the vagrant up command fails. However it only reveals a ssh_exchange_identification: read: Connection reset by peer error, which I can't debug further client side.

    It would probably help to debug this via RDP. I am now starting the VMs in gui mode but no idea how to access the build worker via RDP if that is possible at all.

    Do you have any futher means on how to debug this?

    I don't have any meta information about the build worker IDs who are executing the builds -- maybe that would give another hint (e.g. reveal that it always fails / succeeds on a specific hyper-v worker).

    Other suspects:
    * firewall? * disabled the windows firewall but did not help * checked inside the ubuntu/trusty32 vm: ufw firewall is inactive and neither fail2ban nor denyhosts is installed, so that should not be the reason * reverse dns slowness? * unlikely since the appveyor build workers should use 8.8.8.8 which is pretty stable * however, UseDNS is not explictly disabled in /etc/ssh/sshd_config and the default is yes, so it might have to do with it

    sigh...

    • Edit

12. 12 Posted by ukio on 30 Dec, 2014 12:54 AM

    

    @Feodor can you see this in the VirtualBox GUI? https://ask.openstack.org/en/question/28297/cloud-init-nonet-waitin...

    Not sure why it work then on the azure workers though...

    • Edit

13. Support Staff 13 Posted by Feodor Fitsner on 30 Dec, 2014 02:27 AM

    

    You mean it might be DHCP problem? There is no DHCP on Hyper-V environment.

    -Feodor

    • Edit

14. 14 Posted by ukio on 30 Dec, 2014 12:09 PM

    

    No, not exactly a DHCP problem, but something about the network adapters in ubuntu 14.04, or just a desperate guess because I see no other chance to debug this.

    So weird it works sometimes and sometimes not, but I can't see why.

    Is there any possibility to RDP into a build worker? Or at least getting screenshots of the VirtualBox GUI anyhow?

    • Edit

15. Support Staff 15 Posted by Feodor Fitsner on 31 Dec, 2014 03:11 AM

    

    Theoretically, you can RDP into build worker instance.

    You should do three steps:
    1. Get know instance IP from IP-enabled NIC (say using PowerShell).
    2. Update appveyor account password, for example:
    

    $objUser = [ADSI]("WinNT://$($env:computername)/appveyor")
    $objUser.SetPassword('yourpassword')
    

    3. Allow RDP on firewall.

    Be aware your session will be limited with total time of build (40 minutes). Let me know how it goes.

    • Edit

16. 16 Posted by Stefan Scherer on 25 Nov, 2015 09:49 PM

    

    Great work to try Vagrant on AppVeyor.

    But is it also possible to run 64bit VM's in AppVeyor?
    AFAIK you would need VT-X enabled (nested hypervisor) in the AppVeyor worker instance. Can this be enabled in AppVeyor somewhere?

    • Edit

17. 17 Posted by andrew.pashkin on 27 Feb, 2016 01:54 AM

    

    Also stumbled upon this issue.

    Regarding RDB - there is an article in in the official docs about that:
    https://www.appveyor.com/docs/how-to/rdp-to-build-worker

    • Edit

18. 18 Posted by andrew.pashkin on 27 Feb, 2016 11:05 AM

    

    I was able to resolve my problem, here is the successful build:
    https://ci.appveyor.com/project/AndrewPashkin/vagrant-appveyor-test...

    In short, I've added such configuration to my Vagrantfile:
    

    config.vm.provider "virtualbox" do |vb|
        vb.customize ["modifyvm", :id, "--nictype1", "Am79C973"]
    end
    

    How did I came to the solution:

    1. First of all, I used this method, to access a build workers via RDP:
       https://www.appveyor.com/docs/how-to/rdp-to-build-worker

    2. The other curical point was in enabling GUI for a VM:

       config.vm.provider "virtualbox" do |vb|
           vb.gui = true
       end
       

    3. Finally, I was able to see, what happens inside a VM and spot an error
       (see attached screenshot). Also, I saw this comment earlier.

    • Screenshot_2016-02-27_13-08-04.png 69 KB
    • Edit

19. 19 Posted by Stefan Scherer on 27 Feb, 2016 11:28 AM

    

    Hi Andrew,

    This is awesome! Thanks for sharing nictype1=Am79C973
    I'll try this.

    Stefan

    • Edit

20. 20 Posted by andrew.pashkin on 27 Feb, 2016 12:33 PM

    

    It is also impossible to launch 64-bit boxes. See attached image with the error.

    • Screenshot_2016-02-27_15-13-58.png 102 KB
    • Edit

21. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:04 AM.