Build job has failed to start, but backup cloud is not configured.

pietroalbini's Avatar

pietroalbini

18 Jun, 2019 01:21 PM

Hello, I'm Pietro from the Rust Infrastructure team.

In the past few days we saw a couple jobs being killed a few minutes after being queued, with an empty log and this entry in the "Messages" tab: "Build job has failed to start, but backup cloud is not configured". These are the failed jobs:

Do you know what's happening here and how we could fix the issue?

Thanks,
Pietro.

  1. Support Staff 1 Posted by Ilya Finkelshte... on 18 Jun, 2019 05:05 PM

    Ilya Finkelshteyn's Avatar

    Hi Pietro,

    We see increasing error rate on VM disk creation on GCE. As a temporary workaround we increased number of retries for the whole VM provision operation so I hope for now in worst case build will stay longer in the Starting stage, but will finally start.

    For long-term fix we are working with GCE support on this.

    Also as always you can switch to our Hyper-V build cloud.

    Please keep us in touch if this re-occurs. We from our side will let you know about long-term resolution.

    Ilya.

  2. 2 Posted by pietroalbini on 18 Jun, 2019 08:05 PM

    pietroalbini's Avatar

    Also as always you can switch to our Hyper-V build cloud.

    Are you sure we aren't on Hyper-V already? We reverted the configuration change in March and it's not on our current appveyor.yml.

    Pietro.

  3. Support Staff 3 Posted by Ilya Finkelshte... on 18 Jun, 2019 09:46 PM

    Ilya Finkelshteyn's Avatar

    I think this is because you are using Visual Studio 2017 Preview image, which runs only on GCE. Do you need something special in this image in comparison to the ordinary Visual Studio 2017 one?

    Just FYI: we also have Visual Studio 2019 image (which runs on Azure Standard D2s v3 VMs). And as we told before you can connect AppVeyor to your own AWS/Azure/GCE account and use custom Linux and Windows images and VMs of any size.

  4. 4 Posted by pietroalbini on 19 Jun, 2019 09:47 PM

    pietroalbini's Avatar

    Do you need something special in this image in comparison to the ordinary Visual Studio 2017 one?

    Apparently not... We went directly from not pinning any image at all to using the preview in a PR sent by a sporadic contributor, and there doesn't seem to be a reason why the preview image was chosen over the standard one.

    I opened a PR on our end to switch to the standard image, so that should solve the issue as it doesn't affect Hyper-V!

  5. Support Staff 5 Posted by Ilya Finkelshte... on 19 Jun, 2019 09:51 PM

    Ilya Finkelshteyn's Avatar

    Good, meanwhile Google support confirmed that problem is not on our side and investigating.

  6. Support Staff 6 Posted by Ilya Finkelshte... on 20 Jun, 2019 06:19 PM

    Ilya Finkelshteyn's Avatar

    Google support reported resolution of the issue and I think you noticed it too. But I would still recommend to switch to the mainstream image because it is being updated more regularly. And it works on both Hyper-V and GCE if needed.

  7. Support Staff 7 Posted by Ilya Finkelshte... on 25 Jun, 2019 04:53 PM

    Ilya Finkelshteyn's Avatar

    We have to let you know that we again see delays on GCE VMs creation and again Google Cloud support confirmed the issue and working on it.

  8. Ilya Finkelshteyn closed this discussion on 25 Aug, 2019 09:01 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac