Orphaned VM build images in Azure

operations's Avatar

operations

20 Apr, 2020 05:21 AM

Hi,

We are using custom AppVeyor build images on Azure. I noticed that we are spending about US$600/month to store the VHDs for this, and found that there 55 VHDs in the relevant Azure storage account, spread over the last 18 months or so. These images are costing us about US$600/month to store. Only 8 are in use.

There are also excess Public IPs and Network Interfaces.

So I guess that sometimes the VM assets do not get cleaned up. Are we doing something wrong? I have deleted them now, but I imagine that they will keep appearing. Is this an AppVeyor problem, or is there something we are doing wrong?

Regards,
  Oliver

  1. Support Staff 1 Posted by Feodor Fitsner on 20 Apr, 2020 05:54 PM

    Feodor Fitsner's Avatar

    Hi Oliver,

    I'm really sorry about that!

    Based on our experience (we use Azure as a backup cloud for VS 2019 image), compared to other clouds, Azure could be absolutely horrible in locking resources! That's why we implemented heavy retry logic specifically for Azure cloud provider. You can see the number of retries and interval between retries under Provisioning settings section on Azure cloud settings. Try increasing the number of retries and the interval between them, say 20 mins. Although periodic checks of resource group with AppVeyor builds still won't hurt.

    Also, enable "Use managed disk" option - it will be provisioning Azure managed disks instead of copying blobs which could be also more stable.

    While you are observing the storage with new settings in place if you notice something is not removed please tell us VM name and build date to trace the logs and see what happened. It's absolutely possible that our removal logic is not ideal and could be improved!

    Let me know if you have any questions.

  2. 2 Posted by operations on 21 Apr, 2020 11:01 PM

    operations's Avatar

    Hi Feodor,

    Thanks for that - I will take both your suggestions, and get back to you if I see it again.

    Regards,
      Oliver

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac