Spurious failures uploading to S3

acrichton's Avatar

acrichton

10 Mar, 2017 04:35 AM

Hello! Over the past few months we've had a number of spurious failures on the rust-lang/rust repository due to the deployment section of our appveyor.yml. It appears that AppVeyor hits an error when uploading to S3 which will fail our build.

We've been cataloging failures where the linked PRs have failures like this which has logs like:

Deploying using S3 provider
Uploading artifact "6a91d29be55596ca30b2c1914644dce8c4192356/rust-docs-1.16.0-dev-x86_64-pc-windows-msvc.tar.gz" (0 bytes) to S3 bucket "rust-lang-ci" as "rustc-builds/6a91d29be55596ca30b2c1914644dce8c4192356/rust-docs-1.16.0-dev-x86_64-pc-windows-msvc.tar.gz"
Uploading artifact "6a91d29be55596ca30b2c1914644dce8c4192356/rust-docs-1.16.0-dev-x86_64-pc-windows-msvc.tar.gz.sha256" (0 bytes) to S3 bucket "rust-lang-ci" as "rustc-builds/6a91d29be55596ca30b2c1914644dce8c4192356/rust-docs-1.16.0-dev-x86_64-pc-windows-msvc.tar.gz.sha256"
Uploading artifact "6a91d29be55596ca30b2c1914644dce8c4192356/rust-src-1.16.0-dev.tar.gz" (0 bytes) to S3 bucket "rust-lang-ci" as "rustc-builds/6a91d29be55596ca30b2c1914644dce8c4192356/rust-src-1.16.0-dev.tar.gz"
A WebException with status ReceiveFailure was thrown.

My assumption is that these are "standard spurious network failures" in the sense that there's no real cause, but I was wondering if we could perhaps have an option to automatically retry failed requests? Or if not, you wouldn't happen to have a suggestion of how to cope with such spurious failures, would you?

  1. Support Staff 1 Posted by Ilya Finkelshte... on 11 Mar, 2017 03:43 AM

    Ilya Finkelshteyn's Avatar

    Yes thanks for proposal, we can do this. Please watch this issue.

    Ilya.

  2. 2 Posted by acrichton on 11 Mar, 2017 04:24 PM

    acrichton's Avatar

    Awesome, thank you!

  3. Support Staff 3 Posted by Ilya Finkelshte... on 23 Mar, 2017 03:03 AM

    Ilya Finkelshteyn's Avatar

    It is done. Please let us know how it goes over time. If we see it reoccurs, we can try to add configurable timeout.

  4. 4 Posted by acrichton on 23 Mar, 2017 02:09 PM

    acrichton's Avatar

    Adding the configuration soon and I'll be sure to ping you if it happens again. Thanks so much again!

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

18 Oct, 2017 08:54 AM
18 Oct, 2017 08:21 AM
18 Oct, 2017 07:42 AM
17 Oct, 2017 07:36 PM
17 Oct, 2017 05:39 PM

 

17 Oct, 2017 04:29 PM
17 Oct, 2017 07:49 AM
16 Oct, 2017 11:38 PM
16 Oct, 2017 10:01 PM
16 Oct, 2017 08:25 AM
16 Oct, 2017 07:58 AM