Successful builds failing with code 259

acrichton's Avatar

acrichton

25 Jan, 2019 03:09 PM

We've recently had two builds that seem to fail spuriously:

where both report that they're failing with exit code 259, but according to the logs it looks like the build completed successfully (and the relevant code later landed without changes and passed all tests). We noticed that on Windows 259 corresponds to STILL_ACTIVE which may mean that a process isn't ready for exiting and it needs to be reaped later, so we were wondering if this is a possible bug in AppVeyor's harness?

Showing page 2 out of 2. View the first page

  1. Support Staff 31 Posted by Ilya Finkelshte... on 08 Mar, 2019 12:05 AM

    Ilya Finkelshteyn's Avatar

    PowerShell-based test did not return this error. However similar internal test based on cmd (not sh) returned 259. So we are convinced that it is an issue on our side, in the way we treat process external to the PowerShell. We will deploy a fix today later.

  2. Support Staff 32 Posted by Ilya Finkelshte... on 08 Mar, 2019 02:20 AM

    Ilya Finkelshteyn's Avatar

    Fix has been deployed. You can re-run your 200-minutes-sleep repro, or simple restore your appveyor.yml and build.sh.

    Sorry it took so long to realize the bug -- thanks to your repro.

  3. 33 Posted by pietroalbini on 08 Mar, 2019 06:53 AM

    pietroalbini's Avatar

    Thank you! I reopened our queue so a PR is now testing, I'll let you know of the results.

    Pietro.

  4. 34 Posted by pietroalbini on 08 Mar, 2019 04:00 PM

    pietroalbini's Avatar

    Yep, we had a build going over 3 hours and it didn't fail \o/

    https://ci.appveyor.com/project/rust-lang/rust/builds/22921540

    Thanks!
    Pietro.

  5. Support Staff 35 Posted by Ilya Finkelshte... on 08 Mar, 2019 06:14 PM

    Ilya Finkelshteyn's Avatar

    Thanks so much for your persistence, and sorry once again it took this long to check some specific place in our system. Please feel free to ping anytime :)

  6. 36 Posted by acrichton on 11 Mar, 2019 08:15 PM

    acrichton's Avatar

    I don't believe we've seen this in the past few days, so I'm gonna go ahead and close this from our end.

  7. acrichton closed this discussion on 11 Mar, 2019 08:15 PM.

  8. pietroalbini re-opened this discussion on 29 Mar, 2019 06:20 PM

  9. 37 Posted by pietroalbini on 29 Mar, 2019 06:20 PM

    pietroalbini's Avatar

    We just had another build failing with the 259 error code.

    https://ci.appveyor.com/project/rust-lang/rust/builds/23456316/job/8jm7me4fhgcks5pi

    The symptoms are the same as the previous failures, as this happened after 3 hours and the build otherwise passed, so I guess the repro would also be the same. We don't have the spare capacity on our account to run it though.

  10. Support Staff 38 Posted by Ilya Finkelshte... on 29 Mar, 2019 07:28 PM

    Ilya Finkelshteyn's Avatar

    Hmm... Let me please explain how we apply hotfixes to build workers. Each build image update we bake latest build agent bits into VHD. However updating image for quick fixes is too hard way to go. So we have a code which check if newer build agent available and replace it during VM start. If it cannot be downloaded, default one (baked in the image) works. I afraid this happens here -- non-patched agent run this specific build. If I am right, this is unlikely to happen again. And in any case we will release new image with this fix baked in a week or so. Sorry for the trouble.

  11. 39 Posted by pietroalbini on 29 Mar, 2019 08:01 PM

    pietroalbini's Avatar

    That's reassuring, thanks for the clarification!

    Pietro.

  12. Support Staff 40 Posted by Ilya Finkelshte... on 31 Mar, 2019 09:52 PM

    Ilya Finkelshteyn's Avatar

    Pietro: can you please add appveyor version command sometime in the beginning of the build (say init or install) so in case it happen again (hope not) we check the agent version?

  13. 41 Posted by pietroalbini on 31 Mar, 2019 09:58 PM

    pietroalbini's Avatar

    Sure! Sent a PR to add it, it will land in a few days max.

    Pietro.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac