Intermittent hanging builds

Igor's Avatar

Igor

26 Jan, 2019 01:35 AM

Hi there,
We're having troubles with intermittently hanging builds in Git Extensions project.
https://github.com/gitextensions/gitextensions/issues/6172

Any help in tracking and fixing the issue would be highly appreciated.

Thank you,
Igor

  1. Support Staff 1 Posted by Ilya Finkelshte... on 26 Jan, 2019 04:01 AM

    Ilya Finkelshteyn's Avatar

    Hi Igor,

    Set up non-blocking RDP. When it hung next time connect to build VM and see what happens. It seems like GUI test issue, so you hopefully will see the problem (some dialog box waiting for input or similar) right away.

    Ilya.

  2. 2 Posted by Igor Velikoross... on 28 Jan, 2019 04:08 AM

    Igor Velikorossov's Avatar

    Thank you, but I couldn't get it to work for some reason.

    I followed the steps outlined on the page, however I was unable to login.

    I tried the following:

    * I used 1Password to generate a 16 char long password with 3 numbers
    (A-Za-z0-9)
    * On
    https://ci.appveyor.com/project/gitextensions/gitextensions/settings/environment
    page added a new variable APPVEYOR_RDP_PASSWORD with the generated password
    * Added the PS script at the "init" stage to get RDP info
    The above resulted in login failed.
    [image: image.png]

    Then I tried:
    * Remove the above env variable
    * I encrypted the password on
    https://ci.appveyor.com/account/gitextensions/tools/encrypt page
    * Added APPVEYOR_RDP_PASSWORD variable with encrypted password to
    appveyor.yml
    [image: image.png]
    This did not work either.

    What am I doing wrong?

    Thank you
    Igor

  3. Support Staff 3 Posted by Ilya Finkelshte... on 28 Jan, 2019 08:22 PM

    Ilya Finkelshteyn's Avatar

    I do not see any issues with your steps, other than copy-paste errors which is not very probable. I would try with simpler manually set password for beginning. But actually, if you are in hurry to fix your problem with stuck test you can skip APPVEYOR_RDP_PASSWORD completely -- AppVeyor we generate and display randomly created password.

  4. 4 Posted by Igor Velikoross... on 28 Jan, 2019 09:16 PM

    Igor Velikorossov's Avatar

    Thank you Ilya, that worked.
    I guess my password was the culprit, though I believe it should have met
    the requirements: fuu48sZA6nUQwvaf

    I'll try to generate a new password and try again.

    Regards, Igor

  5. 5 Posted by Igor Velikoross... on 29 Jan, 2019 06:29 AM

    Igor Velikorossov's Avatar

    It is actually quite bizarre.

    I set the password as per the guidelines:
    [image: image.png]
    I tried both upper and lower cased vars, as the guidelines are
    inconsistent. As expected it is the same variable.

    I see that the password variable is set:
    [image: image.png]

    However trying to connect to the RD the password is rejected...

    I'm starting to think something under the hood in Appveyor isn't quite
    working.

    Igor

  6. Support Staff 6 Posted by Ilya Finkelshte... on 30 Jan, 2019 12:30 AM

    Ilya Finkelshteyn's Avatar

    Looking into it. Can you please send a links to those builds?

    And also did you find the root cause for the original problem with stuck test?

  7. 7 Posted by Igor on 30 Jan, 2019 12:43 AM

    Igor's Avatar

    Here's the link to our builds - https://ci.appveyor.com/project/gitextensions/gitextensions/history

    This is the last build I tried from "Add_appveyor_non_blocking_rdp" branch: https://ci.appveyor.com/project/gitextensions/gitextensions/builds/21959504
    I have changed passwords since though.

    I haven't found a root cause for the hanging builds, the issue is quite intermittent.
    I don't feel comfortable exposing RDP details for an unknown period of time.
    When I would be in a position to set a password I could share it with the rest of the core team to jump on a box when we see the issue.

    Thank you,
    Igor

  8. Support Staff 8 Posted by Ilya Finkelshte... on 30 Jan, 2019 01:32 AM

    Ilya Finkelshteyn's Avatar

    What are you risking with open source project? Variables, including secure ones, are not available in RDP session.

  9. 9 Posted by Igor Velikoross... on 30 Jan, 2019 03:25 AM

    Igor Velikorossov's Avatar

    I don't like exposing a compute resource.

  10. Support Staff 10 Posted by Ilya Finkelshte... on 30 Jan, 2019 11:01 AM

    Ilya Finkelshteyn's Avatar

    Please try now.

  11. 11 Posted by Igor Velikoross... on 30 Jan, 2019 08:57 PM

    Igor Velikorossov's Avatar

    I am afraid the result is the same - my credentials (u: appveyor, p:
    specified via envvar) get rejected.
    Here's the last build
    https://ci.appveyor.com/project/gitextensions/gitextensions/builds/22009613,
    password is unchanged, if you wish to try.

  12. Support Staff 12 Posted by Ilya Finkelshte... on 30 Jan, 2019 09:02 PM

    Ilya Finkelshteyn's Avatar

    Are you sure you are trying to RDP when build is still running?

  13. 13 Posted by Igor Velikoross... on 30 Jan, 2019 09:33 PM

    Igor Velikorossov's Avatar

    Yes, I am
    It fails at credentials negotiation stage

  14. 14 Posted by Igor Velikoross... on 30 Jan, 2019 09:35 PM

    Igor Velikorossov's Avatar

    I'll do a screen grab in my evening to show

  15. Support Staff 15 Posted by Ilya Finkelshte... on 30 Jan, 2019 10:41 PM

    Ilya Finkelshteyn's Avatar

    Try blocking RDP to ensure build is not finished and VM is not destroyed by the time you connect.

  16. 16 Posted by Igor on 01 Feb, 2019 11:03 AM

    Igor's Avatar

    Here's an animation.
    The password is setup as per the screenshot above.

  17. Support Staff 17 Posted by Ilya Finkelshte... on 02 Feb, 2019 01:30 AM

    Ilya Finkelshteyn's Avatar

    Thanks a lot for your persistence. Try now :)

  18. 18 Posted by Igor Velikoross... on 02 Feb, 2019 04:11 AM

    Igor Velikorossov's Avatar

    It worked! Thank you for your help

  19. Support Staff 19 Posted by Ilya Finkelshte... on 02 Feb, 2019 09:46 PM

    Ilya Finkelshteyn's Avatar

    Note that because of some tricky race condition it takes longer at init stage now. If you put RDP script later, but still before your tests (at install stage for example), you will save a little bit of build time.

  20. 20 Posted by Igor Velikoross... on 03 Feb, 2019 12:26 PM

    Igor Velikorossov's Avatar

    Thank you, noted.

  21. Owen McDonnell closed this discussion on 06 Feb, 2019 08:43 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac