Deployment Agent Hangs

byron's Avatar

byron

06 Mar, 2015 05:09 PM

Sometimes I notice that an agent-based deployment doesn't happen. I check the server and see that there are no events showing errors and the Deployment Agent is happily running in the services panel. AppVeyor also reports no errors... just that the server never responded to the deployment signal. The work-around right now is to restart the service. Then it begins deploying again. Any thoughts on this?

  1. Support Staff 1 Posted by Feodor Fitsner on 06 Mar, 2015 10:21 PM

    Feodor Fitsner's Avatar

    Yeah, we are going to improve that with "heartbeat" loop: https://github.com/appveyor/ci/issues/130

  2. 2 Posted by byron on 07 Mar, 2015 12:13 AM

    byron's Avatar

    Are there any ways to trick it into working? I don't think a daily reboot will be sufficient. 

    --
    Byron Sommardahl
    CTO, Co-Owner
    Acklen Avenue | www.acklenavenue.com | 615-669-8239
    230 4th Ave N Suite 301H Nashville, TN 37219

    Acklen Avenue is a custom software development company committed to
    creating quality software that people love to use.

  3. Support Staff 3 Posted by Feodor Fitsner on 07 Mar, 2015 12:15 AM

    Feodor Fitsner's Avatar

    Why do you need reboot? Deployment Agent service restart usually is enough. Or what do you mean?

  4. 4 Posted by byron on 07 Mar, 2015 12:17 AM

    byron's Avatar

    Well, something I can automate. 

    --
    Byron Sommardahl
    CTO, Co-Owner
    Acklen Avenue | www.acklenavenue.com | 615-669-8239
    230 4th Ave N Suite 301H Nashville, TN 37219

    Acklen Avenue is a custom software development company committed to
    creating quality software that people love to use.

  5. Support Staff 5 Posted by Feodor Fitsner on 07 Mar, 2015 12:27 AM

    Feodor Fitsner's Avatar

    Rebooting server is not an option, of course. I will be experimenting with SignalR re-connection in the coming days and release Agent update. Will keep you posted about results.

  6. 6 Posted by sebastian on 11 Jun, 2015 03:43 PM

    sebastian's Avatar

    I'm also getting this error quite regularly.

    When I check the Deployment Agent's event log, I get warnings that say 'SignalR connection is slow.'

    No sign of having received the request from AppVeyor.

  7. 7 Posted by rajindersingh on 18 Jun, 2015 05:21 PM

    rajindersingh's Avatar

    I also got the same error. I restarted the app deployment agent and it fixed the problem.
    Event log showed signalr connection is slow".
    I have doing a deployment to azure vm.

  8. Support Staff 8 Posted by Feodor Fitsner on 19 Jun, 2015 05:59 PM

    Feodor Fitsner's Avatar

    Going to dedicate more time to this instability in the coming weeks.

  9. 9 Posted by sebastian on 19 Jun, 2015 07:01 PM

    sebastian's Avatar

    Let us know if we can help in any way!

  10. 10 Posted by sebastian on 25 Jun, 2015 08:52 AM

    sebastian's Avatar

    Hi Feodor, I've noticed a few more different kinds of warnings and errors in our server's deployment agent's event log. I've zipped them up for you, but would prefer to send it privately rather than attach it to this convo. Is it easiest if I email it to you?

  11. Support Staff 11 Posted by Feodor Fitsner on 25 Jun, 2015 02:11 PM

    Feodor Fitsner's Avatar

    Yes, please send to "team AT appveyor.com". Thank you!

    - Feodor

  12. 12 Posted by sebastian on 25 Jul, 2015 08:55 AM

    sebastian's Avatar

    Quick update to keep the issue alive - still getting this issue intermittently.

  13. 13 Posted by sebastian on 20 Aug, 2015 02:00 PM

    sebastian's Avatar

    Hi Feodor,

    We're still getting this issue quite regularly and we're have to always be on the guard whether or not our deployments actually ran or not.

    The deployment agent doesn't register itself for the deployment and the deployment just quietly succeeds with no error, no action taken, example here: https://ci.appveyor.com/project/Martaver/neway-webfreight/deploymen...

    Looking at our server, in our Event Viewer, there's no errors (or events) from the agent from the last 6 hours.

    Just like for byron, restarting the service helps, but we're hoping for a more permanent solution. We're happy to help however we can, but we'd just really like to not have to worry about this anymore.

    Cheers,
    Seb

  14. 14 Posted by david on 24 Aug, 2015 02:15 PM

    david's Avatar

    Same issue here. Periodically, I'll see "Error subscribing agent events: The operation was canceled." in the Event log, and the deployment will quietly "succeed" with no jobs having been run. Restarting the service then fixes the issue for a week or so.

    I wonder if a deployment with no agents should always be consider failed?

    If there's more of the event log or anything I can attach that would be helpful, let me know.

  15. 15 Posted by sebastian on 25 Aug, 2015 08:11 AM

    sebastian's Avatar

    David's suggestion about failing a deployment with no agents seems like a good interim solution from our point of view.

  16. Support Staff 16 Posted by Feodor Fitsner on 26 Aug, 2015 05:43 PM

    Feodor Fitsner's Avatar

    Hi guys,

    We've got an updated Deployment Agent with connection reliability improvements and heartbeat added: http://www.appveyor.com/downloads/3.1.7/AppveyorDeploymentAgent.msi (it's not yet public).

    Please clear "Deployment Agent" event log before installing updated agent to start collecting "fresh" data.

    If anything goes wrong you can always revert to a previous one: http://www.appveyor.com/downloads/AppveyorDeploymentAgent.msi

  17. 17 Posted by david on 31 Aug, 2015 06:41 PM

    david's Avatar

    The updated version has been working fine for me since 8/27. That's not a lot of time but I wanted to chime in that it's off to a good start :-)

  18. 18 Posted by Sebastian Nemet... on 31 Aug, 2015 08:53 PM

    Sebastian Nemeth's Avatar

    Ditto! So far, no complaints!

  19. Support Staff 19 Posted by Feodor Fitsner on 01 Sep, 2015 04:51 PM

    Feodor Fitsner's Avatar

    Nice, nice! Thanks for the update :)

  20. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 01:59 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac