Frequent failures because SQL Server is not able to start

mothman's Avatar

mothman

09 Nov, 2018 09:03 PM

In the last couple of weeks, we've been getting build fails because SQL Server, which we need to run tests, cannot start on the build worker.

Here's the error message:

Starting SQL Server 2016 Developer
Start-Service : Service 'SQL Server (SQL2016) (MSSQL$SQL2016)' cannot be started due to the following error: Cannot start service MSSQL$SQL2016 on computer '.'.
At line:1 char:1
+ Start-Service 'MSSQL$SQL2016'
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OpenError: (System.ServiceProcess.ServiceController:ServiceController) [Start-Service], ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStartService,Microsoft.PowerShell.Commands.StartServiceCommand

Building the same commit again may pass or sometimes fail again with the same issue.
This is causing our build to take much longer than usual only to fail and tip off developers to think they may have issues in their code and this is disrupting our process.

  1. Support Staff 1 Posted by Owen McDonnell on 10 Nov, 2018 07:23 AM

    Owen McDonnell's Avatar

    This is likely a performance issue due to some upgrades happening on our servers. We try to keep it as transparent as possible thought there are some occasional manifestations.

    If this persists past the weekend please let us know.

  2. Support Staff 2 Posted by Ilya Finkelshte... on 12 Nov, 2018 07:42 PM

    Ilya Finkelshteyn's Avatar

    @mothman: things should be good now. If you see any slowness or errors, please let us know.

  3. 3 Posted by mothman on 15 Nov, 2018 04:06 PM

    mothman's Avatar

    Thank you Owen and Ilya, it is much better this week.
    However, I just ran into the same issue in one build:
    https://ci.appveyor.com/project/azavar/lata/builds/20329431?fullLog...

  4. Support Staff 4 Posted by Ilya Finkelshte... on 15 Nov, 2018 08:36 PM

    Ilya Finkelshteyn's Avatar

    Are you starting SQL server from custom script or as described here?

  5. 5 Posted by mothman on 15 Nov, 2018 08:42 PM

    mothman's Avatar

    I'm using the services section in appveyor.yml:

    services:
      - mssql2016
    
  6. Support Staff 6 Posted by Ilya Finkelshte... on 16 Nov, 2018 12:49 AM

    Ilya Finkelshteyn's Avatar

    Checked that build and I believe that it is still a residual effects of I/O issues we had recently and fixed 2 days ago. I think it should not re-occur.

  7. 7 Posted by mothman on 26 Nov, 2018 02:34 PM

    mothman's Avatar
  8. Support Staff 8 Posted by Ilya Finkelshte... on 26 Nov, 2018 07:47 PM

    Ilya Finkelshteyn's Avatar

    I see. Though we solved capacity issues, some nodes in our datacenter seems to have I/O issues. I believe this build happened to run on one of those nodes. We are in the process of replacing them. I cannot give specific date, but I should be completed in 2-3 weeks.Meanwhile you can replace services.mssql2016 with - appveyor-retry net start MSSQL$SQL2016 at say install stage. This will be more reliable for now.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac