Home → Problems →

Repeated build failures due to NuGet package restore timeouts

• Edit

James Skimming

30 Sep, 2014 04:00 PM

We're receiving fairly regular (about 1 in 5) build failures due to timeouts when restoring a solution's NuGet packages (in the before_build step), the error often looks like this (the packages differ every time):

The operation has timed out
Unable to find version '3.2.2.0' of package 'Ninject'.
Unable to find version '3.2.0.0' of package 'Ninject.Extensions.Factory'.
Command exited with code 1

Given AppVeyor creates a new VM every build means the package restore is never cached and as a result the build is dependant on the NuGet servres, I'm not aware of how AppVeyor provisions the VM, I'm amusing there's a VM HDD reset, it is possible the VM IPs are reused and NuGet is throttling the requests?

Short of checking in the packages to source control, is there anything that can be done to mitigate the problem?

1. Support Staff 1 Posted by Feodor Fitsner on 30 Sep, 2014 04:31 PM

   

   Have you looked at build cache solution: http://www.appveyor.com/docs/build-cache ?

   • Edit

2. Support Staff 2 Posted by Feodor Fitsner on 30 Sep, 2014 04:33 PM

   

   I've just got the same error: https://ci.appveyor.com/project/appvyr/simple-web

   Seems like a NuGet.org issue?..

   • Edit

3. Support Staff 3 Posted by Feodor Fitsner on 30 Sep, 2014 04:40 PM

   

   Right, I've removed local NuGet cache and did nuget restore. This is what I currently have on my dev machine - see attached.

   • nuget-timeout-issue.png 32.3 KB
   • Edit

4. 4 Posted by James Skimming on 01 Oct, 2014 10:03 AM

   

   The build cache is awesome, I've chosen to just store the NuGet Cache and not the package folder directly, as that way the build is "cleaner". Here is my appveyor.yml:

   # Retain the NuGet package cache between builds.
   cache:
     - C:\Users\appveyor\AppData\Local\NuGet\Cache
   

   We have another project that's a mixture of .NET and AngulaJS and we're using node NPM and bower as well as NuGet, I've went for the following config

   cache:
     - node_modules
     - C:\Users\appveyor\AppData\Local\NuGet\Cache
     - C:\Users\appveyor\AppData\Roaming\npm-cache
     - C:\Users\appveyor\AppData\Roaming\bower\cache\packages
   

   I chosen to keep the node modules as installing the node packages takes the majority the build time, (I think because it's building a lot of the packages like phantomjs) But it's not restoring the folder node_modules even though that's the example given in the (documentation)(http://www.appveyor.com/docs/build-cache).

   Am I doing something wrong?

   • Edit

5. 5 Posted by James Skimming on 01 Oct, 2014 10:28 AM

   

   Ignore that last comment, I need to put the relative path to the node_modules as it's not off the root of the repository.

   But now I'm receiving the following error after the build

   The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
   

   I suspect it's caused by the deep nesting of node packages. Other than moving the site to the root of the repository, do you know if this can be resolved?

   • Edit

6. Support Staff 6 Posted by Feodor Fitsner on 01 Oct, 2014 03:10 PM

   

   I'll be working on "path too long" issue with build cache. Have some ideas to try.

   What are those NPM packages giving long paths I can try locally to reproduce the issue?

   • Edit

7. 7 Posted by James Skimming on 01 Oct, 2014 08:17 PM

   

   It's just the node packages of the AngularSeed project. I've created a build from my fork, but it's not failing presumably because the project is in he root of the repo.

   If you take the same packages and build it from a deeper folder structure I believe it will fail.

   • Edit

8. Support Staff 8 Posted by Feodor Fitsner on 02 Oct, 2014 06:11 AM

   

   Hi James,

   We've deployed AppVeyor update with a fix for "path too long" issue. You can enable cache for folders of any depth now.

   • Edit

9. 9 Posted by James Skimming on 02 Oct, 2014 09:43 AM

   

   Hi Feodor

   Thanks for the quick turn around.

   I've tried and I'm no longer receiving the path too long error, but I'm receiving a failure with bower.

   In my angular-seed fork, you can see the package restore worked fine when the site was in the rot of the repo, see successful build 0.1.6 using the build cache.

   I then moved the site to a deeper folder. The initial build 0.1.8 went through fine as the node packaged were rebuilt due to not having been cached.

   I then kicked off another build 0.1.9 where the node packages were restored from the build cache, but the build failed because I don't think the packages were correctly restored, resulting in the error with bower.

   Are you able to look at the project to investigate, it's all open source, so you can fork my fork and run it for yourself on the DeepFolders branch..

   • Edit

10. Support Staff 10 Posted by Feodor Fitsner on 02 Oct, 2014 02:28 PM

    

    Will take a look.-Feodor

    • Edit

11. Support Staff 11 Posted by Feodor Fitsner on 03 Oct, 2014 09:28 PM

    

    Could you give it another try please? We've deployed an update with some fixes.

    • Edit

12. 12 Posted by James Skimming on 03 Oct, 2014 09:50 PM

    

    Hi Feodor

    I'm afraid it didn't work. The first build 0.1.10 worked as there was no cache to restore (did you delete it?) at the end the node_modules appear to be saved fine.

    I then kicked off another build 0.1.11 but this time the build failed evan earlier at the point where it was restoring the node_modules.

    Hope this helps with your investigations.

    • Edit

13. Support Staff 13 Posted by Feodor Fitsner on 03 Oct, 2014 09:53 PM

    

    Great, will continue digging.

    • Edit

14. 14 Posted by James Skimming on 03 Oct, 2014 09:55 PM

    

    Hi Feodor

    I'd getting late here in the UK and I'll soon be shutting down for the night, if you want to continue using my build, give me your email address and I'll add you to the Team, I'm following @appveyor so you can DM me @JamesSkimming you don't want to post it here.

    • Edit

15. Support Staff 15 Posted by Feodor Fitsner on 03 Oct, 2014 10:01 PM

    

    No worries, I'll setup it locally.

    • Edit

16. 16 Posted by James Skimming on 15 Oct, 2014 08:28 AM

    

    Did you manage to get any further on this? Caching the various package caches has mostly mitigated any build issues.

    I suspect this one could be a bit of a pain to solve and is inherent in the 255 character file path limit Windows sometimes has. Therefore in the interest of pragmatism, we can live with the occasional failed builds especially if you can provide the rebuild button.

    • Edit

17. Support Staff 17 Posted by Feodor Fitsner on 15 Oct, 2014 05:27 PM

    

    You mean this issue has sporadic nature?

    • Edit

18. 18 Posted by James Skimming on 15 Oct, 2014 05:48 PM

    

    Sorry didn't mean to confuse. The node package restore where the project root is deep but using the build cache to cache the node_modules folder is 100% reproducible. We're now running without that setting.

    I initially raised this because of NuGet, Bower and npm restore issues. Adding the local package caches to the build cache has mostly mitigated the issue, but npm install still occasionally fails.

    Basically, if fixing this too much effort, we'd be happy to live with the occasional build failures if we had a rebuild button.

    • Edit

19. Support Staff 19 Posted by Feodor Fitsner on 15 Oct, 2014 10:44 PM

    

    Sure, I'm going to investigate this caching issue.

    • Edit

20. Support Staff 20 Posted by Feodor Fitsner on 16 Oct, 2014 04:12 AM

    

    Hi James,

    Build cache for long files names has been finally fixed!

    Looks at the results: https://ci.appveyor.com/project/appvyr/angular-seed/build/0.1.2

    Almost 2 times faster: https://ci.appveyor.com/project/appvyr/angular-seed/history :)

    • Edit

21. James Skimming closed this discussion on 27 Dec, 2015 10:58 PM.