Home → Questions →

Build cache manual update

• Edit

Eric J. Smith

22 Feb, 2015 11:47 PM

Is there some way for us to tell you that the build cache is still valid and to not update it? The problem that I'm running into is that it's actually faster for me to download and extract a zip file from a remote network every time than it is for you to restore a cache directory and then validate that it hasn't changed. If I could somehow manually tell you that the cache data hasn't changed then I could save you bandwidth and save us both some build time.

1. Support Staff 1 Posted by Feodor Fitsner on 23 Feb, 2015 12:01 AM

   

   But cache is always downloaded since build worker VMs are transient, i.e. do not preserve state between builds. If it's faster to download and extract zip instead of cache then go that way.

   • Edit

2. 2 Posted by Eric J. Smith on 23 Feb, 2015 12:36 AM

   

   I know that the cache is always downloaded. I'm talking about the part at the end of the build where it's checking to see if it needs to update the cached data. I am starting an ElasticSearch instance and I have to stop the service, which for some reason is really slow to do, or your cache check will get an error because the files are in use. I think what you could do that would be a generally useful feature to everyone is to have an option on the build cache to tell it to never update the cache unless the cache invalidation is triggered (ie. the yml file changes). In my case I would update my yml file to say which version of ES to download and that would trigger the cache to be invalidated and then you would store my cache folder one time and not check at the end of the build if it needs to be updated or not.

   I think this would be a generally useful feature that people could use to download things or set something up one time and have that cached until something triggers it to be invalidated.

   Does that make sense?

   • Edit

3. Support Staff 3 Posted by Feodor Fitsner on 23 Feb, 2015 12:40 AM

   

   Got it! You can specify dependencies for every cache item: https://github.com/appveyor/ci/issues/83

   In your case this could be something like:

   cache:
     - some_big.zip -> appveyor.yml
   

   Changing appveyor.yml will invalidate the cache.

   • Edit

4. 4 Posted by Eric J. Smith on 23 Feb, 2015 12:54 AM

   

   Yes, I know I can do that part. That controls whether or not you will extract the contents of my cache to the build environment on startup.

   What I want is an option that says capture my cache content only once each time that it is invalidated and then never update it at the end of the build. So don't do the checksums to see if the content is changed, just end the build and restore the same content you already had cached from the initial capture.

   cache:
     - some_big.zip -> appveyor.yml some_flag_or_syntax_to_mark_as_one_time_cache

   - first build
   1. You don't have a cache copy so you do nothing.
   2. I download and extract a zip in my build.
   3. Build ends and since you don't have a copy of my cached content, you capture my cache and save it.

   - second build (cache has not been invalidated)
   1. You see that you have my cache and you restore it.
   2. I do a build
   3. Build ends and since you already have a copy of my cache, you don't run checksums to see if the content has changed and you just end the build without saving my cache folder. I could even make changes to the cache folder if I wanted and it wouldn't matter because you aren't going to update it until the cache is invalidated.

   - new build with appveyor.yml change
   1. You see that the cache has been invalidated and you don't restore it.
   2. My build sees that I don't have my data cached and updates it.
   3. Build ends and since the cache was invalidated, you capture my cache data.

   • Edit

5. Support Staff 5 Posted by Feodor Fitsner on 23 Feb, 2015 02:03 AM

   

   But this is exactly how it works when you put dependency! :) When there is file(s) specified on the right AppVeyor stops tracking CRC of the content on the left (some_big.zip) and tracks CRC of dependency only (appveyor.yml). During the build you can do whatever you like with cached folder, even delete it - cache won't be updated util appveyor.yml is changed.

   • Edit

6. 6 Posted by Eric J. Smith on 23 Feb, 2015 05:42 AM

   

   Ahh... ok. Didn't realize that. Thanks for clarifying.

   I spent a lot of time playing around with cache today and unfortunately in my testing it was always faster to just download the files and restore the packages every single build than it was to use a cache. Seems counter-intuitive. Is this expected?

   • Edit

7. Support Staff 7 Posted by Feodor Fitsner on 23 Feb, 2015 05:39 PM

   

   The problem with the cache on Pro environment is latency as cache is stored on Azure. The cache was primarily made for OSS projects running on Azure to compensate relatively slow machines.

   In the meantime we are working on solution using some local storage on Pro environment for cache - this should be faster.

   • Edit

8. 8 Posted by ksykora on 18 Mar, 2016 07:17 PM

   

   Thanks! We're also noticing that on pro accounts that cache restoring is fairly slow (~1min for 204MB) -- Any improvement in this would be very welcome!

   • Edit

9. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:05 AM.