tag:help.appveyor.com,2012-11-13:/discussions/problems/18487-please-do-not-overwrite-the-cache-if-it-failed-to-downloadAppVeyor: Discussion 2019-06-27T05:27:38Ztag:help.appveyor.com,2012-11-13:Comment/466403782018-12-10T07:37:27Z2018-12-10T07:37:31ZPlease do not overwrite the cache if it failed to download<div><p>Not only download fails. Sometimes extraction fails:</p>
<p><a href="https://ci.appveyor.com/project/josch/3dtk/builds/20889529">https://ci.appveyor.com/project/josch/3dtk/builds/20889529</a></p>
<p>With an error like this:</p>
<p>Error uncompressing cache item: 7z.exe process has exited with code -1073741510. Check C:\Users\appveyor\AppData\Local\Temp\1\build-cache-logs\88e45141cae35ced15beaaf29b99d251bd277254.zip.001.log for details.</p>
<p>But the log file doesn't contain anything interesting:</p>
<p>7-Zip 18.05 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30</p>
<p>Scanning the drive for archives:<br>
1 file, 30368484 bytes (29 MiB)</p>
<p>Extracting archive: C:\Users\appveyor\AppData\Local\Temp\1\pzjcvx01.ygj\88e45141cae35ced15beaaf29b99d251bd277254.zip.001</p></div>joschtag:help.appveyor.com,2012-11-13:Comment/466403782018-12-10T22:49:44Z2018-12-10T22:49:44ZPlease do not overwrite the cache if it failed to download<div><p>In this case, why do you have <code>APPVEYOR_SAVE_CACHE_ON_ERROR</code> set to true?<br>
Also, i wonder if you're running into account wide <a href="https://www.appveyor.com/docs/build-cache/#cache-size-beta">cache limit</a> which is 1GB on free account. This build alone looks like it has a cache bigger than 500MB.</p></div>Owen McDonnelltag:help.appveyor.com,2012-11-13:Comment/466403782018-12-10T23:16:05Z2018-12-10T23:16:07ZPlease do not overwrite the cache if it failed to download<div><p>Filling the cache takes more than the time limit of 1 hour. So I need the cache to be saved even after the build timed out. I need to restart it several times until the cache slowly filled itself.</p>
<p>I don't think I'm running into the 1GB limit, because our recent builds are working just fine:</p>
<p><a href="https://ci.appveyor.com/project/josch/3dtk/builds/20908907">https://ci.appveyor.com/project/josch/3dtk/builds/20908907</a></p>
<p>Why is the setting of <code>APPVEYOR_SAVE_CACHE_ON_ERROR</code> relevant for this problem?</p></div>joschtag:help.appveyor.com,2012-11-13:Comment/466403782018-12-11T06:55:24Z2018-12-11T06:55:24ZPlease do not overwrite the cache if it failed to download<div><p>I mentioned <code>APPVEYOR_SAVE_CACHE_ON_ERROR</code> for cases when the extraction fails and you don't want to overwrite cache. Not sure that's what's going on here though.</p>
<p>The cache restore errors may have been some stochastic network error. Looks like most recent builds are not experiencing this error?</p>
<p>As far as sane defaults, I think most users prefer to allow a build to continue in the case of cache failures. <a href="https://github.com/appveyor/ci/issues/1509">This issue</a> suggests so.</p></div>Owen McDonnelltag:help.appveyor.com,2012-11-13:Comment/466403782018-12-11T08:21:11Z2018-12-11T08:21:12ZPlease do not overwrite the cache if it failed to download<div><p>If <code>APPVEYOR_SAVE_CACHE_ON_ERROR</code> had any influence, then appveyor would exit with an error after the cache failed to download or extract but that's not what it's doing. It's happily continueing the build without any error. So there is no way for my build script to know whether the cache failed to download or whether we are just starting off with a fresh cache.</p>
<p>I imagine that other users have different expectations. That's why I suggested points two and three in my initial message:</p>
<ul>
<li>an option that allows to at least not overwrite the existing cache with a new one or</li>
<li>a way for my build script to find out whether the cache failed to download</li>
</ul>
<p>That way, any user could decide for themselves how they want to react to "stochastic network errors".</p>
<p>For us, "stochastic network errors" are a real pain, because it takes several rebuilds to fully populate the cache again.</p></div>joschtag:help.appveyor.com,2012-11-13:Comment/466403782018-12-11T17:45:02Z2018-12-11T17:45:02ZPlease do not overwrite the cache if it failed to download<div><p>The first build you linked to failed because there was an error in restoring the cache, so if, in the case of a cache restore error you do not want to save that now incomplete cache, then don't set <code>APPVEYOR_SAVE_CACHE_ON_ERROR</code> to true. You can always set it to true when you need to build up a new cache.</p>
<p>But most importantly, is this a frequent problem for you? Looking through your build history I could only find that one single occurrence.</p></div>Owen McDonnelltag:help.appveyor.com,2012-11-13:Comment/466403782018-12-11T18:41:19Z2018-12-11T18:41:20ZPlease do not overwrite the cache if it failed to download<div><p>No, the first build I linked to failed because "vcpkg install failed" (see line 492).</p>
<p>If <code>APPVEYOR_SAVE_CACHE_ON_ERROR</code> indeed means "don't save the cache when there is an error retrieving it" then the documentation of that variable is wrong. The docs say:</p>
<blockquote>
<p>save build cache on build failure. By default build cache is being saved only during successful build Finalize steps</p>
</blockquote>
<p>Yes, it's a frequent problem. Just today I got this:</p>
<p><a href="https://ci.appveyor.com/project/josch/3dtk/builds/20921897">https://ci.appveyor.com/project/josch/3dtk/builds/20921897</a></p>
<p>It was lucky that I was attentive because if I hadn't paused the build, then the build cache would've wrongly been updated.</p>
<p>Right now the problem is, that a "cache restore error" is not treated as an error. If it were treated as an error, then the build would stop immediately. But it doesn't as you can see from the build logs. And you also already said that it's undesirable in the default case to treat cache restore failures as an error and abort the build as a result..</p></div>joschtag:help.appveyor.com,2012-11-13:Comment/466403782018-12-11T22:25:13Z2018-12-11T22:25:13ZPlease do not overwrite the cache if it failed to download<div><p>I understand the build failure's proximate cause was the vcpkg install, but I assumed that was due to the cache restore error.</p>
<p>In any case, two things to add.</p>
<p>First, we increased your build time to 90 minutes, so perhaps you can fill your cache in one build now (though keep in mind that there are 3 different datacenters that builds occur in and cache is only saved to most recent one so it may take some time to populate all of them upon cache changes).</p>
<p>Secondly, caching is meant as a best effort feature to speed up builds and act as a fail safe if nuget/npm/maven repos have issues. Your build should not rely on it 100% and package managers <em>should</em> play nicely with it (i.e. download what is needed if the cache is not restored). This is why we don't want cache failures to stop the build.</p></div>Owen McDonnelltag:help.appveyor.com,2012-11-13:Comment/466403782018-12-11T23:04:08Z2018-12-11T23:04:10ZPlease do not overwrite the cache if it failed to download<div><p>Okay, if you think that allowing the user to choose whether stochastic network errors during build cache downloads make the whole build fail or not is not something in scope of appveyor, then we are nevertheless grateful for the bumped build time (it will help us a lot for filling up the cache) and you can consider this issue closed as wontfix. Thanks!</p></div>josch