tag:help.appveyor.com,2012-11-13:/discussions/problems/17697-all-builds-failing-due-to-artifact-storage-limit-exceededAppVeyor: Discussion 2018-11-29T12:53:45Ztag:help.appveyor.com,2012-11-13:Comment/463516522018-10-25T17:17:40Z2018-10-25T17:17:40ZAll builds failing due to artifact storage limit exceeded<div><p>Your artifact storage size has been increased. Let us know if you can build again.</p></div>Owen McDonnelltag:help.appveyor.com,2012-11-13:Comment/463516522018-10-25T18:22:28Z2018-10-25T18:22:28ZAll builds failing due to artifact storage limit exceeded<div><p>Now I get: Maximum allowed artifact storage size of 70000 Mb will be exceeded.</p>
<p>This is a limit for all builds right? Not just for the artifacts on a signal build.</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-10-25T19:39:04Z2018-10-25T19:39:04ZAll builds failing due to artifact storage limit exceeded<div><p>Yes, the limit is for all builds. For now the retention policy is that artifacts older than 6 months are removed making space for more, however we plan to add a more flexible policy in the future.<br>
Storage size has been increased again.</p></div>Owen McDonnelltag:help.appveyor.com,2012-11-13:Comment/463516522018-10-25T20:28:19Z2018-10-25T20:28:19ZAll builds failing due to artifact storage limit exceeded<div><p>90000 Mb exceeded now. Is it possible for you to blow away all of our artifacts?</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-10-25T21:21:17Z2018-10-25T21:21:17ZAll builds failing due to artifact storage limit exceeded<div><p>Do you need <em>all</em> the artifacts your building? Can you possibly restrict artifact creation to the branch that really needs them?</p>
<p>It looks like you have a very active project that would quickly fill any artifact storage.</p></div>Owen McDonnelltag:help.appveyor.com,2012-11-13:Comment/463516522018-10-25T21:22:54Z2018-10-25T21:22:54ZAll builds failing due to artifact storage limit exceeded<div><p>Yup, we have work in progress to cut way down on artifact storage. Just trying to get things moving forward again while we work on that.</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-10-25T21:55:09Z2018-10-25T21:55:09ZAll builds failing due to artifact storage limit exceeded<div><p>It seems we're both caught in between improvements....</p>
<p>We don't have a way to intervene in our artifacts retention policy yet so your limit has been increased again.</p>
<p>Might I suggest adding this to your yaml,</p>
<pre>
<code>for:
-
branches:
only:
- <branches-that-need-artifact-creation>
artifacts:
- path: <path-to-artifact>
name: <artifact-name></code>
</pre>
<p>It may differ depending on how you create artifacts. But we're happy to assist you in implementation.</p></div>Owen McDonnelltag:help.appveyor.com,2012-11-13:Comment/463516522018-10-26T17:43:44Z2018-10-26T17:43:44ZAll builds failing due to artifact storage limit exceeded<div><p>Still failing due to limit. The problem is that we push our installer and pdb to S3 on each build. We continually release daily builds to our users on each change. Which I believe means we need to store them as an artifact first. We don't really need the artifact to be kept in appveyor. We just need it to get to S3. Hence asking whether it possible to just toss all old artifacts for our project since we don't need them.</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-10-29T18:10:35Z2018-10-29T18:10:35ZAll builds failing due to artifact storage limit exceeded<div><p>Still stuck here. Becoming a really bad problem since we are now on multiple weeks without a new Windows build being pushed up for our users.</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-10-31T17:40:13Z2018-10-31T17:40:13ZAll builds failing due to artifact storage limit exceeded<div><p>Still dead in the water. No appveyor builds for multiple weeks now. No response for help for 5 days. At a loss as to what to do other than start considering other competitor options for other open source and paid accounts.</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-10-31T18:04:36Z2018-10-31T18:04:36ZAll builds failing due to artifact storage limit exceeded<div><p>We extended your limit more to unblock you. Sorry for the trouble.</p></div>Ilya Finkelshteyntag:help.appveyor.com,2012-11-13:Comment/463516522018-10-31T18:42:19Z2018-10-31T18:42:19ZAll builds failing due to artifact storage limit exceeded<div><p>Builds are working now. Thank you so much. No we can get working on changing things so we don't use so much artifact storage.</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-11-02T18:21:51Z2018-11-02T18:21:51ZAll builds failing due to artifact storage limit exceeded<div><p>Worked for a couple days. Failing again: <a href="https://ci.appveyor.com/project/Dronecode/qgroundcontrol/builds/20009518">https://ci.appveyor.com/project/Dronecode/qgroundcontrol/builds/200...</a>.</p>
<p>Maximum allowed artifact storage size of 50000 Mb will be exceeded.</p>
<p>Sigh...</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-11-02T19:38:10Z2018-11-02T19:38:10ZAll builds failing due to artifact storage limit exceeded<div><p>It was reset back to default value. You probably subscribed to the different plan recently? This is the only reason I can think of for resetting the artifact size quota. I bumped it again ti unblock you.</p></div>Ilya Finkelshteyntag:help.appveyor.com,2012-11-13:Comment/463516522018-11-02T19:54:44Z2018-11-02T19:54:44ZAll builds failing due to artifact storage limit exceeded<div><p>Ok, thanks. Hmm, may have been someone else. I think we were signing for a paid open source account or something so we could get multiple builds. Heard that mentioned.</p>
<p>So for our builds we want all of our master builds to upload to S3 both the installer and the .pdb files. On S3 this gets overwritten with each new build such that there is only one master installer and .pdb file. But for appveyor this ends up blowing up into saved artifacts for every single build. The concept of making all devs builds available to users would seem to me to be a common thing to do. Is there no way to set this up such that it runs afoul of the artifact limit. We don't really need the artifacts stored in appveyor, we just need to get the files to S3.</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-11-02T20:07:05Z2018-11-02T20:07:05ZAll builds failing due to artifact storage limit exceeded<div><p>Yes, we understand. We plan to implement "transient" artifacts (which live until build ends) for that. I cannot say specific ETA, but it should take too long as we unhappy with this situation too. Meanwhile we are bumping storage for users like you if we do not see an abuse.</p>
<p>Other option available to you right now is to deploy with a script instead of S3 deployment provider. AWS CLI is installed <a href="https://www.appveyor.com/docs/windows-images-software/">on build worker image</a> and you can simple use files, for CLI without artifacts abstraction layer. Secrets can live in <a href="https://www.appveyor.com/docs/build-configuration/#secure-variables">secure variables</a>. In this case you do not need to package your files as artifacts at all. Admittedly this is less handy than 1st class S3 deployment, but you can consider it. We can help with samples if you need.</p></div>Ilya Finkelshteyntag:help.appveyor.com,2012-11-13:Comment/463516522018-11-02T20:09:56Z2018-11-02T20:09:56ZAll builds failing due to artifact storage limit exceeded<div><p>Yeah, that would work perfectly. I guess if transient artifacts don't come soon enough and we keep hitting the limit even with the work we are doing to reduce artifacts we can switch to the hard way of uploading manually. Thanks for the info.</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-11-02T20:20:58Z2018-11-02T20:20:58ZAll builds failing due to artifact storage limit exceeded<div><p>By "manually" you mean with custom script which runs as <code>deploy_script</code>?</p></div>Ilya Finkelshteyntag:help.appveyor.com,2012-11-13:Comment/463516522018-11-03T19:43:20Z2018-11-03T19:43:20ZAll builds failing due to artifact storage limit exceeded<div><p>Correct</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-11-05T16:26:34Z2018-11-05T16:26:34ZAll builds failing due to artifact storage limit exceeded<div><p>Created <a href="https://github.com/appveyor/ci/issues/2727">GitHub issue</a> to track this.</p></div>Ilya Finkelshteyntag:help.appveyor.com,2012-11-13:Comment/463516522018-11-29T02:37:58Z2018-11-29T02:37:58ZAll builds failing due to artifact storage limit exceeded<div><p>Wondering if you could help taking a look at this: <a href="https://github.com/mavlink/qgroundcontrol/pull/7039">https://github.com/mavlink/qgroundcontrol/pull/7039</a>. Trying to make artifact storage only for master and Stable_V3.4 branch on repro. Sorry, not much YAML experience.</p>
<pre>
<code>for:
-
branches:
only: master
artifacts:
-
path: QGroundControl-installer.exe
name: qgcinstaller
only: Stable_V3.4
artifacts:
-
path: symbols\**\*.*_
name: symbols</code>
</pre></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-11-29T02:39:02Z2018-11-29T02:39:02ZAll builds failing due to artifact storage limit exceeded<div><p>That's not quite right. I want the QGroundControl-install.exe to be stored for both master and Stable_V3.4. Whereas symbols should only be stored for Stable_V3.4.</p></div>DonLakeFlyertag:help.appveyor.com,2012-11-13:Comment/463516522018-11-29T03:39:55Z2018-11-29T03:39:55ZAll builds failing due to artifact storage limit exceeded<div><p>Try this:</p>
<pre>
<code>for:
-
branches:
only:
- master
artifacts:
- path: QGroundControl-installer.exe
name: qgcinstaller
-
branches:
only:
- Stable_V3.4
artifacts:
- path: QGroundControl-installer.exe
name: qgcinstaller
- path: symbols\**\*.*_
name: symbols</code>
</pre>
<p>And remove <a href="https://github.com/mavlink/qgroundcontrol/blob/f2a8ae053b95b8b7d19a2feee2bcad3816204cf5/.appveyor.yml#L100-L103">this</a>. You can move deportment setting to under the same <code>for.branches</code> construct, but for deployment it is simpler to use <a href="https://www.appveyor.com/docs/deployment/#conditional-deployment">conditional deployment</a>.</p>
<p>Also check <code>Validate YAML</code> tool under account Settings menu.</p></div>Ilya Finkelshteyntag:help.appveyor.com,2012-11-13:Comment/463516522018-11-29T04:56:43Z2018-11-29T04:56:43ZAll builds failing due to artifact storage limit exceeded<div><p>Awesome, thanks so much Ilya.</p></div>DonLakeFlyer