Home → Problems →

Allow failures behavior is problematic

• Edit

dane

20 Jun, 2014 04:33 PM

I'm having a lot of trouble getting the allow_failures behavior they way I'd like. As someone coming from travis I'm obviously biased by that experience so let me describe.

I use appveyor to build binary packages with node-pre-gyp. It is common for a build for one node.js version to fail while others might succeed. So I don't want a single build, say against node v0.11.x, to stop all the other jobs.

One request is that if one build fails and stops other jobs (independent of the allow_failures setting) it would be great, through the appveyor UI, to be able to click on a job and (re)start it. Currently it appears you can only do this (hitting the "new build" button) for jobs that once ran and not for jobs that never ran because other ones failed. If I try to click "new build" for a job that was blocked by another job failing I get the error of Project not found or access denied..

The above is a common desire because I'm having a lot of trouble figuring out the right syntax for allow_failures so that all jobs run no matter whether others fail. On recent example is node-srs. With this appveyor config (https://github.com/mapbox/node-srs/blob/f39498c309499ca904de088934c...) I saw the build do this: https://ci.appveyor.com/project/springmeyer/node-srs/build/1.0.91. That is not what I expected because 1) the overall color of the build is green (seemingly indicating that either all jobs succeeded or all were allowed to fail), and 2) only one job ran (despite the allow_failures setting) and I want all jobs to run.

I'm currently testing this modification: https://github.com/mapbox/node-srs/commit/ac3b5a48a8b6dffc1cb78a8f2...

So, to recap: I think the allow_failures syntax either is not working or is too hard to use. It should be much easier to achieve the travis behavior of all jobs running. I would really prefer that all jobs running be the default, but until then I'd love to hear ideas of what I'm doing wrong or what I'm missing.

Thanks1

1. 1 Posted by dane on 20 Jun, 2014 04:44 PM

   

   Latest build with:

   matrix:
     allow_failures:
       - platform: x86
         nodejs_version: 0.10
       - platform: x86
         nodejs_version: 0.11
       - platform: x64
         nodejs_version: 0.10
       - platform: x64
         nodejs_version: 0.11
   

   Does not quite work yet. It appears that a failing in one node v0.11.x version did not prevent node v0.10.x from building (yay) but did prevent the other node v0.11.x build from running. This is a problem because all node v0.11.x build are know to fail, but not till the very end due to this aws-sdk bug and before that failure I'm posting build artifacts that are important to work around the aws-sdk bug.

   Here is the build: https://ci.appveyor.com/project/springmeyer/node-srs/build/1.0.92

   • Edit

2. Support Staff 2 Posted by Feodor Fitsner on 23 Jun, 2014 05:56 PM

   

   So, if I correctly understand the main issue right now is default "fail fast" strategy in build matrix where first failed job immediately fails the entire build with running/queued jobs. You need "run all" behavior where all jobs run no matter what and the build is considered failed if at least on job has failed. Correct?

   • Edit

3. 3 Posted by dane on 24 Jun, 2014 03:24 AM

   

   Yes, that's exactly what I'm looking for.

   • Edit

4. Support Staff 4 Posted by Feodor Fitsner on 24 Jun, 2014 03:34 AM

   

   I see. Looks like it's better moving in this direction. I will add a new item - don't think it's hard to implement.

   • Edit

5. 5 Posted by dane on 15 Jul, 2014 07:18 PM

   

   Hello Feodor,

   Have you had a chance to fix this? It is still feeling quite problematic. Most recently I've seen this type of error (http://help.appveyor.com/discussions/problems/539-cloning-gyp-fails) randomly and that cancels all the other builds all of a sudden. Ideally I would only need to restart the one build that fails randomly due to a network problem.

   Thanks, Dane

   • Edit

6. Support Staff 6 Posted by Feodor Fitsner on 15 Jul, 2014 08:39 PM

   

   Will try delivering that this week, sorry for the delay.

   • Edit

7. 7 Posted by dane on 15 Jul, 2014 08:55 PM

   

   Okay, thank you for the update!

   Dane

   • Edit

8. Support Staff 8 Posted by Feodor Fitsner on 17 Jul, 2014 09:02 AM

   

   Hi Dane,

   A new update with fast_finish feature (sorry for borrowing the name from Travis) has been just deployed. Now, unlike it was done before, all build jobs run to the end no matter what. To make a build fail immediately once one of the jobs fails add to appveyor.yml:

   matrix:
     fast_finish: true
   

   Sample output:

   • Default mode (all jobs run, build is red): https://ci.appveyor.com/project/appvyr/grunt/build/1
   • Fast finish (last job fails build, build is red): https://ci.appveyor.com/project/appvyr/grunt/build/2
   • Allow failures for the last job (build is green): https://ci.appveyor.com/project/appvyr/grunt/build/3
   • Edit

9. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 01:46 AM.