max_jobs not working?

Danny Bernal's Avatar

Danny Bernal

13 Jun, 2018 07:50 PM

Hey guys, we recently upgraded to 2 jobs and are trying to use the max_jobs property in appveyor.yml to limit each project to 1 and it appears not to limit the jobs.
I've got multiple projects more or less setup as seen on https://www.appveyor.com/docs/appveyor-yml/
Is this currently not working?
Thanks!

  1. Support Staff 1 Posted by Ilya Finkelshte... on 14 Jun, 2018 05:21 PM

    Ilya Finkelshteyn's Avatar

    Hi Danny,

    Sorry for some delay with answer. It works. Can you please share appveyor.yml of project which does not respect max_jobs and links to jobs which were executed in parallel for that project?

    Ilya,

  2. 2 Posted by Danny Bernal on 15 Jun, 2018 04:29 PM

    Danny Bernal's Avatar

    Thanks for the quick reply.
    I have about a dozen active private repos with applications being developed. A big project among them has over 70 jobs to tackle per commit because we are building a c++ cross platform engine and applications targeting multiple OS/chips/configs.
    We often commit 2+ repos back to back since we might often make changes to the engine along with applications.
    With a single job, developers were often waiting for “the big one” to finish before their projects began building. This could take a significant amount of time and developers were canceling early and sometimes modifying the script to reduce number of jobs etc to get through it faster (since we cant cancel individual jobs to get to the few we might want to test on a feature build).
    I recently upgraded to the 2 job plan to try it out and was hoping to have 1 job working on each project when they go up so that developers aren’t waiting for the big one and also so developers don’t have to wait on each other.

    below is the smallest yaml file we have with 2 jobs for reference. I removed the token and trimmed a few lines in the build section for obvious reasons posting publicly here, but the idea is still here.
    I’ve also attached a screenshot showing 2 jobs running side by side.
    I’ve also set Max_Jobs in the settings tab of the backpanel to 1 and nothing seems to work.

    Would love to know what we’re doing wrong.

    Thanks
    ~Danny

    version: '{build}'
    max_jobs: 1
    
    skip_tags: true
    environment:
      build_script: ./buildscripts/build.sh
      publish_package: ./buildscripts/create_deb.sh
      publish_docker: ./buildscripts/create_docker.sh
      access_token:
        secure: <removed!>
      matrix:
        - arch: ArmV8
          build_flags: "-DCMAKE_TOOLCHAIN_FILE=../Electrolyte-Native/cmakeModules/toolchains/EToolchain-ArmV8.cmake"
        - arch: x64
          build_flags: ""
    
    image: ubuntu
    test: off
    
    install:
    
      # install dependencies
      - env
      - sudo ./buildscripts/install_dependencies_appveyor.sh $arch
      - echo "Finished install step"
    
    build_script:
    
      # do generate and compile
      - $build_script $build_flags
      - echo "Finished build step"
    
      # published based on the branch we're on.
      # Note: dont publish unless its one of these branches.
      - if [ "$APPVEYOR_REPO_BRANCH" = "production" ]; then
         $publish_package $arch VersionOne;
         $publish_docker $arch;
        fi
      - if [ "$APPVEYOR_REPO_BRANCH" = "testing" ]; then
         $publish_package $arch VersionOne-Testing;
         $publish_docker $arch;
        fi
    
      # Publish docker based on arch
    
      - echo "Finished publishing"
    
    on_success:
      - echo "Success!"
    

    [cid:[email blocked]]

  3. Support Staff 3 Posted by Ilya Finkelshte... on 18 Jun, 2018 11:16 PM

    Ilya Finkelshteyn's Avatar

    Apparently we regressed this behavior for build matrix... Thank you for that catch! We will deploy fix today or tomorrow. Please watch this: https://github.com/appveyor/ci/issues/2450

    Also I noticed that your developers sometimes need smaller subset of build jobs. Would you vote for https://github.com/appveyor/ci/issues/2299? We will probably propose simpler YAML syntax for this. I wonder if this functionality -- declare some jobs subset in commit message description -- would help your devs.

  4. 4 Posted by Danny Bernal on 18 Jun, 2018 11:28 PM

    Danny Bernal's Avatar

    Thanks for looking into this!

    I looked at the feature request link you posted. I can see how it would work for some people, but in our case it would just cause text pollution in our commit messages. I feel like this is a work around using a external source to lack of interface on the GUI. A far more useful feature for us would be the ability to cancel / restart individual jobs from the web GUI directly. Or better yet, to reprioritize the build queue via drag and drop. That would enable us to get quick feedback on whether some change is a bust or not. It happens often that if a particular job passes (somewhere in the middle of a long list of jobs), then we have high confidence in the rest of them passing.

    ~Danny

  5. Support Staff 5 Posted by Ilya Finkelshte... on 19 Jun, 2018 12:04 AM

    Ilya Finkelshteyn's Avatar

    I just posted simpler syntax for this feature if you are interested. This might pollute commit messages less.

    Restarting and cancelling though old issue is high priority now, should be done soon. Also filed issue to re-order jobs.

    Now it is right time to collect those ideas, as we plan to invest in to this area this summer.

  6. Ilya Finkelshteyn closed this discussion on 31 Aug, 2018 07:53 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac