Conditional build confirguration

jffelsinger's Avatar

jffelsinger

03 Jun, 2014 06:19 PM

My team deploys to a couple different places depending on where we are with features in the development process, each place with a unique branch and configuration (production, staging, etc). This is pretty easy for us to setup with the deployment system, as we can specify to only deploy when the update came from a particular branch or is with a configuration:

configuration:
  - Release
  - Debug
deploy:
  - provider: Environment
    name: My Environment
    on:
      branch: dev
      configuration: Debug

However, even though we might only be using one configuration for a particular deployment, the build process happens for both. It would be nice if we could specify for a particular configuration to only build from a certain branch, much like we can do with deployments. Maybe something like the following:

configuration:
  - name: Release
    on:
      branch: master
  - name: Debug
    on:
      branch: dev

This would save some time, not having to build both configurations when only one is needed.

  1. Support Staff 1 Posted by Feodor Fitsner on 03 Jun, 2014 06:46 PM

    Feodor Fitsner's Avatar

    Why couldn't you have a specific appveyor.yml for every branch? For example, you can commit appveyor.yml in "dev" branch with configuration: debug only.

  2. 2 Posted by jffelsinger on 03 Jun, 2014 06:56 PM

    jffelsinger's Avatar

    One could do that, right off-hand I'm unsure how easy that would be to maintain though. If one was to make a pull request/merge from one branch to another wouldn't the differences in the two appveyor.yml's cause one to be overwritten? I don't know of a way to ignore a file only in relation to changes in other branches (I'm probably missing something obvious though).

  3. Support Staff 3 Posted by Feodor Fitsner on 03 Jun, 2014 07:03 PM

    Feodor Fitsner's Avatar

    OK, I see. This is what I expected to hear. Right, having build config in appveyor.yml has its pros and cons. The cons is that you can mess "master" config while merging dev branch into it.

    Build matrix and deployment is one thing. Do you need branch-specific settings for other parts of appveyor.yml?

    Btw, another idea would be having support for something like appveyor.<branch>.yml! How about that, can you see any potential issues with this approach? :)

  4. 4 Posted by jffelsinger on 03 Jun, 2014 07:18 PM

    jffelsinger's Avatar

    I don't think my team would need branch-specific settings elsewhere. Just the configurations would be dandy. As far as the branch-specific appveyor.yml files go, I don't see a problem with that one either; it'd be pretty cool.

    Though, with the separate files approach it might be more flexible to do something like: appveyor.<arbitrary_name>.yml, then in the file itself allow the user to specify which branch(s) it'd apply to somehow.

    (You do really great support, thanks.)

  5. Support Staff 5 Posted by Feodor Fitsner on 03 Jun, 2014 07:30 PM

    Feodor Fitsner's Avatar

    OK, thanks for your thoughts. I can envision only username as <arbitrary_name> value. Yes we have user information in every push, but what if two or more devs working on the same branch... looks like those nasty VS .user files in the root of projects :) From technical perspective doing appveyor.<branch>.yml is easier and, perhaps, it will look more natural.

    ...without support such service is pointless. I would say it's not kind of support, but it's more like solving problems, sharing and collecting the experience! I like thinking of AV as a "CI knowledge aggregating platform" which helps other developers to on board faster :)

  6. 6 Posted by jffelsinger on 03 Jun, 2014 08:31 PM

    jffelsinger's Avatar

    Mm, I might have worded that badly. Not really a per-user file per se, but just more decoupled from any one particular branch. I was sort of thinking of a use-case where one might want to use one configuration over a couple different branches.

    In which case they could do something like appveyor.testing.yml, and then supplying in file that it uses this config when building for say... anything that's not a master or release branch, such as dev or feature branches (Maybe just use the functionality for the branches: setting that's already there?). Then an appveyor.release.yml to handle release, master, etc branches.

    Though, I don't know if the use-case is realistic in any way. I just thought of it out of concern for clutter. If one wanted to use one config for more than one branch, appveyor.testing.yml and appveyor.release.yml might be DRYer than appveyor.<branch_1>.yml, appveyor.<branch_2>.yml, appveyor.<branch_3>.yml, etc. If no one actually needed to re-use the configs it wouldn't be worth the effort, and the easier to implement appbeyor.<branch>.yml would be best, like you said. :)

  7. Support Staff 7 Posted by Feodor Fitsner on 03 Jun, 2014 08:46 PM

    Feodor Fitsner's Avatar

    OK, makes sense. Then how would we choose desired config when making a push?

  8. 8 Posted by jffelsinger on 03 Jun, 2014 09:05 PM

    jffelsinger's Avatar

    Mmm, good point. If there were two configs where the branches were set,

    appveyor.config_1.yml

    branches:
      only:
        - dev
        - testing
    

    appveyor.config_2.yml

    branches:
      only:
        - master
        - release
    # or
      except:
        - dev
        - testing
    

    You could use that, to decide which to use. Push to dev and it would use the config that can build for dev, since it is already implemented and someone could only build on a config where the branch matches anyway.

    However, I'm not sure how you would choose if the branches that were setup overlap (like if both were set to build on a push to dev). You could default it to the regular appveyor.yml in such a case, or add a setting for priority, or just pick one of the overlapping yml files arbitrarily. (I do see what you mean about this way being harder to implement if you went with it)

  9. Support Staff 9 Posted by Feodor Fitsner on 03 Jun, 2014 11:53 PM

    Feodor Fitsner's Avatar

    ...or even better we could combine multiple configs in a single appveyor.yml like that:

    # first config
    -
      branches:
        only: master
    
    # second config
    -
      branches:
        only:
          - dev
          - qa
    

    So, if YAML root is a sequence then it has multiple configs, if it's mapping then a single.

    Selecting required config could follow these rules:
    1) if there is no config with matching branches/only AV will fall back to the config without branches/only section defined.
    2) if there is more than one config matching branch - error
    3) if there are no matching configs - build is not started

    And branch name could be regex, e.g. /dev.*/ or /feature-.*/.

    Have I missed something?

  10. 10 Posted by jffelsinger on 04 Jun, 2014 12:35 PM

    jffelsinger's Avatar

    I didn't even think about making the root a sequence, that's a great solution. I think it covers everything too. *thumbs up\*

  11. Support Staff 11 Posted by Feodor Fitsner on 04 Jun, 2014 04:32 PM

    Feodor Fitsner's Avatar

    Good, will implement this scenario then.

  12. 12 Posted by cyberkruz on 10 Jun, 2014 09:19 PM

    cyberkruz's Avatar

    This would be incredibly helpful for my scenario as well. I have a number of build configurations for different scenarios (production to master, staging to dev, etc), but I'm currently not using appveyor for deployments because it takes to long to have production built every time someone commits to a random pull request branch. There is some amazing thinking here. Will there be updates to the status of this feature in this thread?

    PS: Great support by the way.

  13. Support Staff 13 Posted by Feodor Fitsner on 11 Jun, 2014 12:04 AM

    Feodor Fitsner's Avatar

    Absolutely, we'll keep you guys updated! The deployment story is definitely the next thing we are going to improve.

    You can disable pull requests on GitHub webhook or what's the issue you're currently experiencing? Could you please elaborate?

  14. 14 Posted by cyberkruz on 11 Jun, 2014 12:55 AM

    cyberkruz's Avatar

    We do all our development through pull requests (we love the GitHub webhook). The hiccup is our solution has a number of build configurations to do config transforms (Debug, Release, Testing, Live). My end goal is to have AppVeyor build all of the pull requests under Debug (any branch that isn't dev or master) like it currently is. When the pull requests are merged into the dev branch they would automatically build and deploy to our Azure testing server using the Testing build configuration. When dev is tested and merged into master it would automatically deploy to our Azure production Staging instance using the Live build configuration.

    Currently to do this we either have to make multiple projects (like traditional ci servers) or have AppVeyor build every build configuration every time which takes a bit too long. Something like this would be killer:

    -
      branches:
        except:
          - master
          - dev
      configuration: Debug
    -
      branches:
        only: dev
      configuration: Testing
      build:
        publish_azure: true
      # stage deployment details
    -
      branches:
        only: master
      configuration: Live
      build:
        publish_azure: true
      # live deployment details
    

    or even like you guys were talking about before:

    configuration:
      name: Live
      on:
        branch: master
    

    I think either one of those scenarios would solve my use case. Does this make sense or am I not explaining correctly. Sorry about the wall of text.

  15. Support Staff 15 Posted by Feodor Fitsner on 11 Jun, 2014 03:39 AM

    Feodor Fitsner's Avatar

    Yes, that makes sense! Thanks for explaining your use case. It could be cool to include these examples into documentation when the feature is ready. I think we'll deliver it this week.

  16. Support Staff 16 Posted by Feodor Fitsner on 13 Jun, 2014 05:09 AM

    Feodor Fitsner's Avatar

    OK, the feature has been implemented and deployed. Give it a try and let me know what you think! ;)

  17. 17 Posted by cyberkruz on 13 Jun, 2014 11:12 PM

    cyberkruz's Avatar

    Hi there!

    I tried out the feature, and I am having a problem. It will not build dev or master for some reason. I tried to be as explicit as possible with the configuration. Let me know if I am configuring this correctly or not. Also, is there a way I can debug appveyor.yml configuration problems so I can give you better information? Here is my config:

    -
      shallow_clone: true
      version: 3.7.{build}
      branches:
        except:
          - master
          - dev
      configuration: Debug
      before_build:
        - cmd: cd src
        - cmd: nuget restore
        - cmd: cd ..
      build:
        project: src/MyProject.sln
        verbosity: minimal
    -
      shallow_clone: true
      version: 3.7.{build}
      branches:
        only: dev
      configuration: Staging
      before_build:
        - cmd: cd src
        - cmd: nuget restore
        - cmd: cd ..
      build:
        project: src/MyProject.sln
        verbosity: minimal
        #publish_azure: true
    -
      shallow_clone: true
      version: 3.7.{build}
      branches:
        only: master
      configuration: Live
      before_build:
        - cmd: cd src
        - cmd: nuget restore
        - cmd: cd ..
      build:
        project: src/MyProject.sln
        verbosity: minimal
        #publish_azure: true
    
  18. Support Staff 18 Posted by Feodor Fitsner on 13 Jun, 2014 11:29 PM

    Feodor Fitsner's Avatar

    Make sure you have the same appveyor.yml in all branches. When you kick of a build of specific branch AppVeyor is downloading appveyor.yml from that branch.

  19. 19 Posted by cyberkruz on 14 Jun, 2014 08:29 AM

    cyberkruz's Avatar

    Thank you so much. I think I am almost there. It looks like every time that I commit to a branch everything builds and works perfect. Whenever I commit to the dev branch directly it publishes correctly. Awesome work by the way. The only problem is that when I create a pull request from a branch to dev it is building the request under dev instead of the branch. This triggers a publish to the server every time someone creates a request (as well as when we accept the request). I think this is actually intended behavior it would be nice if the publish didn't happen on the request only on the merge. I think until then I am going to just have a third branch (staging) that I merge dev into which will publish to stage.

  20. 20 Posted by cyberkruz on 14 Jun, 2014 08:45 AM

    cyberkruz's Avatar

    For the record, here is my final yml file for your documentation:

    #
    # Current setup:
    # - All development is done in a branch off dev
    # - Pull requests are merged into dev
    # - When publishing to stage we just merge dev to stage
    #   and it auto publishes to the stage server
    # - When publishing to master we just merge stage
    #   to master and it auto publishes to production
    #
    
    -
      shallow_clone: true
      version: 3.7.{build}
      branches:
        except:
          - master
          - stage
      configuration: Debug
      before_build:
        - cmd: cd src
        - cmd: nuget restore
        - cmd: cd ..
      build:
        project: src/MyProject.sln
        verbosity: minimal
    -
      shallow_clone: true
      version: 3.7.{build}
      branches:
        only:
          - stage
      configuration: Staging
      before_build:
        - cmd: cd src
        - cmd: nuget restore
        - cmd: cd ..
      build:
        project: src/MyProject.sln
        verbosity: minimal
        publish_azure: true
      assembly_info:
        patch: true
        file: AssemblyInfo.*
        assembly_version: "{version}"
        assembly_file_version: "{version}"
        assembly_informational_version: "{version}"
      deploy:
        provider: AzureCS
        subscription_id: super-cool-subscription-id
        subscription_certificate: +LSKDJFLJNL
        storage_account_name: mystoragestage
        storage_access_key:
          secure: blehiBlhe==
        service: myservicestage
        slot: production
    -
      shallow_clone: true
      version: 3.7.{build}
      branches:
        only:
          - master
      configuration: Live
      before_build:
        - cmd: cd src
        - cmd: nuget restore
        - cmd: cd ..
      build:
        project: src/MyProject.sln
        verbosity: minimal
        publish_azure: true
      assembly_info:
        patch: true
        file: AssemblyInfo.*
        assembly_version: "{version}"
        assembly_file_version: "{version}"
        assembly_informational_version: "{version}"
      deploy:
        provider: AzureCS
        subscription_id: super-cool-subscription-id
        subscription_certificate: +LSKDJFLJNL
        storage_account_name: mystorage
        storage_access_key:
          secure: blehiBlhe==
        service: myservice
        slot: staging
    

    PS: Thank you so much for your continued work on this. Me and everyone at Rentler appreciates it. I plan on writing an article, and I will link to it in case you are interested.

  21. 21 Posted by charles on 03 Oct, 2014 01:57 PM

    charles's Avatar

    It would be nice of we could only override some settings. For example, in my project I have the branch staging and master. I would like to be able to specify some settings specific to each branch, something like it, I am not a YML expert, I don't even know if this is valid YML:

    branches
      only:
        - staging
          configuration: Staging
        - master
          configuration: Production
    

    All other settings are the same for both staging and master branches.

  22. Support Staff 22 Posted by Feodor Fitsner on 03 Oct, 2014 07:06 PM

    Feodor Fitsner's Avatar

    That's valid suggestion. Will give it a thought. Thanks!

  23. 23 Posted by calinoiu.alexan... on 07 Apr, 2016 07:28 PM

    calinoiu.alexandru's Avatar

    Hi,

    Was this ever implemented? Can you point me to the documentation please?

  24. Support Staff 24 Posted by Feodor Fitsner on 07 Apr, 2016 07:30 PM

    Feodor Fitsner's Avatar
  25. 25 Posted by calinoiu.alexan... on 11 Apr, 2016 01:14 PM

    calinoiu.alexandru's Avatar

    I can't seem to be able to figure this out, I need to run the build with a different env variable for each branch

    So far I have:

    -
      branches:
        only:
          - master
      configuration: Debug
      environment:
        APPSETTING_Data:ConnectionStrings:TestConnectionString:
          secure: eX93C8dC7qTMkH9/IYKwpGwhGxbYtQzuByKRgYv5Zx47Qt8ZiWm1yD83KZnjUSClX+7r2DwyzXHOdqnxukQjKgq5WbxiWzR+jjQXRdD5CijZ3K6Iikztgpum2suizIaj0I0fIbKhyMnAPN+/VRTMsvukwGLlm1cSPZt47nfk9/0P7B+YfCxE/CmpCBcQl63Sh5MnVUddWfA5A4r3kg0QTXh+6DXfETxfYHN8277cF7w=
    
    -
      branches:
        only: 
          - staging
      configuration: Release
      environment:
        APPSETTING_Data:ConnectionStrings:TestConnectionString:
          secure: eX93C8dC7qTMkH9/IYKwpGwhGxbYtQzuByKRgYv5Zx47Qt8ZiWm1yD83KZnjUSClAlBrKla4ZFCKBcz6UV1ruY4uW769e1s/ERUmi6e+VsF3gYdN8OrlEA7Dj+blj5JBiN5IcX33iTU8+FFrvDe9TVLmCtNfqL2AHMWgHhyswL5aF83psmsJV7TDbVRqBgBs30omIDJsHQHchXLVUW1oq4pJg0u4sYs0WFtrHaRK2uYqBEW3L6ZYUcWAtHOxkAho
    
    -
      configuration: Debug
      version: 1.0.{build}
      clone_folder: C:\projects\Test2
      init:
      - ps: Install-Product node $env:nodejs_version
      install:
      - cmd: >-
          npm cache clean -f
    
          npm install -g npm
    
          npm install typescript -g
    
    
          cd TestWebClient
    
          npm install
    
          cd ..
      cache: C:\Users\appveyor\.dnx\packages
      before_build:
      - cmd: >-
          set DNX_BUILD_VERSION=build-%APPVEYOR_BUILD_NUMBER%
    
    
          dnvm upgrade
    
    
          dnu restore
      build:
        project: JLL.Test.V2.sln
        publish_wap: true
        publish_azure: true
        verbosity: minimal
      after_build:
      - cmd: >-
          dnu publish ./TestWebApi --configuration Release -o WebApiOutput --runtime "active" --wwwroot "wwwroot" --wwwroot-out "wwwroot" --iis-command "web" --quiet
    
    
          TestWebClient\node_modules\.bin\gulp -b "C:\projects\Test2\TestWebClient" --color --gulpfile "C:\projects\Test2\TestWebClient\Gulpfile.js" prepare --env development
    
          dnu publish ./TestWebClient --configuration Release -o WebClientOutput --runtime "active" --wwwroot "wwwroot" --wwwroot-out "wwwroot" --iis-command "web" --quiet
      test_script:
      - cmd: >-
          dnx -p TestDataTests\project.json test
    
          dnx -p TestWebApiTests\project.json test
    
          dnx -p TestBusinessLogicTests\project.json test
    
          dnx -p TestWebApiIntegrationTests\project.json test
    
    
          TestWebClient\node_modules\.bin\gulp -b "C:\projects\Test2\TestWebClient" --color --gulpfile "C:\projects\Test2\TestWebClient\Gulpfile.js" unit
      artifacts:
      - path: /WebApiOutput
        name: WebApiOutput
      - path: /WebClientOutput
        name: WebClientOutput
      deploy:
      - provider: WebDeploy
        server: https://Test2-dev-api.scm.azurewebsites.net:443/msdeploy.axd?site=Test2-dev-api
        website: Test2-dev-api
        username: $Test2-dev-api
        password:
          secure: LdyiPgW0hHl4Ier4dJR0mk4G4KQJ5m9Rut+J+X9G8WbTxN3mSeePFdkUb9PYr93+5WfV81LbfPSoKq+v1Ihn0A==
        artifact: WebApiOutput
        aspnet_core: true
        remove_files: true
        aspnet_core_force_restart: true
        on:
          branch: master
      - provider: WebDeploy
        server: https://Test2-dev.scm.azurewebsites.net:443/msdeploy.axd?site=Test2-dev
        website: Test2-dev
        username: $Test2-dev
        password:
          secure: UPsN1nGVlmfNgaSY0xcTPdRI2Ko/MyNOSbVFfQG2iKjp4by8lCrUQmdj3eGoFK0c36zVYjdRNa57PDyCiFy4IQ==
        artifact: WebClientOutput
        aspnet_core: true
        remove_files: true
        aspnet_core_force_restart: true
        on:
          branch: master
      after_deploy:
      - cmd: >-
          cd TestWebApi
    
          dnx ef database update -p TestData
    
          dnx seed
      notifications:
      - provider: Slack
        incoming_webhook: https://hooks.slack.com/services/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
        channel: Test2
        on_build_success: true
        on_build_failure: true
        on_build_status_changed: false
    

    But none of my install and before filters fire.

  26. Support Staff 26 Posted by Feodor Fitsner on 11 Apr, 2016 01:30 PM

    Feodor Fitsner's Avatar

    Because you have to repeat the entire configuration for every branch.

  27. 27 Posted by calinoiu.alexan... on 11 Apr, 2016 01:32 PM

    calinoiu.alexandru's Avatar

    I see, it will be great not to that.

      

      

    Alexandru Calinoiu

    https://github.com/alexandru-calinoiu/

    https://ro.linkedin.com/in/alexandrucalinoiu

  28. Support Staff 28 Posted by Feodor Fitsner on 11 Apr, 2016 01:34 PM

    Feodor Fitsner's Avatar

    Yeah, there is an item for that: https://github.com/appveyor/ci/issues/325

  29. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:05 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

17 Jul, 2019 12:16 AM
17 Jul, 2019 12:15 AM
16 Jul, 2019 10:50 PM
16 Jul, 2019 07:47 PM
16 Jul, 2019 06:34 PM

 

16 Jul, 2019 05:17 PM
16 Jul, 2019 10:17 AM
16 Jul, 2019 07:46 AM
15 Jul, 2019 06:33 PM
15 Jul, 2019 06:09 PM
15 Jul, 2019 05:53 PM