Making tests a prerequisite for others; Caching in between these tests

noel.kuntze's Avatar

noel.kuntze

15 Mar, 2020 05:39 PM

Hello,

Is it possible to make groups of tests, make some depend on others and execute them in order? I'd like to build some dependencies first, cache them and make that cache available to the jobs that depend on them. That would significantly decrease build time for me because large libraries would only need to be built one time instead of N times. I guess that would also require some change to the cache semantics.

Kind regards

Noel

  1. Support Staff 1 Posted by Feodor Fitsner on 17 Mar, 2020 07:26 PM

    Feodor Fitsner's Avatar

    Hi Noel,

    Do you mean dependencies between build jobs within a single build?

  2. 2 Posted by noel.kuntze on 18 Mar, 2020 12:14 PM

    noel.kuntze's Avatar

    Hello Feodor,

    Yes, exactly.

  3. Support Staff 3 Posted by Feodor Fitsner on 18 Mar, 2020 08:01 PM

    Feodor Fitsner's Avatar

    Yes, it's possible. Look at this yaml template defining 3 jobs running in a sequence:

    environment:    
      matrix:
    
    # Hyper-V
    # =======
    
      - job_name: Part 1
    
      - job_name: Delay between parts
        job_depends_on: Part 1
    
      - job_name: Part 2
        job_depends_on: Delay between parts
    
    matrix:
      fast_finish: true
    
    init:
      - appveyor version
      - echo Some common logic here...
    
    test: off
    
    for:
    
    # Part 1
      -
        matrix:
          only:
            - job_name: Part 1
    
        build_script:
        - echo Part 1
    
      -
        matrix:
          only:
            - job_name: Delay between parts
    
        build_script:
        - ps: Start-Sleep -s 60
    
    # Part 2
      -
        matrix:
          only:
            - job_name: Part 2
    
        build_script:
        - echo Part 2
    
  4. Support Staff 4 Posted by Feodor Fitsner on 18 Mar, 2020 08:03 PM

    Feodor Fitsner's Avatar

    You can also have groups of jobs and then depend on a group, for example:

    environment:    
      matrix:
    
    # Hyper-V
    # =======
    
      - job_name: Part 1 - A
        job_group: Part 1
    
      - job_name: Part 1 - B
        job_group: Part 1
    
      - job_name: Delay between parts
        job_depends_on: Part 1
    
      - job_name: Part 2
        job_depends_on: Delay between parts
    
    matrix:
      fast_finish: true
    
    ...
    

    Now jobs Part 1 - A and Part 1 - B will be running in parallel and Delay between parts will run when all jobs within Part 1 group are finished.

    We are going to document this functionality this week.

  5. 5 Posted by noel.kuntze on 19 Mar, 2020 10:09 AM

    noel.kuntze's Avatar

    Hello Feodor,

     

    Thank you for your insight! I will test this out later. :)

     

    Kind regards

     

    Noel

  6. 6 Posted by noel.kuntze on 27 Mar, 2020 09:48 AM

    noel.kuntze's Avatar

    Hello Feodor,

    How are the cache semantics of appveyor exactly? Is the cache updated in between jobs already? So can i pass data from job 1 to job 2 via a cache, if they run consecutively?

    Kind regards

    Noel

  7. Support Staff 7 Posted by Feodor Fitsner on 01 Apr, 2020 05:31 PM

    Feodor Fitsner's Avatar

    Hi Noel,

    The cache is job-specific. However, you can pack your dependencies (intermediate results), upload (deploy) to Azure storage, S3 or FTP account and then download/unpack in the next jobs. This is what, actually, AppVeyor cache does.

  8. 8 Posted by noel.kuntze on 02 Apr, 2020 02:16 PM

    noel.kuntze's Avatar

    Hi Feodor,

    Thank you for your response.

    Kind regards

    Noel

  9. noel.kuntze closed this discussion on 02 Apr, 2020 02:16 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

07 Jul, 2020 06:21 PM
07 Jul, 2020 03:15 PM
05 Jul, 2020 02:33 AM
03 Jul, 2020 07:29 PM
03 Jul, 2020 03:53 AM

 

02 Jul, 2020 09:09 PM
02 Jul, 2020 03:24 PM
01 Jul, 2020 01:12 PM
30 Jun, 2020 04:26 PM
25 Jun, 2020 05:54 PM
24 Jun, 2020 08:11 AM