Options for splitting up build & custom tests

markfinal's Avatar


12 Aug, 2018 09:30 AM


I have a FOSS github project on AppVeyor, that has grown recently and now exceeds the standard 60 minute job time.

It consists of
- A VisualStudio build of a C# solution, with NUnit3 unittests
- Currently, three invocations of a custom Python-based test harness, for integration testing.
The github project contains all of the code and scripts for this.

Currently I just invoke Python in the test_script phase sequentially. It is these Python tests that take a long time and exceed the time out.

However, those tests only depend on the VisualStudio build output, and not on each other, and independently, they run for less than the default timeout. I realise with a FOSS project parallel jobs are not available.

I've read that you can split out larger jobs, by spawning them in script from the main job, and downloading artifacts from the main job. This is a good pattern for what I'm after, assuming each child job has it's own 60 minute allocation, but I'm failing to see how to do the spawning.

What I'd ideally like:
1) commit in github launches main job to build assemblies, and save artifacts from the build (this is working)
2) spawn additional jobs for each of the Python test runs, downloading the artifacts from the main job
3) if child jobs fail, then the main job fails

In the future, I'd also like to take the artifacts of 1) and spawn jobs from different github repositories, but that's a task for another day.

Are there some working examples of the spawn of child jobs?


  1. 1 Posted by Ilya Finkelshte... on 13 Aug, 2018 05:00 PM

    Ilya Finkelshteyn's Avatar

    Hi Mark,

    For simplest solution we increased your build timeout to 90 minutes :)

    If that appears to be not enough, please check this YAML sample. Essentially you can create build matrix based on some environment variable and depending on that variable value run different tests. And use script form the sample to share artifact between build jobs. Again, you can use that variables to condition if build should happen or only test should happen.
    If you decide to go this direction and hit some issues, please feel free to post your YAML and describe issues you hit.

  2. 2 Posted by markfinal on 14 Aug, 2018 05:29 PM

    markfinal's Avatar

    Hi Ilya,

    Many thanks for this.

    I do still want to look into splitting the build into several jobs. I'll do that another day though. I will post another thread if I get suck.

    From the examples you sent, instead of having to manually spawn other jobs, all jobs in a build get spawned together (although run in serial for free tiers), and you can ask jobs to wait on others to finish, in order to define an order.


  3. markfinal closed this discussion on 14 Aug, 2018 05:29 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts


? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac