tag:help.appveyor.com,2012-11-13:/discussions/questions/3115-splitting-large-project-usage-of-parallel-testingAppVeyor: Discussion 2018-08-25T02:14:39Ztag:help.appveyor.com,2012-11-13:Comment/418665642017-02-02T01:49:34Z2017-02-02T01:49:34ZSplitting large project - Usage of parallel testing<div><p>Hi Michael,</p>
<p>Your understanding is right, parallel tests are separate build
jobs. As mentioned <a href="https://www.appveyor.com/docs/parallel-testing/">here</a>, "You
can split tests into jobs on the Free plan, but jobs will run in
series as the Free plan allows 1 concurrent job per account."</p>
<p>You can use <a href="https://github.com/nunit/docs/wiki/Parallel-Test-Execution">NUnit
parallel testing</a>, with your own test script. Note that build
VMs have only 2 CPU cores, so I am not sure if more than 2 parallel
test process will make any improvement though. Also parallel
testing is usually error prone as some tests may have common
dependencies in the environment and if they say modify the same IIS
website in parallel, nobody can predict anything.</p>
<p>Another approach is to run single build which will create and
upload artifacts, and then start (with API) another build which
will download artifacts and run parallel tests though this might
require you to have more parallel buds. If you want to implement
this, we can help with scripting.</p>
<p>Yet another approach is our new <a href="https://www.appveyor.com/docs/enterprise/running-builds-on-azure/">
private build clouds solution</a>, which is still expensive as you
have to pay for Azure subscription. We also support GCE, AWS,
Hyper-V and Docker clouds, this is just not documented yet. With
this solution you can instantiate VM with bigger number of
cores.</p></div>Ilya Finkelshteyntag:help.appveyor.com,2012-11-13:Comment/418665642017-02-06T16:28:23Z2017-02-06T16:28:23ZSplitting large project - Usage of parallel testing<div><p>Hi Ilya,</p>
<p>Thanks for the explanation. Private build seems like a great
solution that we will investigate in the future.</p>
<p>Your approach with starting another build with the API sounds
very promising. Do you have any example of how this could be
done?</p>
<p>For the moment, we will stick with long running jobs to be run
nightly</p>
<p>Bests,<br>
Michael</p></div>michaeltag:help.appveyor.com,2012-11-13:Comment/418665642017-02-06T19:31:43Z2017-02-06T19:31:43ZSplitting large project - Usage of parallel testing<div><p>Hi Michael,</p>
<ul>
<li>
<p><a href="https://gist.github.com/IlyaFinkelshteyn/89d19593318276e2538040787bc4fce7">
How to start build job</a>. Note that you can pass environment
variables, which you can use to tell build job what group of tests
to run. This script can be run as part of "main" job which created
all artifacts.</p>
</li>
<li>
<p><a href="https://www.appveyor.com/docs/api/samples/download-artifacts-ps/">download
build artifacts</a> (or <a href="https://www.appveyor.com/docs/api/samples/download-artifacts-advanced-ps/">
advanced version</a>). Those scripts you can run on "child" jobs to
get artifacts and start testing them</p>
</li>
<li>
<p>Another problem you may optionally need to solve is to have
single source of failure. For that you can monitor "child" jobs
from "main" and fail main job if at least one "child" failed. For
that you may need to</p>
<ul>
<li>remember build version when you start it:
<pre>
<code>$newBuild = Invoke-RestMethod -Uri 'https://ci.appveyor.com/api/builds' -Headers $headers -Body $body -Method POST
$version = $newBuild.version</code>
</pre></li>
<li>Test build status for every version (as you will start several
builds) and throw if at least one failed:
<pre>
<code>(Invoke-RestMethod -Uri "https://ci.appveyor.com/api/projects/<your_account>/<your_project_slug>/build/$version" -Headers $headers -Method GET).build.status</code>
</pre>
You can examine this status in the loop on finish of "main"
job.</li>
</ul>
</li>
</ul>
<p>Please let us know if something looks confusing, and feel free
to send your scripts to review.</p>
<p>Ilya.</p></div>Ilya Finkelshteyn