tag:help.appveyor.com,2012-11-13:/discussions/kb/7-integrating-chutzpah-into-appveyorAppVeyor: Discussion 2014-12-31T11:24:21Ztag:help.appveyor.com,2012-11-13:Comment/344344812014-09-04T05:09:03Z2014-09-04T05:09:13ZIntegrating Chutzpah into AppVeyor<div><p>I've taken this a bit further in terms of investigation.
Chutzpah's console runner contains a flag to output results in
junit format. So doing this:</p>
<pre>
<code>PS C:\Chutzpah> .\chutzpah.console.exe C:\Users\John\Documents\GitHub\Proverb\AngularTypeScript\Proverb.Web.Tests.JavaScript /junit c:\junit-with-error.xml</code>
</pre>
<p>Results in a c:\junit-with-error.xml file that contains this:
(error deliberately included)</p>
<pre>
<code><?xml version="1.0" encoding="UTF-8" ?>
<testsuites>
<testsuite name="C:\Users\John\Documents\GitHub\Proverb\AngularTypeScript\Proverb.Web.Tests.JavaScript\app\sages\sages.js" tests="5" failures="1" time="0">
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sages -&gt; on creation -&gt;:controller should have a title of &apos;Sages&apos;" />
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sages -&gt; on creation -&gt;:controller should have no sages" />
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sages -&gt; on creation -&gt;:datacontext.sage.getById should be called" />
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sages -&gt; activateController -&gt;:should set sages to be the resolved promise values" />
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sages -&gt; activateController -&gt;:should log &apos;Activated Sages View&apos;">
<failure message="Expected spy log to have been called with [ &apos;Activated Sages View2&apos; ] but actual calls were [ &apos;Activated Sages View&apos; ].&#10;&#9; at stack (file:///C:/Chutzpah/TestFiles/jasmine/v2/jasmine.js:1293)&#10;&#9; at buildExpectationResult (file:///C:/Chutzpah/TestFiles/jasmine/v2/jasmine.js:1270)&#10;&#9; at file:///C:/Chutzpah/TestFiles/jasmine/v2/jasmine.js:484&#10;&#9; at file:///C:/Chutzpah/TestFiles/jasmine/v2/jasmine.js:260&#10;&#9; at addExpectationResult (file:///C:/Chutzpah/TestFiles/jasmine/v2/jasmine.js:442)"></failure>
</testcase>
</testsuite>
<testsuite name="C:\Users\John\Documents\GitHub\Proverb\AngularTypeScript\Proverb.Web.Tests.JavaScript\app\sages\sageDetail.js" tests="6" failures="0" time="0">
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sageDetail -&gt; on creation -&gt;:controller should have a title of &apos;Sage Details&apos;" />
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sageDetail -&gt; on creation -&gt;:controller should have no sage" />
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sageDetail -&gt; on creation -&gt;:datacontext.sage.getById should be called" />
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sageDetail -&gt; activateController -&gt;:should set sages to be the resolved promise values" />
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sageDetail -&gt; activateController -&gt;:should log &apos;Activated Sage Details View&apos; and set title with name" />
<testcase name="Proverb.Web -&gt; app-&gt; controllers -&gt; sageDetail -&gt; gotoEdit -&gt;:should set $location.path to edit URL" />
</testsuite>
</testsuites></code>
</pre>
<p>I understand that JUnit and XUnit are very similar so I'm
wondering if I could <a href=
"http://www.appveyor.com/docs/running-tests">upload JUnit output
(as above) to AppVeyor using the XUnit flag</a>?</p></div>johnny_reillytag:help.appveyor.com,2012-11-13:Comment/344344812014-09-04T05:34:06Z2014-09-04T05:34:06ZIntegrating Chutzpah into AppVeyor<div><p>I'm afraid it's different from both xUnit and NUnit, but we
could easily add its support in the meantime!</p></div>Feodor Fitsnertag:help.appveyor.com,2012-11-13:Comment/344344812014-09-04T05:47:26Z2014-09-04T05:47:26ZIntegrating Chutzpah into AppVeyor<div><p>That would be fantastic Feodor!</p></div>johnny_reillytag:help.appveyor.com,2012-11-13:Comment/344344812014-09-04T17:23:16Z2014-09-04T17:23:25ZIntegrating Chutzpah into AppVeyor<div><p>@Feodor I would also very much appreciate if you found the time
to implement the Chutzpah test runner.</p></div>Jacob T. Nielsentag:help.appveyor.com,2012-11-13:Comment/344344812014-09-04T17:39:49Z2014-09-04T17:39:49ZIntegrating Chutzpah into AppVeyor<div><p>You mean importing JUnit test results?</p></div>Feodor Fitsnertag:help.appveyor.com,2012-11-13:Comment/344344812014-09-04T17:49:14Z2014-09-04T17:49:16ZIntegrating Chutzpah into AppVeyor<div><p>I might have misunderstood were your discussion had gone. I
personally have Jasmine unit tests for testing my JavaScript that I
run with the Chutzpah test runner for Visual Studio.</p>
<p>I would like Appveyor to be able to run these as well - is that
possible?</p>
<p>Kind RegardJacob</p></div>Jacob T. Nielsentag:help.appveyor.com,2012-11-13:Comment/344344812014-09-04T18:36:05Z2014-09-04T18:36:05ZIntegrating Chutzpah into AppVeyor<div><p>Will take a look.</p></div>Feodor Fitsnertag:help.appveyor.com,2012-11-13:Comment/344344812014-09-04T19:05:33Z2014-09-04T19:55:49ZIntegrating Chutzpah into AppVeyor<div><p>For what it's worth if AppVeyor supports the JUnit format for
uploading XML test results then I think it should become fairly
straightforward. Not tested but here's how I think it could
work:</p>
<ol>
<li>
<p>Either add Chutzpah to the repo <em>or</em> add the <a href=
"http://www.nuget.org/packages/chutzpah">Chutzpah nuget package</a>
- the second course of action is probably better.</p>
</li>
<li>
<p>Add an "after build step" that looks something like this:</p>
</li>
</ol>
<pre>
<code># Run tests using Chutzpah and export results as JUnit format to chutzpah-results.xml
$ChutzpahCmd = "$($env:APPVEYOR_BUILD_FOLDER)\packages\Chutzpah.3.2.3\tools\chutzpah.console.exe $($env:APPVEYOR_BUILD_FOLDER)\AngularTypeScript\Proverb.Web.Tests.JavaScript /junit .\chutzpah-results.xml"
Write-Host $ChutzpahCmd
Invoke-Expression $ChutzpahCmd
# upload results to AppVeyor
$wc = New-Object 'System.Net.WebClient'
$wc.UploadFile("https://ci.appveyor.com/api/testresults/junit/$($env:APPVEYOR_JOB_ID)", (Resolve-Path .\chutzpah-results.xml))</code>
</pre>
<p>Obviously I'd love AppVeyor to make Chutzpah even easier to plug
in but this looks like a potential approach that should work.</p>
<p>@Feodor - does it sound sensible to you?</p></div>johnny_reillytag:help.appveyor.com,2012-11-13:Comment/344344812014-09-06T07:35:01Z2014-09-06T07:35:01ZIntegrating Chutzpah into AppVeyor<div><p>Coming on in leaps and bounds! I added the Chutzpah NuGet
package to Proverb (my project) and added the following PowerShell
"after build step":</p>
<pre>
<code># Run tests using Chutzpah and export results as JUnit format to chutzpah-results.xml
$ChutzpahCmd = "$($env:APPVEYOR_BUILD_FOLDER)\AngularTypeScript\packages\Chutzpah.3.2.3\tools\chutzpah.console.exe $($env:APPVEYOR_BUILD_FOLDER)\AngularTypeScript\Proverb.Web.Tests.JavaScript /junit .\chutzpah-results.xml"
Write-Host $ChutzpahCmd
Invoke-Expression $ChutzpahCmd
Write-Host "Chutzpah results:"
Get-Content .\chutzpah-results.xml
# upload results to AppVeyor
$wc = New-Object 'System.Net.WebClient'
#$wc.UploadFile("https://ci.appveyor.com/api/testresults/junit/$($env:APPVEYOR_JOB_ID)", (Resolve-Path .\chutzpah-results.xml))</code>
</pre>
<p>Jasmine tests are now running using Chutzpah and results are
visible in the build window. Now just need that JUnit support so
the results can be hooked into the build!</p>
<p>You can see an example of a successful build as describe above
here: <a href=
"https://ci.appveyor.com/project/JohnReilly/proverb/build/1.0.11">https://ci.appveyor.com/project/JohnReilly/proverb/build/1.0.11</a></p></div>johnny_reillytag:help.appveyor.com,2012-11-13:Comment/344344812014-09-06T12:56:54Z2014-09-06T12:56:54ZIntegrating Chutzpah into AppVeyor<div><p>By Jove I think I've got it!</p>
<p>I've now got a powershell "Before tests script" which runs the
tests using Chutzpah, writes to output to a JUnit format and then
plays the results back into AppVeyor result by result:</p>
<pre>
<code># Run tests using Chutzpah and export results as JUnit format to chutzpah-results.xml
$ChutzpahCmd = "$($env:APPVEYOR_BUILD_FOLDER)\AngularTypeScript\packages\Chutzpah.3.2.3\tools\chutzpah.console.exe $($env:APPVEYOR_BUILD_FOLDER)\AngularTypeScript\Proverb.Web.Tests.JavaScript /junit .\chutzpah-results.xml"
Write-Host $ChutzpahCmd
Invoke-Expression $ChutzpahCmd
# Upload results to AppVeyor one by one
$testsuites = [xml](get-content .\chutzpah-results.xml)
foreach ($testsuite in $testsuites.testsuites.testsuite) {
write-host " $($testsuite.name)"
foreach ($testcase in $testsuite.testcase){
$failed = $testcase.failure
if ($failed) {
write-host "Failed $($testcase.name) $($testcase.failure.message)"
Add-AppveyorTest $testcase.name -Outcome Failed -FileName $testsuite.name -ErrorMessage $testcase.failure.message
}
else {
write-host "Passed $($testcase.name)"
Add-AppveyorTest $testcase.name -Outcome Passed -FileName $testsuite.name
}
}
}</code>
</pre>
<p>You can see a build with a deliberately broken test <a href=
"https://ci.appveyor.com/project/JohnReilly/proverb/build/1.0.15">here</a>
demonstrating that both passed and failed tests are collected. The
only tricky thing appears to be getting a failed test to fail the
build - I can't see anything in the API docs about this.</p>
<p>@Feodor - can you be of any assistance?</p>
<p>By the way, I'll try and blog about this when I get a moment so
the knowledge doesn't just stay with me.</p></div>johnny_reillytag:help.appveyor.com,2012-11-13:Comment/344344812014-09-06T13:14:30Z2014-09-06T13:14:30ZIntegrating Chutzpah into AppVeyor<div><p>Cracked it. Now failing the build when any tests fail using the
PS below. See this build for an example of failure: <a href=
"https://ci.appveyor.com/project/JohnReilly/proverb/build/1.0.17">https://ci.appveyor.com/project/JohnReilly/proverb/build/1.0.17</a></p>
<pre>
<code># Run tests using Chutzpah and export results as JUnit format to chutzpah-results.xml
$ChutzpahCmd = "$($env:APPVEYOR_BUILD_FOLDER)\AngularTypeScript\packages\Chutzpah.3.2.3\tools\chutzpah.console.exe $($env:APPVEYOR_BUILD_FOLDER)\AngularTypeScript\Proverb.Web.Tests.JavaScript /junit .\chutzpah-results.xml"
Write-Host $ChutzpahCmd
Invoke-Expression $ChutzpahCmd
# Upload results to AppVeyor one by one
$testsuites = [xml](get-content .\chutzpah-results.xml)
$anyFailures = $FALSE
foreach ($testsuite in $testsuites.testsuites.testsuite) {
write-host " $($testsuite.name)"
foreach ($testcase in $testsuite.testcase){
$failed = $testcase.failure
if ($failed) {
write-host "Failed $($testcase.name) $($testcase.failure.message)"
Add-AppveyorTest $testcase.name -Outcome Failed -FileName $testsuite.name -ErrorMessage $testcase.failure.message
Add-AppveyorMessage "$($testcase.name) failed" -Category Error
$anyFailures = $TRUE
}
else {
write-host "Passed $($testcase.name)"
Add-AppveyorTest $testcase.name -Outcome Passed -FileName $testsuite.name
}
}
}
if ($anyFailures -eq $TRUE){
write-host "Failing build as there are broken tests"
$host.SetShouldExit(1)
}</code>
</pre></div>johnny_reillytag:help.appveyor.com,2012-11-13:Comment/344344812014-09-07T04:48:35Z2014-09-07T04:48:35ZIntegrating Chutzpah into AppVeyor<div><p>I've also blogged about this: <a href=
"http://icanmakethiswork.blogspot.com/2014/09/running-javascript-unit-tests-in-appveyor.html">
http://icanmakethiswork.blogspot.com/2014/09/running-javascript-uni...</a></p></div>johnny_reillytag:help.appveyor.com,2012-11-13:Comment/344344812014-09-07T16:26:35Z2014-09-07T16:26:35ZIntegrating Chutzpah into AppVeyor<div><p>That's great, thank you for your support!</p></div>Feodor Fitsnertag:help.appveyor.com,2012-11-13:Comment/344344812014-09-08T12:51:26Z2014-09-08T12:51:26ZIntegrating Chutzpah into AppVeyor<div><p>My pleasure Feodor!</p></div>johnny_reillytag:help.appveyor.com,2012-11-13:Comment/344344812014-09-09T10:29:13Z2014-09-09T10:29:13ZIntegrating Chutzpah into AppVeyor<div><p>Oh and if you wanted to use the blog content to add to your
<a href=
"http://www.appveyor.com/docs/running-tests">http://www.appveyor.com/docs/running-tests</a>
docs you'd be entirely welcome.</p></div>johnny_reillytag:help.appveyor.com,2012-11-13:Comment/344344812014-09-09T13:43:51Z2014-09-09T13:43:51ZIntegrating Chutzpah into AppVeyor<div><p>Thank you!</p>
<p>-Feodor</p></div>Feodor Fitsner