Is it possible to programmatically (e.g., via PS or CMD) fail a build?

sbohlen's Avatar

sbohlen

10 Jan, 2015 04:55 PM

I have a project (https://ci.appveyor.com/project/sbohlen/ndbunit) that is compiled with a series of 32-bit dependencies. This means that the AV "automatically discover tests" cannot be used b/c that invokes nunit-console.exe (which runs in native 64-bit mode so cannot load my necessary 32-bit dependencies).

My work-around was to disable the AV "automatically discover tests" and to have my own build script invoke my tests with its own copy of nunit-console-x86.exe. I then use a post-build PS script to transmit the TestResult.xml file to the AV REST API so that the test report appears in my dashboard properly (https://ci.appveyor.com/project/sbohlen/ndbunit/build/tests).

In order that the post-build transmission of the test results happens even when there are failed tests, have to invoke nunit-console-x86.exe with the equivalent of CONTINUEONERROR, else the non-zero ERRORCODE returned by the test-run will cancel the build and I'll never see the results in the AV dashboard. Unfortunately, this also now means that AV considers the build PASSING even when it has FAILED tests :(

Ideally, I'd like a way to achieve the following:

- invoke tests using my own test-runner [done, using my own build-script]
- have those results uploaded to the AV REST API even if there are test-failures [done, using the AV REST API after 'swallowing' the ERRORCODE returned from the test-run]
- have a test failure in those results FAIL the build [no way I can see to achieve this one]

Some thoughts:

- since you're applying CSS styling (e.g., color-coding) to NUnit test results in the dashboard, presumably you already have code that parses test results XML from NUnit (and others) to determine each test status; could you perhaps simply add a project option for 'fail build on any failing tests' that could just check the results and do so if it finds any failing tests?

- alternately, is there any way that *I* can fail the build from e.g., the post-build scripting facility provided by AV? In this case, I could seemingly inspect the results of my own test results and then do e.g., "Set-AppVeyor-BuildStatus FAIL" via powershell or similar...

Or, is it also possible that there's a better/more effective way to achieve what I'm trying to do here ...?

Thanks in advance for any suggestions,

-Steve B.

  1. Support Staff 1 Posted by Feodor Fitsner on 10 Jan, 2015 09:20 PM

    Feodor Fitsner's Avatar

    Hi Steve,

    You have two options here:

    1) Upload XML test results in "On build finish script" which is called for both successful and failed builds. And just allow your test runners to fail with exit code.

    2) Use $host.SetShouldExit(yourExitCode) in PowerShell script which makes build to fail on script finishing.

    Let me know if you have any questions.

  2. 2 Posted by sbohlen on 10 Jan, 2015 09:31 PM

    sbohlen's Avatar

    Got it; thx for the speedy reply (as usual!).

    I wasn't aware that the post-build event still fired even with a failed build. In that case, since my REST API call to post the test results XML file is already in the post-build event, I'll just set my test-runner task to no longer ignore non-zero exit codes and I think I should get just what I was seeking.

    Cheers,

    -Steve B.

  3. Support Staff 3 Posted by Feodor Fitsner on 10 Jan, 2015 09:32 PM

    Feodor Fitsner's Avatar

    Exactly!

  4. 4 Posted by gep13 on 12 Jan, 2015 07:44 AM

    gep13's Avatar

    Hello,

    When I was setting up DupFinder and InspectCode testing on ChocolateyGUI, I found some issues using $host.SetShouldExit(youExitCode). The way I found around this was to explicitly throw an error. You can see an example of what was done here:

    https://github.com/chocolatey/ChocolateyGUI/blob/develop/BuildScrip...

    Gary

  5. 5 Posted by sbohlen on 12 Jan, 2015 09:09 PM

    sbohlen's Avatar

    Helpful -- thanks!

  6. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 01:53 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac