custom test framework

programmax's Avatar

programmax

01 Dec, 2018 12:47 PM

Hello,

I have a custom test framework that I would like to hook up to AppVeyor. It seems like my test target is building but it isn't running. You can see that here:
https://ci.appveyor.com/project/ProgramMax/max/build/job/ab8q67ee6y0pie05

(The test build target is maxAutomatedTests.exe)

I have a 2-part question:
1.) How do I get it running my executable for testing?
2.) Is there a special format which AppVeyor expects & can parse? Right now I just print to stdout.

Thanks!

  1. Support Staff 1 Posted by Ilya Finkelshte... on 03 Dec, 2018 02:17 PM

    Ilya Finkelshteyn's Avatar

    Hi Chris,

    By test assemblies AppVeyor assumes .NET test assemblies. AppVeyor checks those assemblies with reflection and runs respective .NET test runners (xunit, nunit etc) against them.

    This does not look like your case. What I believe you have to do is simple run your tests as a custom script. And if your test can produce XML results in one of formats described here, you should be able to upload test results to AppVeyor.

    Please let us know if this helps.

    Ilya.

  2. 2 Posted by programmax on 28 Dec, 2018 01:56 AM

    programmax's Avatar

    Hello Ilya,

    (This is Chris again. I think I hadn't created a forum account for my previous post.)

    I'm working this. I was able to get my tests running. I can format the output, no problem.

    But at some point I need to upload the output. Is there a handy tool that happens to come with the images we're using for uploading? I would prefer not to get too custom.

    Thanks!

  3. Support Staff 3 Posted by Ilya Finkelshte... on 28 Dec, 2018 05:20 AM

    Ilya Finkelshteyn's Avatar

    Hi Chris,

    Do you mean Tests tab for AppVeyor build? If yes, can your tests generate XML output compatible with any of formats described here?

    Or you want to upload test results to some external location?

    Ilya.

  4. 4 Posted by programmax on 02 Jan, 2019 12:56 AM

    programmax's Avatar

    Hello again,

    Happy New Year! And thank you for all the help you've been giving me.

    I do indeed mean the Tests tab. I can get around to the XML output. My first goal is just to figure out how to upload. I wondered about potential tools but it looks like the docs you linked provided a way to create a .NET object from within a PowerShell script.

    I'm trying to run a PowerShell script to upload an example XML file but am struggling with it. Would you mind helping me run this PowerShell script?

    The output is here: https://ci.appveyor.com/project/ProgramMax/max/build/job/u3d1yby5ls2t3skk
    The pull request with a bunch of poorly-described commits (sorry) is here:
    https://github.com/ProgramMax/max/pull/83

  5. 5 Posted by programmax on 02 Jan, 2019 10:12 AM

    programmax's Avatar

    I spent a bit more time on this and realized I'm struggling to get environment values inside a PowerShell script. I've tried $($env:APPVEYOR_BUILD_FOLDER), $env:APPVEYOR_BUILD_FOLDER, $(APPVEYOR_BUILD_FOLDER), and %APPVEYOR_BUILD_FOLDER%.

  6. Support Staff 6 Posted by Ilya Finkelshte... on 03 Jan, 2019 11:45 AM

    Ilya Finkelshteyn's Avatar

    I forked your repo and was able to upload your sample junit test results file: https://ci.appveyor.com/project/IlyaFinkelshteyn/max/builds/21354958

    Please let us know if this helps.

  7. 7 Posted by programmax on 03 Jan, 2019 04:25 PM

    programmax's Avatar

    Thank you. I followed your example of accessing the values inside the AppVeyor yaml file and everything is now working as expected.

    I assume if I wanted an external script I would have to set these values as environment variables.

    Thank you again for your help. You can delete your fork if you wish. :)

  8. Support Staff 8 Posted by Ilya Finkelshte... on 03 Jan, 2019 08:18 PM

    Ilya Finkelshteyn's Avatar

    Good, feel free to ping when needed!

  9. Ilya Finkelshteyn closed this discussion on 03 Jan, 2019 08:18 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac