Home → Questions →

custom test framework

• Edit

programmax

01 Dec, 2018 12:47 PM

Hello,

I have a custom test framework that I would like to hook up to AppVeyor. It seems like my test target is building but it isn't running. You can see that here:
https://ci.appveyor.com/project/ProgramMax/max/build/job/ab8q67ee6y0pie05

(The test build target is maxAutomatedTests.exe)

I have a 2-part question:
1.) How do I get it running my executable for testing?
2.) Is there a special format which AppVeyor expects & can parse? Right now I just print to stdout.

Thanks!

1. 1 Posted by Ilya Finkelshte... on 03 Dec, 2018 02:17 PM

   

   Hi Chris,

   By test assemblies AppVeyor assumes .NET test assemblies. AppVeyor checks those assemblies with reflection and runs respective .NET test runners (xunit, nunit etc) against them.

   This does not look like your case. What I believe you have to do is simple run your tests as a custom script. And if your test can produce XML results in one of formats described here, you should be able to upload test results to AppVeyor.

   Please let us know if this helps.

   Ilya.

   • Edit

2. 2 Posted by programmax on 28 Dec, 2018 01:56 AM

   

   Hello Ilya,

   (This is Chris again. I think I hadn't created a forum account for my previous post.)

   I'm working this. I was able to get my tests running. I can format the output, no problem.

   But at some point I need to upload the output. Is there a handy tool that happens to come with the images we're using for uploading? I would prefer not to get too custom.

   Thanks!

   • Edit

3. 3 Posted by Ilya Finkelshte... on 28 Dec, 2018 05:20 AM

   

   Hi Chris,

   Do you mean Tests tab for AppVeyor build? If yes, can your tests generate XML output compatible with any of formats described here?

   Or you want to upload test results to some external location?

   Ilya.

   • Edit

4. 4 Posted by programmax on 02 Jan, 2019 12:56 AM

   

   Hello again,

   Happy New Year! And thank you for all the help you've been giving me.

   I do indeed mean the Tests tab. I can get around to the XML output. My first goal is just to figure out how to upload. I wondered about potential tools but it looks like the docs you linked provided a way to create a .NET object from within a PowerShell script.

   I'm trying to run a PowerShell script to upload an example XML file but am struggling with it. Would you mind helping me run this PowerShell script?

   The output is here: https://ci.appveyor.com/project/ProgramMax/max/build/job/u3d1yby5ls2t3skk
   The pull request with a bunch of poorly-described commits (sorry) is here:
   https://github.com/ProgramMax/max/pull/83

   • Edit

5. 5 Posted by programmax on 02 Jan, 2019 10:12 AM

   

   I spent a bit more time on this and realized I'm struggling to get environment values inside a PowerShell script. I've tried $($env:APPVEYOR_BUILD_FOLDER), $env:APPVEYOR_BUILD_FOLDER, $(APPVEYOR_BUILD_FOLDER), and %APPVEYOR_BUILD_FOLDER%.

   • Edit

6. 6 Posted by Ilya Finkelshte... on 03 Jan, 2019 11:45 AM

   

   I forked your repo and was able to upload your sample junit test results file: https://ci.appveyor.com/project/IlyaFinkelshteyn/max/builds/21354958

   Please let us know if this helps.

   • Edit

7. 7 Posted by programmax on 03 Jan, 2019 04:25 PM

   

   Thank you. I followed your example of accessing the values inside the AppVeyor yaml file and everything is now working as expected.

   I assume if I wanted an external script I would have to set these values as environment variables.

   Thank you again for your help. You can delete your fork if you wish. :)

   • Edit

8. 8 Posted by Ilya Finkelshte... on 03 Jan, 2019 08:18 PM

   

   Good, feel free to ping when needed!

   • Edit

9. Ilya Finkelshteyn closed this discussion on 03 Jan, 2019 08:18 PM.