or Create a profile

Home → Questions →

Is mspec supported?

byron's Avatar

byron

Jun 13, 2014 @ 11:04 PM

My team uses mspec (machine.specifications) for unit tests. Is there a way to have the build run mspec tests?

  1. Support Staff 1 Posted by Feodor Fitsner on Jun 13, 2014 @ 11:22 PM

    Feodor Fitsner's Avatar

    mspec runner is not installed on build workers, but you can download it from NuGet nuget install Machine.Specifications

  2. 2 Posted by byron on Jun 13, 2014 @ 11:25 PM

    byron's Avatar

    So, does that mean that I can have it in my packages.config file and
    somehow script it to run the tests with mspec? We already have a rake
    script in each project that does this for us. If it's possible to just run
    rake, then we're golden!

  3. 3 Posted by byron on Jun 13, 2014 @ 11:40 PM

    byron's Avatar

    I'd also love to hear about what kind of "test results" the build needs so that the mspec tests show up in the tests tab.

  4. Support Staff 4 Posted by Feodor Fitsner on Jun 13, 2014 @ 11:45 PM

    Feodor Fitsner's Avatar

    I see. There is integration with xUnit, NUnit and MSTest that uses Build Worker API. Generally, you can parse mspec results and push items using that API, but I guess it could be better to have a built-in support for that.

  5. 5 Posted by byron on Jun 13, 2014 @ 11:47 PM

    byron's Avatar

    Yes. Any chance you can "whip up" some built-in support for mspec? It's pretty popular for unit tests.

  6. Support Staff 6 Posted by Feodor Fitsner on Jun 13, 2014 @ 11:51 PM

    Feodor Fitsner's Avatar

    I will take a look for sure. If there is extensibility mechanism - great; otherwise we could deploy a patched runner. I'm adding an item into issues list.

  7. 7 Posted by James Skimming on Jun 16, 2014 @ 09:38 PM

    James Skimming's Avatar

    +1 for built in support for Machine.Specifications.

  8. 8 Posted by byron on Jun 17, 2014 @ 01:02 PM

    byron's Avatar

    When the build is "discovering tests", what is it looking for? I'm running my mspec tests from a rake script and I have a few options for output and can also control where the output files go. Maybe with some guidance, I can get appveyor to recognize my mspec test results as a work-around for now.

  9. Support Staff 9 Posted by Feodor Fitsner on Jun 17, 2014 @ 05:13 PM

    Feodor Fitsner's Avatar

    To discover assemblies with tests AppVeyor goes through all (or specified only) assemblies in the build folder and check their dependencies. So, if the assembly references xunit, nunit or MS Test assemblies it's being scheduled to test with appropriate test runner. mspec references is currently not being checked.

    We can implement custom listener for AppVeyor in the same way it's done for TeamCity

  10. 10 Posted by byron on Jun 17, 2014 @ 11:04 PM

    byron's Avatar

    Yeah, that listener looks promising. If I were to contribute that to the mspec library, would you be able to guide me a little?

  11. Support Staff 11 Posted by Feodor Fitsner on Jun 17, 2014 @ 11:15 PM

    Feodor Fitsner's Avatar

    Sure, let me know if you have any questions. Also, take a look at how the integration is made for xUnit: https://github.com/xunit/xunit/pull/117

  12. 12 Posted by byron on Jun 17, 2014 @ 11:20 PM

    byron's Avatar

    Ok, I hope I have time for this. :) Here's the github issue I will use to guide development.
    https://github.com/machine/machine.specifications/issues/232

  13. Support Staff 13 Posted by Feodor Fitsner on Jun 17, 2014 @ 11:23 PM

    Feodor Fitsner's Avatar

    Great. Whenever you need me to join the discussion mention @FeodorFitsner

  14. 14 Posted by byron on Jun 19, 2014 @ 08:52 PM

    byron's Avatar

    I have the integration with mspec working in my builds on AppVeyor. I submitted a pull request at mspec's github repo today and hopefully it'll get merged in soon and released. Until then, anyone who wants to use mspec with AppVeyor can pull down the branch from my pull request (https://github.com/machine/machine.specifications/pull/235), build it, and use the output in the /build folder to run your specs at build-time. I use rake to script my builds, so for me, it was a simple matter of changing the path for my mspec CLI test runner and viola!

  15. Support Staff 15 Posted by Feodor Fitsner on Jun 23, 2014 @ 03:34 AM

    Feodor Fitsner's Avatar

    Yeah, good job (and fast turnaround)! I've seen the discussion on GitHub.

    We'd probably wait until mspec guys merge your PR into master, then build from there and deploy to AppVeyor build workers. Let me know when PR is closed.

    In the meantime I'm going to pull from your PR and see how it's done.

    Thank you!

  16. 16 Posted by James Skimming on Jun 23, 2014 @ 04:29 PM

    James Skimming's Avatar

    Feodor

    Do you anticipate MSPec will be automatically picked up much like the other test frameworks once you deploy to the build workers?

    We also use MSpec, so I'm happy to provide feedback once delivered.

  17. Support Staff 17 Posted by Feodor Fitsner on Jun 23, 2014 @ 04:42 PM

    Feodor Fitsner's Avatar

    Absolutely! Adding autodiscovery for mspec is part of its support.

  18. 18 Posted by James Skimming on Jun 27, 2014 @ 07:55 PM

    James Skimming's Avatar

    Looks like the changes have been cherry picked and released as a beta on nuget (see this pull request comment).

    Do you have an update when you'll use this, or are you waiting until it's out of beta?

  19. Support Staff 19 Posted by Feodor Fitsner on Jun 28, 2014 @ 04:12 AM

    Feodor Fitsner's Avatar

    Hi James,

    Thanks for letting me know! I'll start integrating mspec in the coming days. Is there any AppVeyor project with sample tests and output I can compare to?

    - Feodor

  20. 20 Posted by James Skimming on Jun 28, 2014 @ 03:09 PM

    James Skimming's Avatar

    I don't have any public projects using MSpec, can you access private projects?

  21. Support Staff 21 Posted by Feodor Fitsner on Jun 28, 2014 @ 03:26 PM

    Feodor Fitsner's Avatar

  22. 22 Posted by James Skimming on Jun 28, 2014 @ 08:39 PM

    James Skimming's Avatar

    Understood. If you have the means to push out selective updates, i.e. to specific projects but not globally, then I'm happy to be a beta tester. If so let me know and I'll send you the details of the project, otherwise let me know when it's released and I'll give it a go.

  23. Support Staff 23 Posted by Feodor Fitsner on Jun 29, 2014 @ 09:28 AM

    Feodor Fitsner's Avatar

    OK, guys, I've started integrating mspec into AppVeyor and got it working - see attached screenshots.

    Few notes:

    • I'm getting wrong test duration (second screenshot) because -1 is getting passed into AppVeyor (relevant code line).
    • I think it makes sense to prefix test name with context name to have, in my case:
    when transferring between two accounts - should debit the from account by the amount transferred
when transferring between two accounts - should credit the to account by the amount transferred
when transferring an amount larger than the balance of the from account - should not allow the transfer
    • While encoding payload to JSON double quotes in value should be escaped (relevant code line)

    When you fix these things - let me know and I'll take from your pull request.

    Thank you! Keep up the great work!

  24. Support Staff 24 Posted by Feodor Fitsner on Jun 29, 2014 @ 09:30 AM

    Feodor Fitsner's Avatar

    Oh, forgot to mention - this is my sample tests: https://github.com/FeodorFitsner/mspec-tests

  25. 25 Posted by byron on Jun 29, 2014 @ 01:06 PM

    byron's Avatar

    Yeah, good catches. I'll fix those tomorrow. --
    Byron Sommardahl
    CTO, Co-Owner
    Acklen Avenue | www.acklenavenue.com | 615-669-8239
    230 4th Ave N Suite 301H Nashville, TN 37219

    Acklen Avenue is a custom software development company committed to
    creating quality software that people love to use.

  26. Support Staff 26 Posted by Feodor Fitsner on Jul 07, 2014 @ 07:55 AM

    Feodor Fitsner's Avatar

    It's been deployed to production: https://ci.appveyor.com/project/appvyr/mspec-tests

  27. 27 Posted by James Skimming on Jul 07, 2014 @ 11:30 PM

    James Skimming's Avatar

    Nice one, having just ran a new build I can see the MSpec tests have been picked up.

    Good work.

  28. Ilya Finkelshteyn closed this discussion on Aug 25, 2018 @ 01:45 AM.

Comments are currently closed for this discussion. You can start a new one.

  • New Issue

  • Conversation Started

  • The discussion is closed

    No more actions from AppVeyor or the discussion starter are required.

  • Re-open the discussion

Private Permissions

This discussion is private. Only you and AppVeyor support staff can see and reply to it.

Public Permissions

This discussion is public. Everyone can see and reply to it.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

16 Apr 19:22 very slow pull request checkout
11 Apr 00:42 Request to increase time limit or any tips to mitigate timeout
06 Apr 05:54 VS 2017 image license expired.
05 Apr 15:59 unable to build new
05 Apr 04:01 please increase 30 minute timeout

 

03 Apr 13:54 Project Quota Exceeded
03 Apr 01:02 Need feedback on GitHub project I am having issues with Windows Scrips
28 Mar 18:00 "Do not cancel running builds" option is not working as expected
18 Mar 20:12 Windows miniconda install of conda-forge packages fails with picosat: out of memory in 'resize'
18 Mar 17:08 curl: (35) schannel: next InitializeSecurityContext failed: CRYPT_E_REVOCATION_OFFLINE (0x80092013) - The revocation function was unable to check revocation because the revocation server was offline.
17 Mar 12:00 Projects quota exceeded.