Home → Questions →

Is mspec supported?

• Edit

byron

13 Jun, 2014 11:04 PM

My team uses mspec (machine.specifications) for unit tests. Is there a way to have the build run mspec tests?

1. Support Staff 1 Posted by Feodor Fitsner on 13 Jun, 2014 11:22 PM

   

   mspec runner is not installed on build workers, but you can download it from NuGet nuget install Machine.Specifications

   • Edit

2. 2 Posted by byron on 13 Jun, 2014 11:25 PM

   

   So, does that mean that I can have it in my packages.config file and
   somehow script it to run the tests with mspec? We already have a rake
   script in each project that does this for us. If it's possible to just run
   rake, then we're golden!

   • Edit

3. 3 Posted by byron on 13 Jun, 2014 11:40 PM

   

   I'd also love to hear about what kind of "test results" the build needs so that the mspec tests show up in the tests tab.

   • Edit

4. Support Staff 4 Posted by Feodor Fitsner on 13 Jun, 2014 11:45 PM

   

   I see. There is integration with xUnit, NUnit and MSTest that uses Build Worker API. Generally, you can parse mspec results and push items using that API, but I guess it could be better to have a built-in support for that.

   • Edit

5. 5 Posted by byron on 13 Jun, 2014 11:47 PM

   

   Yes. Any chance you can "whip up" some built-in support for mspec? It's pretty popular for unit tests.

   • Edit

6. Support Staff 6 Posted by Feodor Fitsner on 13 Jun, 2014 11:51 PM

   

   I will take a look for sure. If there is extensibility mechanism - great; otherwise we could deploy a patched runner. I'm adding an item into issues list.

   • Edit

7. 7 Posted by James Skimming on 16 Jun, 2014 09:38 PM

   

   +1 for built in support for Machine.Specifications.

   • Edit

8. 8 Posted by byron on 17 Jun, 2014 01:02 PM

   

   When the build is "discovering tests", what is it looking for? I'm running my mspec tests from a rake script and I have a few options for output and can also control where the output files go. Maybe with some guidance, I can get appveyor to recognize my mspec test results as a work-around for now.

   • Edit

9. Support Staff 9 Posted by Feodor Fitsner on 17 Jun, 2014 05:13 PM

   

   To discover assemblies with tests AppVeyor goes through all (or specified only) assemblies in the build folder and check their dependencies. So, if the assembly references xunit, nunit or MS Test assemblies it's being scheduled to test with appropriate test runner. mspec references is currently not being checked.

   We can implement custom listener for AppVeyor in the same way it's done for TeamCity

   • Edit

10. 10 Posted by byron on 17 Jun, 2014 11:04 PM

    

    Yeah, that listener looks promising. If I were to contribute that to the mspec library, would you be able to guide me a little?

    • Edit

11. Support Staff 11 Posted by Feodor Fitsner on 17 Jun, 2014 11:15 PM

    

    Sure, let me know if you have any questions. Also, take a look at how the integration is made for xUnit: https://github.com/xunit/xunit/pull/117

    • Edit

12. 12 Posted by byron on 17 Jun, 2014 11:20 PM

    

    Ok, I hope I have time for this. :) Here's the github issue I will use to guide development.
    https://github.com/machine/machine.specifications/issues/232

    • Edit

13. Support Staff 13 Posted by Feodor Fitsner on 17 Jun, 2014 11:23 PM

    

    Great. Whenever you need me to join the discussion mention @FeodorFitsner

    • Edit

14. 14 Posted by byron on 19 Jun, 2014 08:52 PM

    

    I have the integration with mspec working in my builds on AppVeyor. I submitted a pull request at mspec's github repo today and hopefully it'll get merged in soon and released. Until then, anyone who wants to use mspec with AppVeyor can pull down the branch from my pull request (https://github.com/machine/machine.specifications/pull/235), build it, and use the output in the /build folder to run your specs at build-time. I use rake to script my builds, so for me, it was a simple matter of changing the path for my mspec CLI test runner and viola!

    • Edit

15. Support Staff 15 Posted by Feodor Fitsner on 23 Jun, 2014 03:34 AM

    

    Yeah, good job (and fast turnaround)! I've seen the discussion on GitHub.

    We'd probably wait until mspec guys merge your PR into master, then build from there and deploy to AppVeyor build workers. Let me know when PR is closed.

    In the meantime I'm going to pull from your PR and see how it's done.

    Thank you!

    • Edit

16. 16 Posted by James Skimming on 23 Jun, 2014 04:29 PM

    

    Feodor

    Do you anticipate MSPec will be automatically picked up much like the other test frameworks once you deploy to the build workers?

    We also use MSpec, so I'm happy to provide feedback once delivered.

    • Edit

17. Support Staff 17 Posted by Feodor Fitsner on 23 Jun, 2014 04:42 PM

    

    Absolutely! Adding autodiscovery for mspec is part of its support.

    • Edit

18. 18 Posted by James Skimming on 27 Jun, 2014 07:55 PM

    

    Looks like the changes have been cherry picked and released as a beta on nuget (see this pull request comment).

    Do you have an update when you'll use this, or are you waiting until it's out of beta?

    • Edit

19. Support Staff 19 Posted by Feodor Fitsner on 28 Jun, 2014 04:12 AM

    

    Hi James,

    Thanks for letting me know! I'll start integrating mspec in the coming days. Is there any AppVeyor project with sample tests and output I can compare to?

    - Feodor

    • Edit

20. 20 Posted by James Skimming on 28 Jun, 2014 03:09 PM

    

    I don't have any public projects using MSpec, can you access private projects?

    • Edit

21. Support Staff 21 Posted by Feodor Fitsner on 28 Jun, 2014 03:26 PM

    

    I can't. Will be using samples: https://github.com/machine/machine.specifications/tree/master/Sourc...

    • Edit

22. 22 Posted by James Skimming on 28 Jun, 2014 08:39 PM

    

    Understood. If you have the means to push out selective updates, i.e. to specific projects but not globally, then I'm happy to be a beta tester. If so let me know and I'll send you the details of the project, otherwise let me know when it's released and I'll give it a go.

    • Edit

23. Support Staff 23 Posted by Feodor Fitsner on 29 Jun, 2014 09:28 AM

    

    OK, guys, I've started integrating mspec into AppVeyor and got it working - see attached screenshots.

    Few notes:

    • I'm getting wrong test duration (second screenshot) because -1 is getting passed into AppVeyor (relevant code line).
    • I think it makes sense to prefix test name with context name to have, in my case:

    when transferring between two accounts - should debit the from account by the amount transferred
    when transferring between two accounts - should credit the to account by the amount transferred
    when transferring an amount larger than the balance of the from account - should not allow the transfer
    

    • While encoding payload to JSON double quotes in value should be escaped (relevant code line)

    When you fix these things - let me know and I'll take from your pull request.

    Thank you! Keep up the great work!

    • mspec-console.png 43.5 KB
    • mspec-test-results.png 15.4 KB
    • Edit

24. Support Staff 24 Posted by Feodor Fitsner on 29 Jun, 2014 09:30 AM

    

    Oh, forgot to mention - this is my sample tests: https://github.com/FeodorFitsner/mspec-tests

    • Edit

25. 25 Posted by byron on 29 Jun, 2014 01:06 PM

    

    Yeah, good catches. I'll fix those tomorrow. --
    Byron Sommardahl
    CTO, Co-Owner
    Acklen Avenue | www.acklenavenue.com | 615-669-8239
    230 4th Ave N Suite 301H Nashville, TN 37219

    Acklen Avenue is a custom software development company committed to
    creating quality software that people love to use.

    • Edit

26. Support Staff 26 Posted by Feodor Fitsner on 07 Jul, 2014 07:55 AM

    

    It's been deployed to production: https://ci.appveyor.com/project/appvyr/mspec-tests

    • Edit

27. 27 Posted by James Skimming on 07 Jul, 2014 11:30 PM

    

    Nice one, having just ran a new build I can see the MSpec tests have been picked up.

    Good work.

    • Edit

28. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 01:45 AM.