Code coverage results

Scott Dorman's Avatar

Scott Dorman

31 May, 2014 01:15 AM

It would be great if we could get code coverage analysis as part of running the unit tests. I think this could be done now as part of a post build script that then pushes the code coverage information into the build log, but that isn't as nice as having it be an integrated part of the build process where the results are surfaced as an additional tab next to the Tests tab.

  1. Support Staff 1 Posted by Feodor Fitsner on 31 May, 2014 07:18 PM

    Feodor Fitsner's Avatar

    What tool would you recommend for running code coverage analysis?

  2. 2 Posted by Scott Dorman on 31 May, 2014 09:10 PM

    Scott Dorman's Avatar

    Personally, I'd love to see the ability to use the Microsoft code coverage tools but I don't know if (or how well) it works with test frameworks other than MSTest. The only other tool I know of is NCover, of which I have never been a big fan.

    Another option for this would be to give us a way to add new "tabs" ourselves. TeamCity has this option, where you basically configure it so that a "tab" points to a log file and then the contents of that log file appear as the tab contents. This would allow us to run any code coverage tool we want (or really any tool) through scripts and have the output log appear as build results.

  3. 3 Posted by Scott Dorman on 01 Jun, 2014 03:59 AM

    Scott Dorman's Avatar

    I'm not sure if you use vstest.console.exe to run just MSTest based unit tests, but it looks like you can use the /Enablecodecoverage switch on it to capture code coverage analysis results.

  4. 4 Posted by Antoine Blanche... on 02 Jun, 2014 12:04 AM

    Antoine Blanchet's Avatar

    dotCover is a great tool from JetBrains and it's available out of the box with TeamCity. It could be a great thing to have in AppVeyor but it's not free.

  5. Support Staff 5 Posted by Feodor Fitsner on 02 Jun, 2014 03:40 AM

    Feodor Fitsner's Avatar

    I'd look into vstest.console code coverage results and OpenCover as the most available options.

    DotCover is very popular, but it's commercial product with strict licensing from...hm...our competitor. I'm just wondering if I ask them about hosting DotCover on AppVeyor :)

  6. 6 Posted by Scott Dorman on 06 Aug, 2014 09:03 PM

    Scott Dorman's Avatar

    Any updates on this request?

  7. Support Staff 7 Posted by Feodor Fitsner on 07 Aug, 2014 01:00 PM

    Feodor Fitsner's Avatar

    Haven't looked into that yet, sorry. It doesn't sound trivial and we are trying to prioritize. There were Azure-related issues with build workers provisioning. It's resolved by now, but we want to make sure that part works well. Another thing we are going to deal with is deployment improvements.

  8. 8 Posted by giggio on 26 Dec, 2014 02:47 AM

    giggio's Avatar

    I am looking into OpenCover. Maybe you want to try that:
    https://github.com/OpenCover/opencover/

  9. 9 Posted by Barry on 28 Dec, 2014 01:31 PM

    Barry's Avatar

    +1 for this request.

    Be wonderful to be able to use DotCover in this way (or even add tabs from any script as Scott describes...)

  10. Support Staff 10 Posted by Feodor Fitsner on 28 Dec, 2014 10:22 PM

    Feodor Fitsner's Avatar

    DotCover requires license, no?

  11. 11 Posted by giggio on 29 Dec, 2014 03:22 AM

    giggio's Avatar

    DotCover does require a license, but I think they are free for Open Source projects.

  12. 12 Posted by Barry on 29 Dec, 2014 05:29 PM

    Barry's Avatar

    It seems to me to be not too complex to run OpenCover (& ReportGenerator) to do basic code coverage, here are some notes for how I hacked it together:

    http://www.endzone.co.uk/opencover-nunit-test-coverage-on-appveyor/

    Thoughts?

  13. Support Staff 13 Posted by Feodor Fitsner on 30 Dec, 2014 08:52 PM

    Feodor Fitsner's Avatar

    Looks great for the start! Additionally, you can write a simple script to push XML test results to "Tests" tab of build. Take a look at this thread for example: http://help.appveyor.com/discussions/kb/7-integrating-chutzpah-into...

    The API: http://www.appveyor.com/docs/build-worker-api

  14. 14 Posted by Barry O'Kane on 31 Dec, 2014 07:59 AM

    Barry O'Kane's Avatar

    I used the default nunit-console in the script - which means I am seeing
    the results to the "Tests" already (no extra work!)

    Is that what you meant?

  15. Support Staff 15 Posted by Feodor Fitsner on 31 Dec, 2014 08:03 AM

    Feodor Fitsner's Avatar

    Yeah, exactly. Nice that it worked as is :)

  16. 16 Posted by shaun_wilde on 01 Jan, 2015 11:58 PM

    shaun_wilde's Avatar

    If you are looking to add OpenCover please feel free to raise a support issue. I am currently trying out apveyor for OpenCover building and I use opencover to gather coverage on opencover tests (dogfooding).

    Since you are probably running as a non-admin then remember to use the -register:user option when running nunit, mstest, xunit etc etc.

  17. 17 Posted by giggio on 13 Jan, 2015 01:27 PM

    giggio's Avatar

    I have been able to integrate OpenCover, ReportGenerator and Coveralls with Appveyor.
    Results are here:
    https://ci.appveyor.com/project/code-cracker/code-cracker/build/1.0.0.303
    Download the "log" artifacts and you will be able to see the report results. And on Coveralls:
    https://coveralls.io/r/code-cracker/code-cracker?branch=master
    Source code is here:
    https://github.com/code-cracker/code-cracker
    I am using coveralls.net to do it. All build files are on the root folder.

  18. 18 Posted by Barry O'Kane on 13 Jan, 2015 02:03 PM

    Barry O'Kane's Avatar

    Nice.

    I wish coveralls.io supported BitBucket

  19. 19 Posted by giggio on 13 Jan, 2015 02:45 PM

    giggio's Avatar

    Just ask them! :)
    https://github.com/lemurheavy/coveralls-public/issues
    I actually just opened an issue so that they can add Appveyor to their supported build servers:
    https://github.com/lemurheavy/coveralls-public/issues/429
    Maybe you want to help by commenting there. :)

  20. 20 Posted by Barry O'Kane on 13 Jan, 2015 02:50 PM

    Barry O'Kane's Avatar
  21. 21 Posted by Christian Sverr... on 29 Jul, 2015 01:32 PM

    Christian Sverrhag's Avatar

    Any updates on built in support for code coverage in AppVeyor?

  22. 22 Posted by aliman on 17 Oct, 2015 03:29 PM

    aliman's Avatar

    Would also be interested in such integration. OpenCover + ReportGenerator looks like a good tandem for this

  23. 23 Posted by jacob.bjorke on 02 Nov, 2015 09:48 PM

    jacob.bjorke's Avatar

    Any update on this?

  24. 24 Posted by joymon on 05 Jan, 2016 04:43 PM

    joymon's Avatar

    Is this feature available?
    + 1 for out of the box support in AppVeyor for code coverage at least with MSTests.
    Thanks in advance
    Joy

  25. 25 Posted by tomasz on 23 Mar, 2016 07:28 PM

    tomasz's Avatar

    For everyone's information there's also http://codecov.io, which works great with appveyor and opencover. It takes a little setup though. I've got it working with nunit and xunit. See my appveyor.xml in wikibus/JsonLD.Entities and wikibus/Argolis github repos

  26. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:04 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac