Fail AppVeyor build if test count drops significantly

jeroen.heijmans's Avatar

jeroen.heijmans

19 Jun, 2019 09:35 AM

Is there anything available in AppVeyor to notice a significant change in Test Count? I would like my build to fail if the number of tests in my project suddenly drops by (say) 5% or more.

I've tried searching for this feature but found no relevant results.

The motivation here is that due to a problematic refactor, a significant number of tests (some of which would've been broken) no longer got seen by xUnit. Nothing in my build pipeline warned me about this, so it's not until days later until we found out that we were ignore a subset of our tests. I'd like to prevent that from happening.

I'm happy to write some custom YML or PS1 to do this, but then I'd need to know if there's variables available that give me the current and previous build's (or master build's) total number of tests. The only workaround I could think of is some kind of Baron von Munchhausen thing where the AppVeyor build queries the AppVeyor API with my own secret.... but that sounds fragile, at best.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

 

26 Sep, 2024 03:49 PM
26 Sep, 2024 09:02 AM
25 Sep, 2024 07:07 PM
24 Sep, 2024 08:39 PM
24 Sep, 2024 06:47 AM
20 Sep, 2024 05:50 PM