Home → Suggestions →

Filtering for Tests in test report

• Edit

bradphelan

24 Apr, 2014 01:39 PM

I have over 800 tests and scanning for failed ones is a bit hard with that number. A button to filter only failed tests and a field for regexp tests would be great.

If possible links to the source at the specific version in github would be brilliant.

1. Support Staff 1 Posted by Feodor Fitsner on 24 Apr, 2014 05:32 PM

   

   That's clear test results page must be more "friendly" for large projects. What testing framework do you currently use?

   • Edit

2. 2 Posted by bradphelan on 25 Apr, 2014 05:39 AM

   

   I'm using XUnit. Our tests are heavy on numerical algorithms and we use data theories quite a bit. Therefore it would be nice to group all the theory tests by their parent test.

   Another nice feature would be to click on a failing issue and automatically create an issue to fix it under github issues with a link back to the failing test. When that test goes green possibly on the next run you could have an option to automatically change the status to closed or flag it for review or something like that.

   • Edit

3. Support Staff 3 Posted by Feodor Fitsner on 25 Apr, 2014 05:54 PM

   

   Grouping theory tests is non-trivial. From integration standpoint it looks like xUnit is running the same test again (with the same name). I think it could be "hacked" using reflection, but this is not a few hours :)

   I like the idea of integrating tests into GitHub flow, but maybe one issue per all failed tests? What if 100 tests failed?

   • Edit

4. 4 Posted by bradphelan on 28 Apr, 2014 05:03 AM

   

   It kind of depends on the failure. If 100 tests go from passing to failing in a single checkin then it is most likely that it is a single source of error. If however there are 100 failing tests that have built up over time then you might want to associate specific issues or milestones with that test.

   I think two cases should be handled

   • generically create an issue and link the failing log in it's entirety.
   • link a specific test case to a new issue.

   It may even be useful to link a passing test case to an issue if it might be required to add more tests or something other of note arises when review the tests.

   • Edit

5. 5 Posted by Einar Ingebrigt... on 16 Jul, 2014 02:00 PM

   

   Is this something that is being worked on? We have 1986 tests (MSpec and custom JavaScript test runner) and now there are 2 failing - I luckily happen to know which ones, but that is seldom the case: ( https://ci.appveyor.com/project/Dolittle/bifrost/build/1.0.22 ).

   • Edit

6. Support Staff 6 Posted by Feodor Fitsner on 16 Jul, 2014 02:08 PM

   

   While a new test results page with better experience is being planned I think showing failed first will do the current one a little better.

   • Edit

7. 7 Posted by Einar Ingebrigt... on 16 Jul, 2014 02:36 PM

   

   That would really help.

   Other than that; love your service! Keep up the great work!
   —
   Sent from Mailbox

   • Edit

8. Support Staff 8 Posted by Feodor Fitsner on 17 Jul, 2014 08:39 AM

   

   Update has been just deployed. Failed tests come first in test results report.

   • Edit

9. 9 Posted by Einar Ingebrigt... on 17 Jul, 2014 08:42 AM

   

   Wow!
   Great stuff!

   Thanks a lot, this was really helpful!

   Amazed at your responsiveness, keep up the great work!

   • Edit

10. 10 Posted by chudinia on 25 Jun, 2015 12:45 PM

    

    It would be nice to have "Failed only" filter by default for tests
    And paging

    Having 2k tests is really hangs the page...

    • Edit

11. 11 Posted by michael on 29 Jan, 2017 02:52 AM

    

    I have to second the request. from llya. We have close to 4k tests in our test suite and the test page is unusable. I keep getting error from chrome saying that the page has become unresponsive and asking me if I want to kill it.

    It would be great to ONLY show failed tests by default. Passing tests are irrelevant in most cases. Are you planing on implementing this feature at some point?

    • Edit

12. 12 Posted by Einar Ingebrigt... on 29 Jan, 2017 02:53 AM

    

    I'm on business travel - will be back on the 6th of February. Won't be checking this email all that much.

    Best regards,

    Einar

    -

    Jeg er p? jobbreise, tilbake 6. februar. Vil frem til da ikke sjekke denne eposten noe saerlig.

    Mvh,

    Einar

    • Edit

13. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:14 AM.