Home → Problems →

xUnit test execution hangs

• Edit

fresa

07 Jul, 2018 01:41 PM

I have a project where the building just hangs during test execution.

https://ci.appveyor.com/project/Fresa/test-it-while-hosting-your-web-application/build/1.0.6

I have several other projects that are similar and they work just fine, for example:
https://ci.appveyor.com/project/Fresa/test-it-while-hosting-your-windows-service/build/3.0.0

I'm seriously banging my head to the wall trying to figure out what the problem is. Any ideas? The tests works perfect locally. I've tried the xunit console runner locally (see attached screenshot) with all kinds of versions and it works flawless. I'm also using Resharper's test runner and Ncrunch, and the tests runs fine with them too.

Why does Appveyor hang at this point? I've seen many other similar topics in the forum, but I have not seen any solutions or indications of what the problem is.

• xunit.console.ouput.PNG 166 KB

1. 1 Posted by fresa on 07 Jul, 2018 04:36 PM

   

   There are some very weird behaviour here. I've been tinkering around with disabling test specifications, and I got it to run successfully when 1 out of 3 test specifications are disabled. To me this makes no sense.

   Take a look at this build that success (2 out of 3 test specifications enabled): https://ci.appveyor.com/project/Fresa/test-it-while-hosting-your-web-application/build/1.0.18

   Here I enable the third test specification and Appveyor hangs during test execution.
   https://ci.appveyor.com/project/Fresa/test-it-while-hosting-your-web-application/build/1.0.19

   What is going on here? Is there any way to get logs from the client running the tests?

   All of the tests in the 19 builds I've executed succeeds locally, so something weird is going on in the test execution process on the appveyor host.

   EDIT: Here is the commit history of the repository, so you can see the changes I've made between the working build and the hanged build: https://github.com/Fresa/Test.It.While.Hosting.Your.Web.Application/commits/master

   • Edit

2. Support Staff 2 Posted by Ilya Finkelshte... on 08 Jul, 2018 04:52 AM

   

   try to run tests in a single thread with custom test script like this:

   %xunit20%\xunit.console.x86 "C:\projects\test-it-while-hosting-your-web-application\tests\Test.It.While.Hosting.Your.Web.Application.Tests\bin\Release\net47\Test.It.While.Hosting.Your.Web.Application.Tests.dll" -appveyor -maxthreads 1
   

   I see that you are using UI, so you need to switch to SCRIPT from AUTO in Tests tab and add this command.

   Let us know how it goes.

   • Edit

3. 3 Posted by fresa on 08 Jul, 2018 10:22 AM

   

   Thanks for your reply!

   It did what you said, but it made no difference:
   https://ci.appveyor.com/project/Fresa/test-it-while-hosting-your-web-application/build/1.0.20

   • Edit

4. Support Staff 4 Posted by Ilya Finkelshte... on 10 Jul, 2018 02:54 AM

   

   Check PR and build. You can revert from test script back to auto-detection if you merge this change.

   • Edit

5. 5 Posted by fresa on 10 Jul, 2018 07:12 AM

   

   Turning off parallel test execution is not really a solution in this case since parallel test execution works. Locally I run the tests in parallel within the test assembly, both with the xunit console runner and ncrunch, and it works fine.

   This repo is a testing framework. Being able to run tests in parallel is quite important.

   • Edit

6. Support Staff 6 Posted by Ilya Finkelshte... on 10 Jul, 2018 08:15 PM

   

   Right, it works OK on AppVeyor too if I spin up "Premium" build VM (which is in beta testing now) with 4 cores for this build. Standard build VMs have 2 CPU cores. I am quite sure that your development VM has more than 2 cores too. Also it is pretty possible (though I can be mistaken) that if you run the same tests on some another VM outside of AppVeyor with 1 or 2 cores, you will see the same issue.
   Definitely it is a bug somewhere, not necessary in AppVeyor or your tests. We are investigating this, but it can easily happen that another xunit or .NET update will fix it before we even understand what it was.
   PR was to unblock you. If you want to be able to run tests in parallel locally, but still have AppVeyor build, you can create or update AssemblyInfo.cs on the fly at the init stage of the build with say PowerShell script.

   • Edit

7. 7 Posted by fresa on 11 Jul, 2018 09:17 PM

   

   I solved the issue! It was due to how xUnit synchronization context works with async/await in parallel test mode. There is a long discussion about it here: https://github.com/xunit/xunit/issues/864

   Really tricky stuff to understand without deep knowledge how xUnit has implemented it's test parallelism concept. Some documentation would have been nice atleast :)

   Anyhow, I rewrote my application and the tests, and now everything runs smoothly during the build!

   Thanks for your help! I really appreciate your support!

   • Edit

8. Support Staff 8 Posted by Ilya Finkelshte... on 11 Jul, 2018 09:40 PM

   

   Oh, great! Will look deeper at your commit and recommend this to other customers when needed, if you don't mind :)

   • Edit

9. 9 Posted by fresa on 11 Jul, 2018 09:47 PM

   

   Knock yourself out :)

   • Edit