Home → Problems →

XUnit Tests Not Finishing

• Edit

micah

02 Aug, 2015 06:51 PM

I recently updated my project to VS2015. I also recently updated to XUnit 2.0 (from 2.0 beta). My jobs are stalling out when running tests, as seen here:
https://ci.appveyor.com/project/Zoltu/repocreator/build/51

Looking at the tests tab, I can see that most of my tests ran fine and passed, but there are 3 each time that get stuck. Normally, I would assume that this was a problem with my tests but when I run them with the exact same command locally they work fine:

C:\Users\Micah\Source\RepoCreator>packages\xunit.runner.console.2.0.0\tools\xunit.console.x86 "Tests\bin\Release\Zoltu.RepoCreator.Tests.dll" -appveyor
xUnit.net console test runner (32-bit .NET 4.0.30319.42000)
Copyright (C) 2015 Outercurve Foundation.

Discovering: Zoltu.RepoCreator.Tests
Discovered:  Zoltu.RepoCreator.Tests
Starting: Zoltu.RepoCreator.Tests.DLL
   Zoltu.RepoCreator.Tests.GitHubTests.create_repo_with_replacements [SKIP]
      Not implemented yet.
Finished: Zoltu.RepoCreator.Tests.DLL

=== TEST EXECUTION SUMMARY ===
   Zoltu.RepoCreator.Tests.DLL  Total: 14, Errors: 0, Failed: 0, Skipped: 1, Time: 7.079s

My only guess is that it has something to do with whatever the -appveyor command line switch does or that there is some difference in the environment from my local environment.

Has anyone else run into this problem?

1. 1 Posted by micah on 02 Aug, 2015 07:07 PM

   

   In case it matters, one of the tests it is failing on connects to GitHub. Are the tests run behind a firewall of any kind?

   • Edit

2. Support Staff 2 Posted by Feodor Fitsner on 02 Aug, 2015 07:17 PM

   

   There is no firewall - any kind of outbound traffic is allowed. What resource/URL do you request? Did it work before?

   - Feodor

   • Edit

3. 3 Posted by micah on 02 Aug, 2015 07:25 PM

   

   It worked before upgrading to VS2015 and updating my xunit to 2.0 (from beta). It hits the GitHub API. Disabling the integration test doesn't appear to have resolved the issue, it just moves the problem to another test (that is a unit test, no external dependencies).

   • Edit

4. Support Staff 4 Posted by Feodor Fitsner on 02 Aug, 2015 08:16 PM

   

   OK, in order to check if the issue is connected to AppVeyor real-time logging you may try disabling it.

   1) Copy console runner execution command from build log, disable tests auto-discovery and use script instead with original command and -appveyor switch removed.
   2) remove APPVEYOR_API_URL environment variable in before_test: like

   before_test:
   - set APPVEYOR_API_URL=
   

   Also, you may want to check your tests locally on a build worker via RDP: http://www.appveyor.com/docs/how-to/rdp-to-build-worker

   • Edit

5. 5 Posted by micah on 02 Aug, 2015 08:25 PM

   

   The RDP thing answered the question immediately. My test asserted apparently (rather than throwing an exception). It appears to be an issue with code contracts on my assembly under test:

   [Window Title]
   xunit.console.x86.exe - Precondition failed.
   
   [Main Instruction]
   xunit.console.x86.exe - Precondition failed.
   
   [Content]
   Description: An assembly (probably "Zoltu.RepoCreator.Library") must be rewritten using the code contracts binary rewriter (CCRewrite) because it is calling Contract.Requires<TException> and the CONTRACTS_FULL symbol is defined.  Remove any explicit definitions of the CONTRACTS_FULL symbol from your project and rebuild.  CCRewrite can be downloaded from http://go.microsoft.com/fwlink/?LinkID=169180. 
   After the rewriter is installed, it can be enabled in Visual Studio from the project's Properties page on the Code Contracts pane.  Ensure that "Perform Runtime Contract Checking" is enabled, which will define CONTRACTS_FULL.
   
   [Expanded Information]
   Stack Trace:
      at System.Diagnostics.Contracts.Contract.Requires[TException](Boolean condition, String userMessage)
      at Zoltu.RepoCreator.Library.GitHub.RepoCloner..ctor(IGitHubClient templateGitHubClient, IGitHubClient destinationGitHubClient, Job job, Func`2 updateProgress) in C:\projects\repocreator\Library\GitHub.cs:line 78
      at Zoltu.RepoCreator.Library.GitHub.<CloneRepositoryFromTemplate>d__4.MoveNext() in C:\projects\repocreator\Library\GitHub.cs:line 234
      at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
      at Zoltu.RepoCreator.Library.GitHub.CloneRepositoryFromTemplate(IGitHubClient templateGitHubClient, IGitHubClient destinationGitHubClient, Job job, Func`2 updateProgress)
      at Zoltu.RepoCreator.Tests.GitHubTests.<create_repo_integration>d__13.MoveNext() in C:\projects\repocreator\Tests\GitHubTests.cs:line 260
      at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
      at Zoltu.RepoCreator.Tests.GitHubTests.create_repo_integration()
      at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
      at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
      at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
      at Xunit.Sdk.TestInvoker`1.<>c__DisplayClassc.<<InvokeTestMethodAsync>b__9>d__e.MoveNext()
      at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
      at Xunit.Sdk.TestInvoker`1.<>c__DisplayClassc.<InvokeTestMethodAsync>b__9()
      at Xunit.Sdk.ExecutionTimer.<AggregateAsync>d__0.MoveNext()
      at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
      at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction)
      at Xunit.Sdk.TestInvoker`1.<>c__DisplayClassc.<InvokeTestMethodAsync>b__8()
      at Xunit.Sdk.ExceptionAggregator.<RunAsync>d__0.MoveNext() in c:\TeamCity\buildAgent\work\74856245f07a90f0\src\xunit.core\Sdk\ExceptionAggregator.cs:line 90
      at System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[TStateMachine](TStateMachine& stateMachine)
      at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code)
      at Xunit.Sdk.TestInvoker`1.<InvokeTestMethodAsync>d__17.MoveNext()
      at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
      at Xunit.Sdk.TestInvoker`1.InvokeTestMethodAsync(Object testClassInstance)
      at Xunit.Sdk.TestInvoker`1.<<RunAsync>b__0>d__4.MoveNext()
      at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
      at Xunit.Sdk.TestInvoker`1.<RunAsync>b__0()
      at Xunit.Sdk.ExceptionAggregator.<RunAsync>d__3`1.MoveNext() in c:\TeamCity\buildAgent\work\74856245f07a90f0\src\xunit.core\Sdk\ExceptionAggregator.cs:line 107
      at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.Start[TStateMachine](TStateMachine& stateMachine)
      at Xunit.Sdk.ExceptionAggregator.RunAsync[T](Func`1 code)
      at Xunit.Sdk.TestInvoker`1.RunAsync()
      at Xunit.Sdk
   

   I can probably troubleshoot this specific problem from here, though it would be nice if an assertion was somehow propagated to the build output (this may not be possible). Would have saved me quite a bit of time troubleshooting.

   • Edit

6. Support Staff 6 Posted by Feodor Fitsner on 02 Aug, 2015 08:31 PM

   

   Great, thanks for the update!

   You may want to submit an issue to xUnit project.

   • Edit

7. 7 Posted by micah on 02 Aug, 2015 08:32 PM

   

   It appears that the VS2015 build image doesn't have code contracts installed. Can this be added to the build image?

   I am assuming that is the problem since when I build with os: Visual Studio 2015 my tests throw this assertion and I don't see the Code Contracts stuff running in the build output like I do on Windows Server 2012 R2.

   • Edit

8. Support Staff 8 Posted by Feodor Fitsner on 02 Aug, 2015 08:34 PM

   

   Sure, code contracts for VS 2015 will be installed: https://github.com/appveyor/ci/issues/346

   • Edit

9. Support Staff 9 Posted by Feodor Fitsner on 02 Aug, 2015 08:35 PM

   

   As a workaround for now you can use this simple script to install them during the build: https://gist.github.com/FeodorFitsner/9769cb61790387005445

   • Edit

10. 10 Posted by micah on 02 Aug, 2015 08:41 PM

    

    Thanks!

    • Edit

11. micah closed this discussion on 02 Aug, 2015 08:41 PM.