Home → Problems →

Xunit 2 tests discovered but runner not found

• Edit

julian.elve

16 Jul, 2015 05:17 PM

Discovering tests...OK
%xunit20%\xunit.console.x86 "c:\projects\SSAT.Xrm.Core.Code\SSAT.XRM.Test\bin\Debug\SSAT.Xrm.Test.dll" -appveyor -notrait "category=Skip_CI" xUnit.net console test runner (32-bit .NET 4.0.30319.34209)
Copyright (C) 2015 Outercurve Foundation.

Discovering: SSAT.Xrm.Test
System.ArgumentException: Unknown test framework: Could not find xunit.dll or xunit.execution.desktop.dll.

1. Support Staff 1 Posted by Feodor Fitsner on 16 Jul, 2015 06:09 PM

   

   Take a look at this thread: http://help.appveyor.com/discussions/problems/1841-xunit-version-2-...

   Looks like nuget restore command before build should do a trick.

   • Edit

2. 2 Posted by julian.elve on 16 Jul, 2015 06:21 PM

   

   Hmm, have already got nuget restore before build, but will read that thread.. Thanks Feodor

   • Edit

3. 3 Posted by julian.elve on 17 Jul, 2015 07:27 AM

   

   Sorry that link is returning "private or deleted" - even when I am logged in

   • Edit

4. Support Staff 4 Posted by Feodor Fitsner on 17 Jul, 2015 03:26 PM

   

   OK, then I'll just copy some of my replies there.

   One:

   The problem is that xunit.execution.desktop.dll is missing from test project output directory. My "rough" solution was explicitly copying that assembly into build output directory, but apparently "correct" way of fixing that was updating project references. Seems like a breaking change in xunit itself.

   Take a look at last messages in this conversation: http://help.appveyor.com/discussions/problems/677-xunit-20-support#...

   Two:

   Take a look at this test xUnit 2.0 project: https://github.com/FeodorFitsner/xunit-20-tests and its build results: https://ci.appveyor.com/project/FeodorFitsner/xunit-20-tests - it works. Pay attention to line 110 which shows contents of project output folder - xunit.execution.desktop.dll is there.

   The question is why xunit.execution.desktop.dll is not getting copied in your case? If you look inside Xunit20Tests_AnyCPU.csproj you'll see that it includes xunit.core.props which adds xunit.execution.desktop.dll to the build output. Make sure a) you have this include in your test project, the path to it is correct and that file exists.

   This conversation could help too: http://help.appveyor.com/discussions/problems/677-xunit-20-support#...

   • Edit

5. 5 Posted by kirk.marple on 01 Aug, 2015 07:56 PM

   

   I'm having similar problems here, and haven't figured it out yet. Any help would be much appreciated.

   I'm using DNX projects, so managing references is a bit different than described above. If I look at my local build, I'm not getting any xunit*.dll in the output directory, not surprisingly, since it's included as a package dependency.

   Doing a nuget restore doesn't have any effect, before running the tests.

   One thing is it needs to find xunit.execution.dnx.dll (not desktop), which on my system lives in:

   C:\Users\Administrator\.dnx\packages\xunit.extensibility.execution\2.1.0-beta3-b
   uild3029\lib\dnx451\xunit.execution.dnx.dll

   ------------

   nuget restore
   Discovering tests...OK
   %xunit20%\xunit.console.x86 "C:\projects\candela-onboarding-paas\test\artifacts\bin\Frameworks.Data.Tests\Release\dnx46\Candela.Frameworks.Data.Tests.dll" -appveyor
   xUnit.net console test runner (32-bit .NET 4.0.30319.42000)
   Copyright (C) 2015 Outercurve Foundation.
    
   Discovering: Candela.Frameworks.Data.Tests
   System.ArgumentException: Unknown test framework: Could not find xunit.dll or xunit.execution.desktop.dll.
   Parameter name: C:\projects\candela-onboarding-paas\test\artifacts\bin\Frameworks.Data.Tests\Release\dnx46\Candela.Frameworks.Data.Tests.dll
   Command exited with code 1

   • Edit

6. Support Staff 6 Posted by Feodor Fitsner on 01 Aug, 2015 07:58 PM

   

   How do you run those tests locally then?

   • Edit

7. 7 Posted by kirk.marple on 01 Aug, 2015 08:01 PM

   

   I've been running them through VS 2015. They are auto-discovered into the Test Explorer.

   My project.json has references to the DNX runner:

          "xunit": "2.1.0-beta3-*",
           "xunit.runner.dnx": "2.1.0-beta3-*",
    
   And has one test command:

       "commands": {
           "test": "xunit.runner.dnx"
       },

   Also, looking at Tests output window, they appear to be getting executed as:

   Starting Microsoft.Framework.TestHost [C:\Users\Administrator\.dnx\runtimes\dnx-clr-win-x86.1.0.0-beta6\bin\dnx.exe --appbase "E:\Projects\Candela Onboarding PaaS\test\Candela.Frameworks.Resources.Tests" Microsoft.Framework.ApplicationHost --port 57331 Microsoft.Framework.TestHost --port 58618]
   Connected to Microsoft.Framework.TestHost
   Discovering tests in 'E:\Projects\Candela Onboarding PaaS\test\Candela.Frameworks.Resources.Tests\project.json'

   • test-explorer.PNG 21.2 KB
   • Edit

8. Support Staff 8 Posted by Feodor Fitsner on 01 Aug, 2015 08:04 PM

   

   Found this: https://github.com/xunit/dnx.xunit

   • Edit

9. 9 Posted by kirk.marple on 01 Aug, 2015 08:07 PM

   

   Yep, that's what I'm using. I guess VS 2015 is hiding a lot of the work.

   Let me try just running 'dnx . test' directly as a test script

   # Restore NuGet packages
   dnu restore

   # Run tests (change "." with a folder path if tests are not in the current directory)
   dnx . test

   • Edit

10. 10 Posted by kirk.marple on 01 Aug, 2015 09:02 PM

    

    OK, I was able to get this working, with a little batch command to auto-discover all the project.json files. Had to hack the filename wildcard so it skips project.lock.json. Just using literal 'project.json' isn't treated as a wildcard and doesn't actually find all the files.

    --------

    dnu restore

    FOR /R .\test %%I IN (projec?.json) DO dnx %%I test

    ------------

    FOR /R .\test %%I IN (projec?.json) DO dnx %%I test
    xUnit.net DNX Runner (32-bit DNX 4.6)
      Discovering: Candela.Frameworks.Data.Tests
      Discovered: Candela.Frameworks.Data.Tests
      Starting: Candela.Frameworks.Data.Tests
        Candela.Frameworks.Data.Tests.SampleTests.FailingTest [FAIL]
          Assert.Equal() Failure
          Expected: 5
          Actual: 4
          Stack Trace:
            test\Candela.Frameworks.Data.Tests\SampleTests.cs(20,0): at Candela.Frameworks.Data.Tests.SampleTests.FailingTest()
      Finished: Candela.Frameworks.Data.Tests
    === TEST EXECUTION SUMMARY ===
       Candela.Frameworks.Data.Tests Total: 2, Errors: 0, Failed: 1, Skipped: 0, Time: 0.112s
    xUnit.net DNX Runner (32-bit DNX 4.6)
      Discovering: Candela.Frameworks.Files.Tests
      Discovered: Candela.Frameworks.Files.Tests
      Starting: Candela.Frameworks.Files.Tests
        Candela.Frameworks.Files.Tests.SampleTests.FailingTest [FAIL]
          Assert.Equal() Failure
          Expected: 5
          Actual: 4
          Stack Trace:
            test\Candela.Frameworks.Files.Tests\SampleTests.cs(20,0): at Candela.Frameworks.Files.Tests.SampleTests.FailingTest()
      Finished: Candela.Frameworks.Files.Tests
    === TEST EXECUTION SUMMARY ===
       Candela.Frameworks.Files.Tests Total: 2, Errors: 0, Failed: 1, Skipped: 0, Time: 0.101s
    xUnit.net DNX Runner (32-bit DNX 4.6)
      Discovering: Candela.Frameworks.Resources.Tests
      Discovered: Candela.Frameworks.Resources.Tests
      Starting: Candela.Frameworks.Resources.Tests
        Candela.Frameworks.Resources.Tests.SampleTests.FailingTest [FAIL]
          Assert.Equal() Failure
          Expected: 5
          Actual: 4
          Stack Trace:
            test\Candela.Frameworks.Resources.Tests\SampleTests.cs(20,0): at Candela.Frameworks.Resources.Tests.SampleTests.FailingTest()
      Finished: Candela.Frameworks.Resources.Tests
    === TEST EXECUTION SUMMARY ===

    • Edit

11. Support Staff 11 Posted by Feodor Fitsner on 02 Aug, 2015 06:20 PM

    

    All this new "dnx/dnu/..." stuff looks so weird :)

    Looks like it's time to start learning it though...

    • Edit

12. 12 Posted by kirk.marple on 02 Aug, 2015 09:36 PM

    

    I've tweaked my script a little bit, so it sets the proper working directory to find config.json files.

    Only downside to running this as a script is it doesn't fail the build, if any tests fail. I do see the failed tests in the Tests page, but the build still succeeds. I'm assuming it's because the exit code is from the last 'dnx' execution. I can probably figure out how to accumulate the exit codes and provide a non-zero exit code, but it would be better if this was all done automatically :)

    ----------

    dnu restore

    FOR /R .\test %%I IN (projec?.json) DO (pushd "%%~dpI" && dnx . test && popd)

    • Edit

13. Support Staff 13 Posted by Feodor Fitsner on 02 Aug, 2015 10:50 PM

    

    Yes, AppVeyor relies on exit code to decide whether build should be terminated or not.
    You can do .cmd, check exit code in FOR body and then goto exit if it's not zero. Alternatively, you can use PowerShell to implement that logic.

    • Edit

14. julian.elve closed this discussion on 08 Oct, 2015 07:28 AM.