AppVeyor test service erroring

James's Avatar

James

05 Jan, 2016 09:40 AM

I get this error when running tests:

Exception: When sending 'POST http://localhost:1034/api/tests' with body '{"TestName":"Newtonsoft.Json.Tests.Linq.JsonPath.JPathExecuteTests.EvaluateArrayOutOfBoundsIndxerWithError","FileName":"Newtonsoft.Json.Tests.dll","TestFramework":"xUnit","Outcome":"Running","DurationMilliseconds":null,"ErrorMessage":null,"ErrorStackTrace":null,"StdOut":null,"StdErr":null}', exception was thrown: A task was canceled.

  1. Support Staff 1 Posted by Feodor Fitsner on 05 Jan, 2016 04:20 PM

    Feodor Fitsner's Avatar

    Hi James,
    Do you get this error every time you send such request?

    -Feodor

  2. 2 Posted by James Newton-Ki... on 05 Jan, 2016 06:50 PM

    James Newton-King's Avatar

    It failed 4 builds in a row before I gave up retrying, although the test it
    failed on changed with each build.

  3. Support Staff 3 Posted by Feodor Fitsner on 05 Jan, 2016 06:52 PM

    Feodor Fitsner's Avatar

    How do you send request to API?

    -Feodor

  4. 4 Posted by James Newton-Ki... on 05 Jan, 2016 06:54 PM

    James Newton-King's Avatar

    I don't call it myself, I'm running tests with xUnit and it is
    automatically calling it. You can see its name against the TestFramework
    property in the error message's JSON.

  5. Support Staff 5 Posted by Feodor Fitsner on 05 Jan, 2016 06:57 PM

    Feodor Fitsner's Avatar

    Can you drop a link to a build please?

    -Feodor

  6. 6 Posted by James Newton-Ki... on 05 Jan, 2016 06:58 PM

    James Newton-King's Avatar
  7. Support Staff 7 Posted by Feodor Fitsner on 05 Jan, 2016 07:10 PM

    Feodor Fitsner's Avatar
  8. 8 Posted by James Newton-Ki... on 05 Jan, 2016 07:30 PM

    James Newton-King's Avatar

    It passed. Did you change anything? If it is a transient error you should
    add some logging to figure out why it is happening and how often.

  9. Support Staff 9 Posted by Feodor Fitsner on 05 Jan, 2016 07:42 PM

    Feodor Fitsner's Avatar

    Nope, I changed nothing.

    It would be nice to reproduce this error though. Looking at your logs it's "all or nothing" and fails on the first test or all tests pass. Is it DNX runner?

  10. 10 Posted by James Newton-Ki... on 05 Jan, 2016 07:44 PM

    James Newton-King's Avatar

    Yes.

  11. Support Staff 11 Posted by Feodor Fitsner on 05 Jan, 2016 08:03 PM

    Feodor Fitsner's Avatar

    OK, will give it a try. Theoretically, it should fail on a simple build with just one test only.

  12. 12 Posted by James Newton-Ki... on 08 Jan, 2016 10:39 PM

    James Newton-King's Avatar
  13. Support Staff 13 Posted by Feodor Fitsner on 08 Jan, 2016 10:59 PM

    Feodor Fitsner's Avatar

    I haven't looked into that yet. I suspect it might be something with xUnit DNX runner or the way it's called during the build. It would be very helpful to have a few very simple xUnit tests in a public repo with a build script calling DNX runner.

  14. Support Staff 14 Posted by Feodor Fitsner on 08 Jan, 2016 11:39 PM

    Feodor Fitsner's Avatar

    OK, could you please add the following script before calling your tests. It does 2 HTTP requests (similar to DNX runner ones) directly to build worker API: one adding test and the second one marking it as passed. If calling this script works while DNX runner fails then it'd help narrow down the issue.

    $body = '{
        "testName": "Newtonsoft.Json.Tests.Documentation.Samples.Xml.ConvertJsonToXml.Example123",
        "testFramework": "xUnit",
        "fileName": "Newtonsoft.Json.Tests.dll",
        "outcome": "Running",
        "DurationMilliseconds": null,
        "ErrorMessage": null,
        "ErrorStackTrace": null,
        "StdOut": null,
        "StdErr": null
    }'
     
    $headers = @{
      "Content-type" = "application/json"
    }
    Invoke-WebRequest -Uri "$($env:appveyor_api_url)api/tests" -Headers $headers -Method Post -Body $body
    
    Start-Sleep -s 1
    
    $body = '{
        "testName": "Newtonsoft.Json.Tests.Documentation.Samples.Xml.ConvertJsonToXml.Example123",
        "outcome": "Passed",
        "DurationMilliseconds": 100
    }'
    Invoke-WebRequest -Uri "$($env:appveyor_api_url)api/tests" -Headers $headers -Method Put -Body $body
    

    Alternatively, you can call this script on "Build failure" step.

  15. Support Staff 15 Posted by Feodor Fitsner on 21 Jan, 2016 07:33 PM

    Feodor Fitsner's Avatar

    Hi James,

    While working on some real-time logging improvements in Build Agent I revisited the issue with DNX runner. With updated build agent fail rate looks better now: https://ci.appveyor.com/project/FeodorFitsner/newtonsoft-json/history

    However, sometimes there is still "A task was cancelled" exception. I found the place where it comes from. It's AppVeyorClient in logging reporter: https://github.com/xunit/xunit/blob/fe8d78df7cfec55ddab93c29ea96b00...

    I think 10 seconds is too optimistic and we can try increasing it to at least 30 seconds.
    Also, I noticed it always errors in different places, different test, but after being able to send to AppVeyor one test only. There must be some sort of buffering of test results.

    I've submitted a PR for now: https://github.com/xunit/xunit/pull/738

  16. 16 Posted by James Newton-Ki... on 21 Jan, 2016 10:28 PM

    James Newton-King's Avatar

    Thanks for figuring this out.

  17. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:03 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac