Test failures on AppVeyor, not duplicated locally

roy.ivy.iii's Avatar


05 Jan, 2016 06:04 AM

I've added a PowerShell function with related tests to my repository which is being tested here (see https://ci.appveyor.com/project/rivy/scoop/build/348-dev-posh-v2). All tests (using `pester`) pass locally, but a set are failing on AppVeyor.

I've tested locally on three differing machines (Win7, Win8.1, and Win10). The Win8.1 machine has exactly the same $PSVersionTable as the AppVeyor test platform. So, I'm at a loss on how to trace down the cause of the test failure. I can see which test is failing, but the earlier tests of the same function are passing, and the failing AppVeyor test passes on all other local machines.

Do you have any suggestions on tracing the problem?


- Roy Ivy

  1. 1 Posted by roy.ivy.iii on 05 Jan, 2016 06:22 AM

    roy.ivy.iii's Avatar

    I've got the build passing by skipping the tests, but, of course, that's cheating.

  2. Support Staff 2 Posted by Feodor Fitsner on 05 Jan, 2016 07:20 PM

    Feodor Fitsner's Avatar

    I don't know the specifics of those failing tests, but can you please create a very simple example of script demonstrating the issue, preferably without pester at all.

  3. 3 Posted by roy.ivy.iii on 06 Jan, 2016 11:44 PM

    roy.ivy.iii's Avatar

    Yep, I'll set something up and reply again. It may be a couple of days.

    - Roy

  4. 4 Posted by roy.ivy.iii on 07 Jan, 2016 01:58 AM

    roy.ivy.iii's Avatar

    I had some time tonight and pared everything down.

    In branch `t-appveyor` of the repository, `.\test.ps1` passes locally on all machines (including a Win8.1 machine with a duplicate `$PSVersionTable`) but fails on AppVeyor.

    The specific test is at (https://ci.appveyor.com/project/rivy/scoop/build/374-t-appveyor).

    Does that narrow down the issue?

  5. Support Staff 5 Posted by Feodor Fitsner on 08 Jan, 2016 10:16 PM

    Feodor Fitsner's Avatar

    Yeah, that's definitely helps, thanks.

    I've tried running the build and in fact $o contains $null: https://ci.appveyor.com/project/FeodorFitsner/scoop

    Where does it get $null from?

  6. 6 Posted by roy.ivy.iii on 08 Jan, 2016 11:17 PM

    roy.ivy.iii's Avatar

    Ok, I think that narrows down the problem.

    The function loads an assembly dll (Newtonsoft.Json). The version AppVeyor shows is '', but my version locally on all of my test machines is ''. Your machine must be loading a prior version, and I need ''.

    I'll try to force a reload and see if that will fix the test issue.

  7. 7 Posted by roy.ivy.iii on 09 Jan, 2016 02:15 AM

    roy.ivy.iii's Avatar

    I'm perplexed at this point.

    I believe the problem boils down to an incorrect assembly being loaded. But I don't understand how the '' version is being loaded. If you look at https://ci.appveyor.com/project/rivy/scoop/build/392-t-appveyor , it appears that the assembly loaded from the clone is ''. But the same code, cloned locally, loads the assembly with version ''.

    Am I missing something here?

    Where would the other version be coming from?

  8. Support Staff 8 Posted by Feodor Fitsner on 11 Jan, 2016 05:08 AM

    Feodor Fitsner's Avatar

    AppVeyor build agent is currently referencing Newtonsoft.Json 7.0.1 if that helps.

  9. 9 Posted by roy.ivy.iii on 16 Jan, 2016 05:19 AM

    roy.ivy.iii's Avatar

    I've tried several different methods of loading the assembly ('-force', etc), but I keep getting a copy with version '' as shown in the test.

    I'm at a loss on how to fix this... any ideas on your end?

  10. Support Staff 10 Posted by Feodor Fitsner on 16 Jan, 2016 11:26 PM

    Feodor Fitsner's Avatar

    It loads even like that: https://github.com/FeodorFitsner/scoop/blob/t-appveyor/appveyor.yml...

    This might happen because AppVeyor Build Agent already references Newtonsoft.Json 7.0.1 and maybe import-module finds this assembly in current app domain and loads it from memory rather from disk.

  11. Support Staff 11 Posted by Feodor Fitsner on 16 Jan, 2016 11:32 PM

    Feodor Fitsner's Avatar
  12. 12 Posted by roy.ivy.iii on 19 Jan, 2016 09:10 PM

    roy.ivy.iii's Avatar

    Thank you! I wouldn't have been able to track this down without your help.

    It did take me some time to refactor the testing script to integrate the use of a powershell sub-shell for the actual testing. But, I've got it working now (see https://ci.appveyor.com/project/rivy/scoop/build/577-t).

    Thanks again for your help.

  13. Support Staff 13 Posted by Feodor Fitsner on 19 Jan, 2016 09:12 PM

    Feodor Fitsner's Avatar

    Looks great, thanks for the update!

  14. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:03 AM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts


? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac