Unit testing passes differently locally than on appveyor.com

msdobrescu's Avatar

msdobrescu

20 Feb, 2019 08:00 AM

Hello, I try to update UTF-unknown project on github. This is a code page detection tool. I've created a fork with a branch here: https://github.com/msdobrescu/UTF-unknown/tree/Adding_flexibility_for_sample_size.
There is an NUnit testsing project there. For a file that is in Greek, there are possible two encodings, one windows-1253, one is ISO-8859-7. Locally, the file is detected as ISO-8859-7, so the windows-1253 test fails. On appveyor, it is reversed, meaning it is recognized as windoes-1253, so the ISO-8859-7 test fails. I expect one of these to fail, but to fail the same locally and on appveyor.
The testing is done by listing the dirs having encoding names (all are lowercase) and their files inside should have the expected encoding declared by the containing directory name. The first encoding met is considered the right one. All are tested in the same order. The first bigger probability is taken. The check is done by "<" operator.
In this specific case, the Greek encoding ISO-8859-7 comes first for all cases, so it should be chosen. As note, for that file, both encoding meet the same value, but using strict operator (not "or equal" one) only one encoding should be chosen because they are also tested in the same order, always.
I have taken into account the line endings too, in appveyor config.

My question is: why the tests are different locally than on appveyor?

  1. Support Staff 1 Posted by Ilya Finkelshte... on 20 Feb, 2019 05:57 PM

    Ilya Finkelshteyn's Avatar

    I am not sure, but this can be related to the line ending (\n vs \r\n). Try add this at init stage (which happens before clone):

    init:
      - git config --global core.autocrlf true
    

    Let us know if this helps.

  2. 2 Posted by msdobrescu on 20 Feb, 2019 06:00 PM

    msdobrescu's Avatar

    Hi, that's already in place. I've tried that to no change.

  3. Support Staff 3 Posted by Ilya Finkelshte... on 20 Feb, 2019 10:21 PM

    Ilya Finkelshteyn's Avatar

    Sorry, I did not notice if you mentioned that already. I would check how it behave with some another clean VM. For example create new Windows VM on Azure or AWS, clone your repo, install .NET Core runtime and SDK and run your tests.

    My feeling is that this can be related to the language settings on you computer, but I am not sure.

  4. 4 Posted by msdobrescu on 20 Feb, 2019 11:43 PM

    msdobrescu's Avatar

    We have already tried on several machines, only appveyor is anomalous so
    far.
    Actually, it does not reflect the code logic at all.

  5. 5 Posted by msdobrescu on 21 Feb, 2019 01:07 AM

    msdobrescu's Avatar

    Well, I have added a debug string to see the order of decision for the best matching charset. From that moment it worked as expected. I have absolutely no clue why.

  6. Support Staff 6 Posted by Ilya Finkelshte... on 21 Feb, 2019 02:31 AM

    Ilya Finkelshteyn's Avatar

    This feels like a race condition, which is being exposed on AppVeyor from time to time (because AppVeyor VMs are much less beefier than dev ones -- 2 Cores).

  7. 7 Posted by msdobrescu on 21 Feb, 2019 06:41 AM

    msdobrescu's Avatar

    This was very deterministic, happened each time for a week.
    I'd suspect some aggressive caching or patch applied to appveyor.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac