Conda/Python build error

Seb's Avatar

Seb

03 Nov, 2017 05:02 PM

My build on appveyor has started to fail today, seemingly without any related changes

This is the error:

Solving package specifications:
Error: Could not find some dependencies for numpy 1.13.1*: vc 9.*, vc 10.*, vc 14.*
Command exited with code 1

For reference my conda environment yml file starts like this:

name: XXX
dependencies:
- numpy=1.13.1

I have tried re-running an old commit that worked and I get the same error

  1. Support Staff 1 Posted by Ilya Finkelshte... on 03 Nov, 2017 05:33 PM

    Ilya Finkelshteyn's Avatar

    Could you please send us a links to last good and first failed builds?

  2. 2 Posted by Sebastian Byas on 06 Nov, 2017 08:40 AM

    Sebastian Byas's Avatar

    Last good build:
    https://ci.appveyor.com/project/RESSoftwareTeam/resolve-performance-monitoring/build/1.0.151

    First failed build for this issue:
    https://ci.appveyor.com/project/RESSoftwareTeam/resolve-performance-monitoring/build/1.0.152

    Kind Regards,

    Sebastian Byas
    Data Scientist

    [email blocked]<mailto:[email blocked]> | www.res-group.com<http://www.res-group.com/>

    [cid:[email blocked]]<http://www.res-group.com/> [cid:[email blocked]] <https://www.linkedin.com/company/res-group> [cid:[email blocked]] <https://twitter.com/RESGroup> [cid:[email blocked]] <https://www.youtube.com/c/RESGroupVideo>

    Home Safe - Every Day

    Renewable Energy Systems Limited, registered in England and Wales with Company Number 1589961
    Registered Office: Beaufort Court, Egg Farm Lane, Kings Langley, Hertfordshire WD4 8LR

  3. 3 Posted by Sebastian Byas on 06 Nov, 2017 11:13 AM

    Sebastian Byas's Avatar

    Hi,
    This has happened to another of our repositories with the same error – previously working commits now fail when rebuilding – this is very disruptive to our development.
    Regards,
    Sebastian Byas
    Data Scientist

    [email blocked]<mailto:[email blocked]> | www.res-group.com<http://www.res-group.com/>

    [cid:[email blocked]]<http://www.res-group.com/> [cid:[email blocked]] <https://www.linkedin.com/company/res-group> [cid:[email blocked]] <https://twitter.com/RESGroup> [cid:[email blocked]] <https://www.youtube.com/c/RESGroupVideo>

    Home Safe - Every Day

    Renewable Energy Systems Limited, registered in England and Wales with Company Number 1589961
    Registered Office: Beaufort Court, Egg Farm Lane, Kings Langley, Hertfordshire WD4 8LR

  4. Support Staff 4 Posted by Ilya Finkelshte... on 06 Nov, 2017 08:21 PM

    Ilya Finkelshteyn's Avatar

    Hi Sebastian,

    We compared everything in good and bad build and could not find what difference could cause this. Build worker VM images was updated earlier (Oct 22) and build worked after that.

    I see that you already rebuilt last good commit without a success so it is very unlikely related to your changes.

    Does conda env create -f ResolvePerformanceMonitoring.yml works locally for you?

    Ilya.

  5. 5 Posted by Sebastian Byas on 07 Nov, 2017 09:27 AM

    Sebastian Byas's Avatar

    Yes, conda env create -f ResolvePerformanceMonitoring.yml works fine for us locally
    Sebastian Byas
    Data Scientist

    [email blocked]<mailto:[email blocked]> | www.res-group.com<http://www.res-group.com/>

    [cid:[email blocked]]<http://www.res-group.com/> [cid:[email blocked]] <https://www.linkedin.com/company/res-group> [cid:[email blocked]] <https://twitter.com/RESGroup> [cid:[email blocked]] <https://www.youtube.com/c/RESGroupVideo>

    Home Safe - Every Day

    Renewable Energy Systems Limited, registered in England and Wales with Company Number 1589961
    Registered Office: Beaufort Court, Egg Farm Lane, Kings Langley, Hertfordshire WD4 8LR

  6. Support Staff 6 Posted by Ilya Finkelshte... on 07 Nov, 2017 11:51 PM

    Ilya Finkelshteyn's Avatar

    To rule out AppVeyor build worker image update, please try to use Previous Visual Studio 2015 build worker image. You can set it with image: Previous Visual Studio 2015 in YAML or in the Environment tab if you use UI. I did not ask for that earlier as I see that it worked after Oct 22, but let us please double-check.

  7. 7 Posted by Lee Cameron on 08 Nov, 2017 09:23 AM

    Lee Cameron's Avatar

    Hi Ilya,

    I tried this with the resEnergyStorage project which is suffering the same problem and it did not work. We receive the same error:


    Error: Could not find some dependencies for numpy 1.11.2*: vc 9.*, vc 10.*, vc 14.*

    Have you confirmed that “conda –env create –f SomeOtherEnvironmentFile.yml” works on your build servers? In both occasions the error we see is related to numpy, but I am wondering if “conda –env create –f” works at all on your side.

    Thanks,

    Lee Cameron
    Senior Data Scientist & Storage Specialist

    D +44 1923 299 331
    [email blocked]<mailto:[email blocked]> | www.res-group.com<http://www.res-group.com/>

    [cid:[email blocked]]<http://www.res-group.com/> [cid:[email blocked]] <https://www.linkedin.com/company/res-group> [cid:[email blocked]] <https://twitter.com/RESGroup> [cid:[email blocked]] <https://www.youtube.com/c/RESGroupVideo>

    Home Safe - Every Day

    Renewable Energy Systems Limited, registered in England and Wales with Company Number 1589961
    Registered Office: Beaufort Court, Egg Farm Lane, Kings Langley, Hertfordshire WD4 8LR

  8. Support Staff 8 Posted by Ilya Finkelshte... on 09 Nov, 2017 03:43 AM

    Ilya Finkelshteyn's Avatar

    Lee,

    I made a search for appveyor.yml conda env create and got impressive number of results. Checked commits statuses of one of the repositories and see it worked recently.

    I believe that if conda –env create –f do not work generally on AppVeyor we would get at least few more complains already. I have strong feeling that this is related to some external dependency.

    As I already said, to rule out AppVeyor update, please try to use "Previous" build worker image. Also I would recommend to remove dependencies one-by-one from ResolvePerformanceMonitoring.yml to find guilty one. And finally try to run conda env create -f ResolvePerformanceMonitoring.yml on clean machine, instead of development workstation, which can have some dependencies already installed.

  9. 9 Posted by Sebastian Byas on 09 Nov, 2017 08:58 AM

    Sebastian Byas's Avatar

    Hi Ilya,

    We have used the ‘previous’ build worker image with no success.

    What troubles me is that these builds used to work – re-building old successful commits now fails which implies that something has changed within AppVeyor to cause a different result.

    In my repository ResolvePerformanceMonitoring.yml has not changed in 2 months so I feel like dependency changes are very unlikely to be an issue and as this has built successfully on AppVeyor many times since the last change then it implies the conda env Create command works on a clean machine.

    Even if we found an issue with a dependency after recent AppVeyor changes we would be reluctant to compromise our code for issues in the CI software

    Regards,

    Sebastian Byas
    Data Scientist

    [email blocked]<mailto:[email blocked]> | www.res-group.com<http://www.res-group.com/>

    [cid:[email blocked]]<http://www.res-group.com/> [cid:[email blocked]] <https://www.linkedin.com/company/res-group> [cid:[email blocked]] <https://twitter.com/RESGroup> [cid:[email blocked]] <https://www.youtube.com/c/RESGroupVideo>

    Home Safe - Every Day

    Renewable Energy Systems Limited, registered in England and Wales with Company Number 1589961
    Registered Office: Beaufort Court, Egg Farm Lane, Kings Langley, Hertfordshire WD4 8LR

  10. 10 Posted by Sebastian Byas on 09 Nov, 2017 09:29 AM

    Sebastian Byas's Avatar

    Hi Ilya,

    I have managed to get a successful build on AppVeyor by modifying my install section of the appveyor.yml to this:

    install:
      - set PATH=C:\Miniconda3;C:\Miniconda3\Scripts;%PATH%
      - cmd: conda config --set always_yes yes --set changeps1 no
      - cmd: conda update -q conda
      - cmd: conda info -a
      - cmd: conda env create -f ResolvePerformanceMonitoring.yml

    I have no idea what the conda config change command does or is needed for but it was in a project you sent a link to – something must have changed for these lines to now be required.

    Do you publicise or notify users of breaking changes on new releases or updates to AppVeyor – because we really need to know if builds are going to start failing because of AppVeyor changes!

    Kind Regards,

    Sebastian Byas
    Data Scientist

    [email blocked]<mailto:[email blocked]> | www.res-group.com<http://www.res-group.com/>

    [cid:[email blocked]]<http://www.res-group.com/> [cid:[email blocked]] <https://www.linkedin.com/company/res-group> [cid:[email blocked]] <https://twitter.com/RESGroup> [cid:[email blocked]] <https://www.youtube.com/c/RESGroupVideo>

    Home Safe - Every Day

    Renewable Energy Systems Limited, registered in England and Wales with Company Number 1589961
    Registered Office: Beaufort Court, Egg Farm Lane, Kings Langley, Hertfordshire WD4 8LR

  11. 11 Posted by Lee Cameron on 09 Nov, 2017 12:18 PM

    Lee Cameron's Avatar

    Hi Ilya,

    Just to confirm that the solution also worked on our other project that was experiencing the same issues. Thanks for your help on this.

    As Eb has mentioned, we rely on AppVeyor for CI with these Python projects and as such would appreciate notifications regarding how changes of the AppVeyor build environment can affect use of conda. If testing “conda env” is not already part of your testing process before updating the AppVeyor build server environment we would suggest it should be. As your search of GitHub shows there is a lot of projects relying on this and I think many of them would also experience the same issue as us using the new AppVeyor environment.

    Thanks.

    Lee Cameron
    Senior Data Scientist & Storage Specialist

    D +44 1923 299 331
    [email blocked]<mailto:[email blocked]> | www.res-group.com<http://www.res-group.com/>

    [cid:[email blocked]]<http://www.res-group.com/> [cid:[email blocked]] <https://www.linkedin.com/company/res-group> [cid:[email blocked]] <https://twitter.com/RESGroup> [cid:[email blocked]] <https://www.youtube.com/c/RESGroupVideo>

    Home Safe - Every Day

    Renewable Energy Systems Limited, registered in England and Wales with Company Number 1589961
    Registered Office: Beaufort Court, Egg Farm Lane, Kings Langley, Hertfordshire WD4 8LR

    From: Sebastian Byas
    Sent: 09 November 2017 09:29
    To: Ilya Finkelshteyn
    Cc: Lee Cameron
    Subject: RE: Conda/Python build error [Problems #9528]

    Hi Ilya,

    I have managed to get a successful build on AppVeyor by modifying my install section of the appveyor.yml to this:

    install:
      - set PATH=C:\Miniconda3;C:\Miniconda3\Scripts;%PATH%
      - cmd: conda config --set always_yes yes --set changeps1 no
      - cmd: conda update -q conda
      - cmd: conda info -a
      - cmd: conda env create -f ResolvePerformanceMonitoring.yml

    I have no idea what the conda config change command does or is needed for but it was in a project you sent a link to – something must have changed for these lines to now be required.

    Do you publicise or notify users of breaking changes on new releases or updates to AppVeyor – because we really need to know if builds are going to start failing because of AppVeyor changes!

    Kind Regards,

    Sebastian Byas
    Data Scientist

    [email blocked]<mailto:[email blocked]> | www.res-group.com<http://www.res-group.com/>

    [cid:[email blocked]]<http://www.res-group.com/> [cid:[email blocked]] <https://www.linkedin.com/company/res-group> [cid:[email blocked]] <https://twitter.com/RESGroup> [cid:[email blocked]] <https://www.youtube.com/c/RESGroupVideo>

    Home Safe - Every Day

    Renewable Energy Systems Limited, registered in England and Wales with Company Number 1589961
    Registered Office: Beaufort Court, Egg Farm Lane, Kings Langley, Hertfordshire WD4 8LR

  12. Support Staff 12 Posted by Feodor Fitsner on 09 Nov, 2017 05:52 PM

    Feodor Fitsner's Avatar

    Thanks for the update! I'm glad your builds are fixed now.

  13. Support Staff 13 Posted by Ilya Finkelshte... on 09 Nov, 2017 07:39 PM

    Ilya Finkelshteyn's Avatar

    Hi Lee and Sebastian,

    We take it seriously that you rely on us. Here is what we do to prevent breaking changes:

    • After any build worker image update we provide you with Previous build worker image

    • Do not roll next build worker image update until all blocking issues fixed. If customer cannot build because of change in the image, it is blocking issue.

    • We have number of ways to deploy small hotfixes quickly between build worker image updates.

    • We collect different build scenarios as integration tests and run them before updates. If you provide us with the open source project to verify your scenario, we would be happy to add it to that tests.

    • Also follow @appveyor to be aware of what happens.

    Ilya.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac