ImportError: DLL load failed: The specified module could not be found.

Benjamin Bengort's Avatar

Benjamin Bengort

13 Feb, 2019 01:50 AM

Hello,

Recently our builds started failing on https://ci.appveyor.com/project/districtdatalabs/yellowbrick in our Miniconda3-x64 matrix (though they pass on the vanilla Python builds). I don't believe we made any changes to our codebase that might have caused the builds to fail (the last few PRs have been primarily documentation related). The builds are failing with the following traceback:

python -m nltk.downloader popular
Traceback (most recent call last):
  File "C:\Miniconda3-x64\lib\runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "C:\Miniconda3-x64\lib\runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "C:\Miniconda3-x64\lib\site-packages\nltk\__init__.py", line 152, in <module>
    from nltk.stem import *
  File "C:\Miniconda3-x64\lib\site-packages\nltk\stem\__init__.py", line 29, in <module>
    from nltk.stem.snowball import SnowballStemmer
  File "C:\Miniconda3-x64\lib\site-packages\nltk\stem\snowball.py", line 32, in <module>
    from nltk.corpus import stopwords
  File "C:\Miniconda3-x64\lib\site-packages\nltk\corpus\__init__.py", line 66, in <module>
    from nltk.corpus.reader import *
  File "C:\Miniconda3-x64\lib\site-packages\nltk\corpus\reader\__init__.py", line 105, in <module>
    from nltk.corpus.reader.panlex_lite import *
  File "C:\Miniconda3-x64\lib\site-packages\nltk\corpus\reader\panlex_lite.py", line 15, in <module>
    import sqlite3
  File "C:\Miniconda3-x64\lib\sqlite3\__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "C:\Miniconda3-x64\lib\sqlite3\dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: DLL load failed: The specified module could not be found.

This doesn't appear to be a problem with nltk, but rather with the python site-packages library sqlite3. We've googled around and found several related issues where there appears to be a stray DLL, which I believe is part of the image, and not part of our testing process:

https://github.com/ContinuumIO/anaconda-issues/issues/10034

We have tried several updates to the .appveyor.yaml file to try to fix the problem: https://github.com/DistrictDataLabs/yellowbrick/pull/743 but with no success. Any advice would be appreciated!

Best Regards,
The scikit-yb developers

  1. Support Staff 1 Posted by Owen McDonnell on 13 Feb, 2019 02:48 AM

    Owen McDonnell's Avatar

    Currently investigating a similar issue with another user.

    For now, you can switch your image to Previous Visual Studio 2015. This is a temporary workaround as we investigate the root cause of this.

  2. 2 Posted by Benjamin Bengor... on 13 Feb, 2019 02:56 AM

    Benjamin Bengort's Avatar

    Thank you for looking into it! We'll use the previous image in the meantime.

  3. 3 Posted by Jonathan on 04 Apr, 2019 01:55 PM

    Jonathan's Avatar

    I am having a similar error when trying to run a server. I'm new to Django so I was unsure if I was doing something wrong. Python Version 3.7.1 Windows 64 bit Operating System.

    (MegaWattHr) c:\Users\adroi\Desktop\MegaWattHr\MegaWattHr> python manage.py run server
    Traceback (most recent call last):
      File "manage.py", line 21, in <module>
        main()
      File "manage.py", line 17, in main
        execute_from_command_line(sys.argv)
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line
        utility.execute()
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\core\management\__init__.py", line 357, in execute
        django.setup()
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\__init__.py", line 24, in setup
        apps.populate(settings.INSTALLED_APPS)
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\apps\registry.py", line 114, in populate
        app_config.import_models()
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\apps\config.py", line 211, in import_models
        self.models_module = import_module(models_module_name)
      File "C:\Users\adroi\Envs\MegaWattHr\lib\importlib\__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 728, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\contrib\auth\models.py", line 2, in <module>
        from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\contrib\auth\base_user.py", line 47, in <module>
        class AbstractBaseUser(models.Model):
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\db\models\base.py", line 117, in __new__
        new_class.add_to_class('_meta', Options(meta, app_label))
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\db\models\base.py", line 321, in add_to_class
        value.contribute_to_class(cls, name)
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\db\models\options.py", line 204, in contribute_to_class
        self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\db\__init__.py", line 28, in __getattr__
        return getattr(connections[DEFAULT_DB_ALIAS], item)
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\db\utils.py", line 201, in __getitem__
        backend = load_backend(db['ENGINE'])
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\db\utils.py", line 110, in load_backend
        return import_module('%s.base' % backend_name)
      File "C:\Users\adroi\Envs\MegaWattHr\lib\importlib\__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "C:\Users\adroi\Envs\MegaWattHr\lib\site-packages\django\db\backends\sqlite3\base.py", line 13, in <module>
        from sqlite3 import dbapi2 as Database
      File "c:\users\adroi\anaconda3\Lib\sqlite3\__init__.py", line 23, in <module>
        from sqlite3.dbapi2 import *
      File "c:\users\adroi\anaconda3\Lib\sqlite3\dbapi2.py", line 27, in <module>
        from _sqlite3 import *
    ImportError: DLL load failed: The specified module could not be found.

    (MegaWattHr) c:\Users\adroi\Desktop\MegaWattHr\MegaWattHr>

  4. Support Staff 4 Posted by Owen McDonnell on 04 Apr, 2019 03:14 PM

    Owen McDonnell's Avatar

    Can you please link to your specific build.

  5. 5 Posted by Benjamin Bengfo... on 18 Apr, 2019 02:27 PM

    Benjamin Bengfort's Avatar

    Hello, this issue has started again, we are currently using the Previous Visual Studio 2017 image.

    The failing build is:

    https://ci.appveyor.com/project/districtdatalabs/yellowbrick/builds...

    We are working through two fix attempts:

    1. Removing the image: declaration from our .appveyor.yaml to see if the default image works.
    2. Reverting once again to the Previous Visual Studio 2015 image.
  6. 6 Posted by Benjamin Bengfo... on 18 Apr, 2019 03:25 PM

    Benjamin Bengfort's Avatar

    Unfortunately, neither of these fixes worked. We've had to comment out miniconda from our build matrix which is less than ideal. If there is another solution, we'd appreciate it because testing against miniconda is critical to our workflow.

    Failing builds:

    https://ci.appveyor.com/project/districtdatalabs/yellowbrick/builds...

    https://ci.appveyor.com/project/districtdatalabs/yellowbrick/builds...

  7. Support Staff 7 Posted by Owen McDonnell on 22 Apr, 2019 07:06 PM

    Owen McDonnell's Avatar

    Sorry for delayed response. I'm far from an expert in python development, so i haven't been able to find any obvious cause.
    Just to clarify, you have a local build using miniconda that does work?

  8. 8 Posted by Benjamin Bengfo... on 15 May, 2019 12:59 PM

    Benjamin Bengfort's Avatar

    Hi Owen, sorry it's taken a while to respond. Yes - we do have a local build of miniconda that works, and the previous images worked. We do have a PR open that we're currently reviewing that may fix the install issues but causes our tests to break. Still, this is not an issue with our code base but is rather an install issue for one of our dependencies in the setup phase of the tests. We are also looking into conda-forge to do the install, but these seem like a lot of hoops to jump through for Python testing.

  9. Support Staff 9 Posted by Owen McDonnell on 17 May, 2019 07:32 AM

    Owen McDonnell's Avatar

    I forked your repo and after many partly random/partly educated guesses i got a successful build on the current VS 2017 image.
    The relevant change is in the line,

    - "SET PATH=%PYTHON%;%PYTHON%\Scripts;%PYTHON%\Library\bin;%PATH%"
    
    in the install: stage.

    Though, I'm frankly not well versed enough in Python module machinery to understand why this is necessary and still confused as to how this is working on your local machine. It's difficult to imagine how the installation of miniconda on our build VMs could be faulty.

  10. 10 Posted by Benjamin Bengfo... on 21 May, 2019 07:36 PM

    Benjamin Bengfort's Avatar

    If it is a path issue, then I think it's more about how the image installs conda/pip -- with a local install that is handled once when you start working and then you don't think about it again. In this case, my guess is that the installer couldn't find a module because it was installed in a location on the $PATH that didn't match the modules being found? Did the path or the install location change during the image revision?

  11. Support Staff 11 Posted by Owen McDonnell on 21 May, 2019 10:21 PM

    Owen McDonnell's Avatar

    No changes to the install path of miniconda were made in either image updates.
    The version of miniconda changed on this platform update. That initially broke your build so you used previous as a temporary workaround until another platform update made the "previous" image the newer one.
    How do you know it's not the newer version of miniconda that causes this failure?

    I suggest that you RDP to the build machine to investigate, and let us know what your idea of what is wrong with the build image is.

  12. 12 Posted by Vincent Rouvrea... on 06 Jun, 2019 07:48 AM

    Vincent Rouvreau's Avatar

    Hello,

    I have a similar problem. Here is my build log: https://ci.appveyor.com/project/GUDHI/gudhi-devel/builds/25082612

    I tried:

    SET PATH=%PYTHON%;%PYTHON%\Scripts;%PYTHON%\Library\bin;%PATH%
    

    I also had to:

    SET PYTHONPATH=%PYTHON%\Lib\site-packages;%PYTHONPATH%
    
    python setup.py install
    

    is built and installed successfully (cf. log: copying build\lib.win-amd64-3.7\gudhi.cp37-win_amd64.pyd -> C:\Python37-x64\Lib\site-packages )

    DIR %PYTHON%\\Lib\\site-packages
       ...
        Directory of C:\Python37-x64\Lib\site-packages
       ...
       06/06/2019  07:39 AM         1,320,960 gudhi.cp37-win_amd64.pyd
       ...          .
    

    But import fails:

    python -c "import gudhi; print(gudhi.version)"
      ImportError: DLL load failed: The specified module could not be found.
    

    I really do not understand why and some help would be appreciate ;-)

    Vincent.

  13. 13 Posted by Vincent Rouvrea... on 12 Jun, 2019 05:37 AM

    Vincent Rouvreau's Avatar

    Hello,

    and sorry for the noise.

    The issue was coming from a dll dependency installed with vcpkg.
    I added dll path to the PATH and the module is imported correctly.

    Vincent.

  14. Support Staff 14 Posted by Owen McDonnell on 12 Jun, 2019 09:47 PM

    Owen McDonnell's Avatar

    Thanks for the update.

  15. Ilya Finkelshteyn closed this discussion on 13 Aug, 2019 09:01 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac