Comments (15)
This is output
from pyinstaller-hooks-contrib.
hmm.. the torch hook should be able to do it
from pyinstaller-hooks-contrib.
What is your build log?
from pyinstaller-hooks-contrib.
@hyunjung0208 You can add --collect-binaries torch
to PyInstaller as an temporary solution.
I believe the get_package_paths()
and add it to datas
is not a good method, as it will miss dynamic libraries's dependencies.
As shown in the warning:
WARNING: lib not found: torch_python.dll dependency of E:\development\Anaconda\envs\pyi-testenv\lib\site-packages\torch\_C.cp39-win_amd64.pyd
from pyinstaller-hooks-contrib.
Where is that torch_python.dll
file then if it's not in conda/bin
or in site-packages/torch
?
from pyinstaller-hooks-contrib.
@eric15342335
I did your recommend, and try pyinstaller 4mmdetection.spec
but it can't install and also has error.
"NameError: name 'get_package_paths' is not defined"
What else can I do to solve this problem?
Just in caseI added from PyInstaller.utils.hooks import get_package_paths to the py file, and then tried a new pyInstaller.
@hyunjung0208 You can add
--collect-binaries torch
to PyInstaller as an temporary solution.I believe the
get_package_paths()
and add it todatas
is not a good method, as it will miss dynamic libraries's dependencies.
As shown in the warning:WARNING: lib not found: torch_python.dll dependency of E:\development\Anaconda\envs\pyi-testenv\lib\site-packages\torch\_C.cp39-win_amd64.pyd
from pyinstaller-hooks-contrib.
@hyunjung0208
Can you try manually copying c:\users\kim\anaconda3\envs\mm\Library\bin\uv.dll
to your program's dist directory, i.e., dist\4mmdetection
?
from pyinstaller-hooks-contrib.
Where is that
torch_python.dll
file then if it's not inconda/bin
or insite-packages/torch
?
@bwoodsend
after pyinstaller, it is in dist/4mmdetection/torch/lib/
And
in C:\Users\kim\anaconda3\envs\mm\Lib\site-packages\torch\lib
in C:\Users\kim\anaconda3\pkgs\pytorch-1.9.0-py3.7_cuda10.2_cudnn7_0\Lib\site-packages\torch\lib\
Do I need to add torch-related path to the environmental variable?
from pyinstaller-hooks-contrib.
@hyunjung0208
Can you try manually copyingc:\users\kim\anaconda3\envs\mm\Library\bin\uv.dll
to your program's dist directory, i.e.,dist\4mmdetection
?
@rokm @
I manually copied that, and run -> ModuleNotFoundError: No module named 'mmcv._ext' [5096]Failed to execute script '4mmdetection' due to unhandled exception!
So
I revised 4mmdetection.spec hiddenimports=['mmcv._ext']
and run -> IT has same error
[ERROR]
OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\username\scar2\dist\4mmdetection(file name)\torch\lib\torch_python.dl l" or one of its dependecies.
[11176] Failed to execute script '4mmdetection' due to unhandled exception!
from pyinstaller-hooks-contrib.
Have you manually copied the uv.dll
again after revising the 4mmdetection.spec and rebuilding?
from pyinstaller-hooks-contrib.
Just in caseI added from PyInstaller.utils.hooks import get_package_paths to the py file, and then tried a new pyInstaller.
No you don't need to. I'm talking about the hook.
Hmmm, did pyinstaller --collect-binaries torch 4mmdetection.py
work? It works for me with an import torch
from pyinstaller-hooks-contrib.
Have you manually copied the
uv.dll
again after revising the 4mmdetection.spec and rebuilding?
Oh, I made a mistake there.
Copy uv.dll manually again, so it runs well. Thank you so much.
By the way, may I know how you found this error?
May I know what uv.dll is?
When I try to run this program on another computer, I'm asking this question because I think this uv.dll is causing problems.
Is there a way to copy the program file with usb and run it on another computer?
from pyinstaller-hooks-contrib.
By the way, may I know how you found this error?
OSError: [WinError 126] The specified module could not be found. Error loading "XYZ" or one of its dependecies.
usually means that there's a DLL missing. It usually helps to inspect the python extensions and other DLLs in using Dependency Walker, and see what's missing.
May I know what uv.dll is?
It's a shared library that comes from the libuv package, one of packages that pytorch
depends on.
When I try to run this program on another computer, I'm asking this question because I think this uv.dll is causing problems.
Is there a way to copy the program file with usb and run it on another computer?
If the frozen program does not work on another computer, then you could be missing some other DLLs that were present on your build computer (or in its conda environment, even).
Like @eric15342335 said, the problem is that our pytorch hook collects everything as data files, so shared libraries do not get properly analyzed for their dependencies. With pytorch installed from pip, this is not so much of a problem, because all dependencies are part of the wheel. But with anaconda installation, some of the dependencies are shared and installed outside of the package's location, and due to lack of dependency analysis, we end up missing them (this is what happens with the uv.dll
and that's why you need to copy it manually).
from pyinstaller-hooks-contrib.
Like @eric15342335 said, the problem is that our pytorch hook collects everything as data files, so shared libraries do not get properly analyzed for their dependencies. With pytorch installed from pip, this is not so much of a problem, because all dependencies are part of the wheel. But with anaconda installation, some of the dependencies are shared and installed outside of the package's location, and due to lack of dependency analysis, we end up missing them (this is what happens with the
uv.dll
and that's why you need to copy it manually).
I really appreciate your detailed explanation.
So if I install the pythorch as a pip instead of conda, I don't need to manually copy uv.dll?
When I did pip install torch, there was an error, so I did pip install torch.
Then, the torch version comes out as 1.9.0+cpu, so you can't use cuda and gpu, but I'm going to ignore it for now and try to proceed.
I installed torchvision as conda, but if I don't want to manually copy uv.dll, do I have to install it as a pip?
from pyinstaller-hooks-contrib.
I think it should work with torch
installed via pip
. But I would generally recommend against mixing conda
and pip
. I.e., stick to a pure conda environment, or install python from python.org, and use venv
and pip
.
That said, if you don't want to manually copy the uv.dll
in your conda-based installation, you could copy it from
c:\users\kim\anaconda3\envs\mm\Library\bin\uv.dll
into
c:\Users\kim\anaconda3\envs\mm\Lib\site-packages\torch\lib
and it should be collected automatically, along with other files from that directory.
Or you could collect the missing libraries as part of extra pre-processing in your .spec file (the example below is for CUDA10-enabled version):
# PyInstaller spec file
import ctypes.util
extra_torch_dlls = [
# UV
'UV.DLL',
# CUDA
'CUSPARSE64_10.DLL',
'CURAND64_10.DLL',
'CUSOLVER64_10.DLL',
'NVTOOLSEXT64_1.DLL',
'CUFFT64_10.DLL',
'CUBLAS64_10.DLL',
'CUBLASLT64_10.DLL',
'NVRTC64_102_0.DLL',
]
binaries = []
for dll in extra_torch_dlls:
dll_file = ctypes.util.find_library(dll)
if dll_file:
binaries += [(dll_file, '.')]
a = Analysis(...
binaries=binaries, # pass the extra collected binaries to PyInstaller's Analysis
...)
# the rest of the spec file...
from pyinstaller-hooks-contrib.
Related Issues (20)
- hook-rtree.py: libspatialindex is either not installed ... HOT 2
- ModuleNotFoundError: No module named 'dbus_fast._private.marshaller' HOT 1
- for python3 it is no need `prettytable` hooks HOT 2
- Missing imports during packaging with PyInstaller 5.13 and pyinstaller-hooks-contrib 2023.12 HOT 2
- ImportError: cannot import name '_pylon' from 'pypylon' after pyinstaller 6.3.0 HOT 2
- OpenCV missing data files in /cv2/data directory HOT 1
- sklearn dropped neighbors._typedef in version 1.0.2 HOT 1
- Add PyInstaller in 'I want to help' section HOT 2
- Falcon hook
- PyInstaller: Unable to find '/usr/sbin/neato' when adding binary and data files HOT 8
- hook-transformers fails to run due to partially initialized huggingface_hub HOT 4
- cryptography with OpenSSL 3.x HOT 18
- `gcloud-aio-*` (and probably also `gcloud-rest-*`) trigger `gcloud` hook, which fails because `gcloud` is not installed. HOT 1
- 2024.5 issue importing cryptography hook during build HOT 15
- pyarrow HOT 8
- Trying to import "isolated" from "PyInstaller" when pyinstaller version is lower than 5 HOT 1
- A problem converting a grib2 file to a netCDF file using cfgrib and xarray HOT 4
- pysintaller with scipy HOT 9
- hook-cryptography 2024.7 is incompatible with pyinstaller 4.5.1 HOT 1
- Create version tag after commiting to changelog HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyinstaller-hooks-contrib.