Comments (15)
TL;DR: it's definitely possible, but it sucks and would need a bunch of integration work + testing
I played with this a bunch on Linux today, here's what I've found so far:
- I think this requires you to use a shared library version of opencv, because it doesn't seem like you can link directly against the statically linked cv2 python module... or at least I haven't found the right combination of flags to allow you to do so
- Assuming they all end up in site-packages in the cv2 directory, the shared libraries need to be able to find each other, so you have to set the rpath (can use a cmake arg:
CMAKE_INSTALL_RPATH=$ORIGIN
) - Special magic required for a python module that tries to link to this stuff
- Needs to find the special include file for the API (presumably in site-packages in the cv2 directory)
- Needs to find the libraries on site-packages, and user needs to know which libraries they want to link to
- Need to set the RPATH on the user's custom python module to
$ORIGIN/../cv2
-- though, this magic varies on where your compiled module ends up, and it appears that different install mechanisms may put your binary in different places (setuptools vs distutils)
- I created a patch for OpenCV to export the numpy <-> cv2 array conversion routines, it's not so bad and I'll approach upstream about it even if this isn't a successful effort.
It's really surprising to me that nobody has tried to do this sort of thing before, but it seems that people don't mix OpenCV python and OpenCV c++, except as a custom exe. I'm undecided if I'm giving up yet, but I welcome any input from others who may have ideas.
from opencv-python.
Looks like someone already filed a bug for creating a capsule: opencv/opencv#8872
from opencv-python.
This project ships cv2.so
/.pyd
exactly as OpenCV build script builds it -- statically linked. There's no way to change this via build options AFAICS, so you should go to the OpenCV project anyway if you wish to change that.
There are other package managers that work in Windows and can ship C libraries separately -- e.g. Chocolatey and Anaconda -- and already have packages for Python and OpenCV. To achieve that, they are forced to put additional restrictions to avoid dependency hell that Python's ecosystem doesn't -- like same compilers and compiler options for everyone, only a single instance of a library per environment and/or strict file naming and placements conventions and little interoperability with other software installed on the system.
If that doesn't bother you, you can make your product as a package for one of those package managers.
from opencv-python.
Just tried it on my mac:
[100%] Linking CXX shared module ../../lib/python3/cv2.cpython-36m-darwin.so
[100%] Built target opencv_python3
user@host ~/dev/ext/opencv/build
$ otool -L lib/python3/cv2.cpython-36m-darwin.so
lib/python3/cv2.cpython-36m-darwin.so:
@rpath/libopencv_ml.3.4.dylib (compatibility version 3.4.0, current version 3.4.4)
@rpath/libopencv_dnn.3.4.dylib (compatibility version 3.4.0, current version 3.4.4)
@rpath/libopencv_shape.3.4.dylib (compatibility version 3.4.0, current version 3.4.4)
[snip]
from opencv-python.
Thanks! I started this project because I wanted to make the installation as easy as possible and I'm happy to hear that I have reached that target.
I haven't tried that, it might be possible. Trying it shouldn't break anything so you may go ahead :)
from opencv-python.
What about passing --allow-shlib-undefined
to the linker?
from opencv-python.
It's really surprising to me that nobody has tried to do this sort of thing before
There's no gain here, and you're tryng to use a Python module for something that it's not designed to -- that's why.
cv2.so
/.pyd
is a Python library and is supposed to be used via Python interface. Python's packaging system is intentionally not designed to ship C libraries.
- If you're writing a Python extension, use it through Python's C interface as a Python module should.
- If you wish to use OpenCV's C++ interface, use your own copy of OpenCV. There's absolutely no difference functionality-wise -- you'll get all the same functions.
- The only functional difference would be if you wish to access the same OpenCV object instances both as wrapped into Python objects and as underlying C++ objects. But that will require a layer of middleware in
cv2
to enforce correct shared access due to Python and C++ having different memory models, different object lifecycle management, locking etc -- all for questionable gain.- The only package I know of that exposes both Python and C interface is
numpy
. I'm sure that it does have such middleware layer so this is where you should look if you wish to pursue this further for whatever reason.
- The only package I know of that exposes both Python and C interface is
- The only functional difference would be if you wish to access the same OpenCV object instances both as wrapped into Python objects and as underlying C++ objects. But that will require a layer of middleware in
from opencv-python.
@native-api I would like to use a library that is written using OpenCV/C++ from python. Not duplicating a ton of carefully written C++ code seems like a huge gain to me.
I agree that python's packaging system isn't designed to do this, and that accomplishing my goal using opencv-python would be tricky (and perhaps not possible).
My target audience are high school students primarily using Windows. If I could ship a wheel that bundled/wrapped my OpenCV/C++ dependency that depended on opencv-python, that would be fantastic and significantly lower the bar for users (particularly on Windows) to use such a library. The dependency I'm using is getting Windows support this year, so I'll probably dig into this yet again in the coming months.
from opencv-python.
Yes, I'm aware of all of that. I'd rather use stock python + wheels, I think it's a much simpler proposition from a novice user perspective (though certainly more difficult from a packager perspective).
It is possible to build cv2.so/pyd to use shared libraries (perhaps you didn't read above?). And yes, as also mentioned above, it is likely that there are some things that would need to be patched upstream to make this work.
Your input is welcome if you have other ideas on how one might be able to get this to work.
from opencv-python.
It is possible to build cv2.so/pyd to use shared libraries (perhaps you didn't read above?)
There's no way to change this via build options AFAICS, so you should go to the OpenCV project anyway if you wish to change that.
from opencv-python.
Historically cmake -DBUILD_SHARED_LIBS=ON ...
has built a cv2 python module that utilizes the libopencv*.so/dll libraries.
from opencv-python.
Haven't done anything in particular towards this, but recently we've done something similar that is relevant.
Our new robot libraries now wrap existing C++ libraries, consisting of lots of different .so/.dll/.dylib files. Our new build system (https://github.com/robotpy/robotpy-build) generates python wrappers for the C++ libraries, embeds the shared library version of the C++ library in the python package, adds include files, and then autogens a snippet that looks like this:
from ctypes import cdll
_lib = cdll.LoadLibrary(join(_root, "lib", "libwpilibc.so"))
As long as that snippet is imported before the python bindings are imported, the library symbols are resolved and everything works great. We even are shipping wheels that depend on libraries shipped in other wheels. The only exception to this is on OSX, where we have to fix up the rpath to include a site-packages relative @loader_path
section.
This technique could be adapted for opencv-python instead of using statically linked libraries. However, to be truly useful, would really want opencv/opencv#8872 to be resolved (as mentioned above).
from opencv-python.
Hi, I found this issue because I am interested in the same thing, it seems silly that you would include OpenCV two times. From my experience, PyTorch is doing something similar, there is an extension that links the core shared libraries, bundled in the Python package. The package also includes C++ headers and some python functions to locate them. You can then compile a PyTorch C++ extension and link it with torch C++ libraries. In Python code you have to first load torch Python C++ module that loads shared libraries and you can also load your extension. Would something like that work for OpenCV as well?
from opencv-python.
I also really want to get this to work, I have a Python project that uses Python opencv but also has Python binds for a C++ library that links with opencv.
Are there really significant headaches other than _GLIBCXX_USE_CXX11_ABI to deal with in terms of compiler support for producing these files?
Is there any way right now to have my setup work without requiring opencv compilation be a part of project setup (extends setup by several hours with cuda support on low end devices)?
from opencv-python.
FWIW, I have some (arm only, for my specific platform) wheels that allow this sort of thing at https://www.tortall.net/~robotpy/wheels/2022/roborio/ (see robotpy-opencv, robotpy-opencv-core, robotpy-opencv-core-dev, which are all used by robotpy-cscore). They are built at https://github.com/robotpy/roborio-opencv via github actions.
The hard part, as stated above, is getting the library and include paths correct. I did it in setup.py (https://github.com/robotpy/robotpy-cscore/blob/7afe6afe5607c27fc1a563c44e0b25278b39b2d7/setup.py#L150) for this specific usage. However, a general purpose build tool would be better at setting all of this up (and in fact is what my robotpy-build tool does for my other packages).
So, this is definitely possible. Someone just needs time to do it -- though a deeper integration of cmake or something with python packages might be the right place to do all of this. Either way, I don't have that time.
from opencv-python.
Related Issues (20)
- PyPI: macOS arm64 wheel missing
- cv2.createThinPlateSplineShapeTransformer not exist in 4.10
- imdecode jpeg2000 segfault or deadlock in multithreading HOT 2
- Typing of compare does not allow scalar for src1 or src2
- Why Building wheel for opencv-python when using "pip install opencv-python"
- Typing stubs generation has failed when building wheel HOT 1
- error: OpenCV(4.9.0) D:\a\opencv-python\opencv-python\opencv\modules\dnn\src\layers\convolution_layer.cpp:396: error: (-2:Unspecified error) Number of input channels should be multiple of 3 but got 4 in function 'cv::dnn::ConvolutionLayerImpl::getMemoryShapes'
- Gstreamer support on windows 10
- why ı am getting this error
- BUG: cv2.fillPoly() causing segmentation fault on Ubuntu 22.04.4 LTS HOT 1
- [Suggestion] Metapackage as solution to current issues with opencv as dependency HOT 1
- Build from source python 3.12 fails HOT 6
- error: OpenCV(4.8.0) /io/opencv/modules/core/src/copy.cpp:71: error: (-215:Assertion failed) cn <= 4 in function 'scalarToRawData'
- ModuleNotFoundError: No module named 'distutils' in latest release HOT 7
- MacOS arm64 wheels missing from PyPI 4.10 release HOT 2
- ImportError: numpy.core.multiarray --- Numpy 2.0.0 HOT 1
- cv2/cv2.abi3.so, 2): Library not loaded: /usr/local/opt/libunistring/lib/libunistring.2.dylib Referenced from: /usr/local/opt/gnutls/lib/libgnutls.30.dylib HOT 1
- When will opencv-python support numpy 2.0? HOT 6
- compile bug
- AttributeError: module 'numpy' has no attribute 'object'. `np.object` was a deprecated alias for the builtin `object`. To avoid this error in existing code, use `object` by itself. Doing this will not modify any behavior and is safe. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opencv-python.