Comments (5)
Ok I see, thanks for the additional details. The main problem to me does not seem the lack of cmake_build_extension
in conda, rather the usage of -e
in your requirements.txt
. In fact, editable installation do not pull any dependency of the local submodule even if they are listed either in your setup.py
or your setup.cfg
.
Without changing your pattern, what I would suggest is to use the YAML file to prepare the environment in which you then install your module in editable mode. This means:
- Removing the
-e lib/my_submodule
fromrequirements.txt
- Add
cmake-build-extension
torequirements.txt
- After creating the conda environment, run
pip install -e .
I'm not a big fan of requirements.txt
and I never used it personally (I believe that a well structured setup.cfg
is a better pattern). I'm not sure if the order of packages is kept. If yes, alternatively to what I proposed above, you maybe could update it to something similar:
cmake-build-extension
# [...] other packages that may or may not require cmake-build-extension
-e lib/submodule
from cmake-build-extension.
Hi @alexus37, thanks for opening this issue! In the past few months in the organization I'm working we started being more and more interested in conda and conda-forge, and we started (mainly @traversaro) to package our software for conda usage. My knowledge on packaging is still limited, but I'm getting acquainted with the process and in many cases is quite straightforward, especially for pure-Python projects like this one.
Though, for this particular case, I'm a bit skeptical. The aim of this setuptools extension is smoothing the experience of hybrid C++ / Python projects of integrating them with tools that comply with PEP517 and PEP518 (like pip
and pypa/build
). The package generated by these tools are installed in the active site-packages
folder (that could be system, virtual environment, or also a conda environment), but the exported CMake project is bundled in the same site-packages/<package_name>
folder. In order for another package to import the CMake project and consume the targets, either it has to use similarly cmake-build-extension
with the cmake_depends_on
option, or the user has to manually add an entry to CMAKE_PREFIX_PATH=[...]:<path/to/site-packages>/package_name
.
Conda, AFAIK, has a different approach. The installation prefix of hybrid C++ / Python projects is different for the two languages. C++ resources are installed in $CONDA_PREFIX/lib
where all the other environment libraries are installed, and CMake's find_package
resources are active by default. Python resources, instead, go similarly in $CONDA_PREFIX/lib/pythonX.Y/site-packages
.
This being said, it seems to me that using cmake-build-extension
for a conda package kind of defeats the purpose of conda itself. Maybe I'm not expert enough with the conda ecosystem (I'm definitely not), but can you elaborate a bit more on your use case? My fear is to trigger a pattern that, in my limited knowledge, does not really comply well with the approach that should be used. Maybe here an opinion of @traversaro could be enlightening.
I'm not against a priori to bring cmake-build-extensions to conda-forge, but since then I'd also have to maintain it (regardless of the nice automation they have in place), I'd like first to understand if there's actually a reason for that effort. For instance, we do maintain hybrid projects both in PyPI and conda-forge (or custom conda channel), and we have the need of cmake-build-extension
only in the former case. But again, maybe it's my knowledge that is still too limited. This discussion is definitely interesting, and I'd like to hear your opinion.
from cmake-build-extension.
Hi,
thanks for the elaborate answer. First I want to state that I am not a packaging expert and I decided to fork the lib and modify it to my needs. However, my initial plan was the following:
- I am using conda to create an env for my project. To do this I created a YAML file
name: myEnv
channels:
- conda-forge
dependencies:
- python=3.7
- pip
- pybind11
- cmake_build_extension
- pip:
- -r requirements.txt
- This installs all dependencies and runs pip afterwards. In the requirements.txt I have a local submodule that would require cmake_build_extension and is installed as a local link.
-e lib/my_submodule
Since my my_submodule
requires cmake_build_extension to be installed I thought it makes sense to install it via conda forge first.
conda env create -f conda_env.yaml
However, there might be better ways to achieve this.
Feel free to close the issue
from cmake-build-extension.
What discussed above remains valid, though I might have found an interesting use case to have cmake_build_extension
in conda-forge #25. Therefore, I reopen the issue.
Edit: also, packages that use our context manager to fix Windows imports must have cmake-build-extension
installed in their system. Therefore, it's no longer a build-only requirement (that could be handled directly by pip
during conda packaging) but it's a runtime dependency. In this case, having it in conda-forge would be necessary.
from cmake-build-extension.
Forgot to comment here. After conda-forge/staged-recipes#16572, cmake_build_extension
is available in conda-forge 🚀
Check the resulting feedstock for more details: https://github.com/conda-forge/cmake-build-extension-feedstock/
from cmake-build-extension.
Related Issues (20)
- Expose binaries of the CMake project
- Extend the example with also pybind11 bindings HOT 1
- Use gitignore when packaging with custom sdist command
- Multiple calls operating on the same build folder leak resources from isolated environment HOT 8
- Allow disabling `CMakeExtension`s through environment variable
- Using cmake extension modules alongside other extension modules HOT 5
- linking against imported library HOT 4
- Adjust supported Python versions HOT 1
- Python 3.7 on Windows fails in CI
- Downstream packages inherit setuptools_scm behavior HOT 2
- sdist doesn't package source_dir HOT 7
- write __init__.py into a wrong folder
- Should always check ninja in run() of BuildExtension? HOT 2
- Would SWIG example `normalize_numpy` leak memory? HOT 2
- Evaluate if recommend `~=` usage for dependencies versions HOT 1
- publish new release HOT 1
- Support building w/o Ninja for cross-compile targets
- Update `README.md` to use the more common `src` code folder HOT 3
- Example cannot be imported in Windows if python >= 3.8 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cmake-build-extension.