Comments (19)
relevant: I am pretty sure user scopes are going away in spack
If you have a packages.yaml config (even in ~/.spack config) why can't you hand that dir to uberenv using the existing logic?
from uberenv.
@cyrush because in his case, the package.yaml is not portable but specific to his machine.
from uberenv.
relevant: I am pretty sure user scopes are going away in spack
If you have a packages.yaml config (even in ~/.spack config) why can't you hand that dir to uberenv using the existing logic?
I'm hoping to have a sharable scripts/spack/configs/darwin/package.py
in my source repo that I use for most machines. However, I want to be able to supplement that with the paths to cmake, doxygen, and other tools that weren't built by spack and are not in standard locations. It's essentially the same use case as for supporting --upstream
, but for manually-installed software instead.
from uberenv.
Can environments help with this? or possibly handle this on the uberenv side where you combine two packages.yaml? We run into a similiar thing with devtools on axom/serac where the it would be helpful if you could have a base file that you append to in some manner.
from uberenv.
as @white238 mentioned --- I think exploring composing packages.yaml
might provide the convenience you are looking for.
Using a combo of an upstream and user config seems like a pretty fragile road to travel.
from uberenv.
this works well, so long as the needed tools are in standard locations
There is a solution now: check in per-host host spack configs -- like (esteban's machine) and revision control those.
Maybe you want to avoid duplication? If so composing packages.yaml
seems like a good thing to invest in.
from uberenv.
My initial thought was to use --upstream
for this. Indeed, the goal is to share already installed configurations. So even if it is externally installed
I thought that if it is installed in an upstream spack, then we could use it.
So I created a Spack instance and registered my external CMake install in it.
Then I declared CMake as an buildable: false
package in Serac darwin config.
Using --upstream
to point to the first Spack instance, Uberenv creates a local instance, and if I run spack find cmake
in it, it detects the external installation of CMake!
However, when I try to effectively install something with it, Spack fails to find it... This incoherence looks like a bug.
from uberenv.
The more duct tape involved, the less chance there is for a robust success :-(
from uberenv.
Well, not easy to design something both robust and flexible when the context is "a developer machine".
from uberenv.
Anyways, --upstream
behavior is not a bug. Local instance expect an external install, so it does not look for the one defined in the upstream...
from uberenv.
Like @cyrush said, being able to compose packages.yaml
files seems like a good option. That's what I was hoping to get via --config-scope
. If there's a better way, that would be great. Looking at what goes in a configuration (https://spack.readthedocs.io/en/latest/configuration.html), compilers.yaml
also looks like something I might want to stack. Really, I can see a case for all of this. It kind of goes against what uberenv is trying to accomplish, but at the same time, it helps accomplish it by letting you mostly share the configuration in a single place and just put a few machine-specific things in a different place when needed.
from uberenv.
@estebanpauli, config-scope
is a command line argument to a spack call. As a consequence, it means uberenv would have to add this anywhere relevant in the script, but more problematic, once outside uberenv using the local instance of spack without this argument would result in a different behavior, if not fail.
That's why merging the packages.yaml
is a more "uberenv-ready" solution. However, I think it would not be easy to make it robust, and it would basically consist in re-implementing the config-scopes.
from uberenv.
At first glance, merging packages.yaml
files shouldn't be that difficult -- read them in the order specified, adding the contents of each one to a dictionary, while replacing existing keys. If you limit the merging to the top level, it should be easy. If you want to merge at lower levels, then I agree that it becomes more complicated. However, I would think that as a user, you would just want the merge to happen at the top level. That should handle the most common use cases of overriding the contents of the default packages.py
. Anything beyond that would be harder to reason about and remember the rules for as a user.
from uberenv.
@estebanpauli,
config-scope
is a command line argument to a spack call. As a consequence, it means uberenv would have to add this anywhere relevant in the script, but more problematic, once outside uberenv using the local instance of spack without this argument would result in a different behavior, if not fail.
I hadn't thought about the user case of making later calls to spack. That would present a serious complication.
from uberenv.
Maybe we could simply allow not to remove the user scope of Spack. E.g. an --unsafe
option. I know it does not solve @white238 use case.
from uberenv.
Since we are patching Spack, we could as well patch it to point the user config to user-defined-scope
instead of ~/.spack
.
from uberenv.
OK, maybe add this to Uberenv:
spack --config-scope <my-config-scope> config get <section>
That would flush the merged configuration, and then we can use the result as the only configuration in the local spack. @white238 what to you think?
from uberenv.
I was wondering if were could leverage Spack's environment files ability to include files, variables, and the "when" logic to solve this.
For a completely untested and contrived example:
definitions:
- cmake_path: /path/to/specific/cmake
when: env.get("MACHINE", "") == "estebansMachine"
- cmake_path: cmake
packages:
cmake:
buildable: false
path: $cmake_path
from uberenv.
@white238 -- if folks can pave the way, that sounds awesome
from uberenv.
Related Issues (20)
- Unify git commands
- Create new function that unifies logic of picking options from various places
- self.spec_hash is not set consistently HOT 1
- logic error in spack platform detection logic
- Add documentation about using uberenv's copy of spack
- Make package activation more specific HOT 1
- add spack --keep-stage option HOT 4
- new spack install of clingo may cause error
- Improve handling of mirroring (caching) against branches
- a few items related to envs and spack 17 support
- newer versions of spack breaks uberenv disabling things in spack/lib/spack/spack/config.py HOT 2
- Warn when spack is too recent to use original concretizer
- change from spack_commit to spack_branch ignored
- Add CI tests for the remaining two build modes
- Spack `activate` is going away in 0.19.0
- env path foward? spack install with yaml file on command line HOT 4
- Spack now defaults to `--reuse` we need to add `--fresh` HOT 2
- add spack debug option HOT 2
- Move to Spack Environments HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from uberenv.