Comments (5)
certik
commented on May 30, 2024
Using the new hash in mystack.py gives:
ondrej@hawk:~/repos/hashdist(master)$ python mystack.py target
Status:
profile/ufwe.. [needs build]
hdf5/jbko.. [needs build]
virtual:gcc-stack/host (=gcc-stack/vi4w..) [ok]
virtual:hdist-cli/r0 (=hdist-cli/o5au..) [ok]
szip/m4pr.. [needs build]
virtual:unix/host (=unix/djnp..) [ok]
virtual:hdist-cli/r0 (=hdist-cli/o5au..) (see above)
virtual:gcc-stack/host (=gcc-stack/vi4w..) (see above)
zlib/y4jn.. [needs build]
virtual:gcc-stack/host (=gcc-stack/vi4w..),virtual:unix/host (=unix/djnp..) (see above)
virtual:unix/host (=unix/djnp..) (see above)
szip/m4pr..,zlib/y4jn.. (see above)
Build needed
[zlib] Building y4jn.., follow log with:
[zlib] tail -f /home/ondrej/.hdist/bld/zlib-1.2.6-y4jn/build.log
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
[zlib ERROR] hdist command failed; raising
Traceback (most recent call last):
File "mystack.py", line 25, in <module>
hr.cli.stack_script_cli(profile)
File "/home/ondrej/repos/hashdist/hashdist/recipes/cli.py", line 61, in stack_script_cli
build_recipes(build_store, source_cache, config, [root_recipe], keep_build=args.keep)
File "/home/ondrej/repos/hashdist/hashdist/recipes/recipes.py", line 302, in build_recipes
_depth_first_build(recipe)
File "/home/ondrej/repos/hashdist/hashdist/recipes/recipes.py", line 286, in _depth_first_build
_depth_first_build(dep_pkg)
File "/home/ondrej/repos/hashdist/hashdist/recipes/recipes.py", line 294, in _depth_first_build
**kw)
File "/home/ondrej/repos/hashdist/hashdist/core/build_store.py", line 414, in ensure_present
artifact_dir = builder.build(config, keep_build)
File "/home/ondrej/repos/hashdist/hashdist/core/build_store.py", line 510, in build
self.build_to(artifact_dir, config, keep_build)
File "/home/ondrej/repos/hashdist/hashdist/core/build_store.py", line 530, in build_to
self.run_build_commands(build_dir, artifact_dir, env, config)
File "/home/ondrej/repos/hashdist/hashdist/core/build_store.py", line 561, in run_build_commands
env, self.virtuals, build_dir, config)
File "/home/ondrej/repos/hashdist/hashdist/core/run_job.py", line 316, in run_job
out_env = executor.run(job_spec['script'], env, cwd)
File "/home/ondrej/repos/hashdist/hashdist/core/run_job.py", line 618, in run
self.run_command([cmd] + args, env, cwd, silent=silent)
File "/home/ondrej/repos/hashdist/hashdist/core/run_job.py", line 662, in run_command
self.hdist_command(command_lst, env, cwd, logger)
File "/home/ondrej/repos/hashdist/hashdist/core/run_job.py", line 818, in hdist_command
cli_main(argv, env, logger)
File "/home/ondrej/repos/hashdist/hashdist/cli/main.py", line 110, in main
retcode = args.subcommand_handler(ctx, args)
File "/home/ondrej/repos/hashdist/hashdist/cli/build_tools_cli.py", line 105, in run
source_cache.unpack(key, target, unsafe_mode=True, strip=strip)
File "/home/ondrej/repos/hashdist/hashdist/core/source_cache.py", line 308, in unpack
handler.unpack(type, hash, target_path, unsafe_mode, strip)
File "/home/ondrej/repos/hashdist/hashdist/core/source_cache.py", line 589, in unpack
raise subprocess.CalledProcessError(retcode, tar_cmd[0])
hashdist.core.common.BuildFailedError: CalledProcessError: Command 'tar' returned non-zero exit status 2
So it looks like the download failed. Let's check:
ondrej@hawk:~/repos/hashdist(master)$ cat ~/.hdist/src/packs/tar.gz/d3lib55einogrnsam4e3wcehik2wfn5o
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /zlib-1.2.6.tar.gz was not found on this server.</p>
<p>Additionally, a 404 Not Found
error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body></html>
Ok. So we need to robustify hdist against this problem.
from hashdist.
certik
commented on May 30, 2024
Then the same problem with hdf5:
RuntimeError: File downloaded from "http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.10.tar.bz2" has hash 3n7gqdnb6npmfadqdp3hr4wa6yw2pdty but expected +m5rN7eXbtrIYHMrh8UDcOO+ujrnhNBfFvKYwDOkWkQ
from hashdist.
certik
commented on May 30, 2024
The correct files are:
http://zlib.net/zlib-1.2.7.tar.gz
http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.10-patch1.tar.bz2
Updating the mystack accordingly fixes the issue. Now it's working
ondrej@hawk:~/repos/hashdist(master)$ python mystack.py target
Status:
profile/ph2t.. [ok]
hdf5/uoat.. [ok]
virtual:gcc-stack/host (=gcc-stack/vi4w..) [ok]
virtual:hdist-cli/r0 (=hdist-cli/o5au..) [ok]
szip/m4pr.. [ok]
virtual:unix/host (=unix/djnp..) [ok]
virtual:hdist-cli/r0 (=hdist-cli/o5au..) (see above)
virtual:gcc-stack/host (=gcc-stack/vi4w..) (see above)
zlib/rthi.. [ok]
virtual:gcc-stack/host (=gcc-stack/vi4w..),virtual:unix/host (=unix/djnp..) (see above)
virtual:unix/host (=unix/djnp..) (see above)
szip/m4pr..,zlib/rthi.. (see above)
Everything up to date!
[hashdist] Created "target" -> "/home/ondrej/.hdist/opt/profile/ph2t"
Cool!
from hashdist.
certik
commented on May 30, 2024
All problems raised here are pretty much issue #29. So that can be the first thing for me to work on.
from hashdist.
dagss
commented on May 30, 2024
So the story here is that I thought base64-hashes were so ugly, and changed the hashing in 'master'. There's a warning at the top that wasn't big enough telling you to use the 'stable' branch...
Since hr.recipes is going to be removed in the next release and replaced with something else, the tutorial needs to be redone, so makes no sense to fix it.
from hashdist.
Related Issues (20)
- install fails (hashcode differs from the expectedone)
- add environment variable for hashdist home
- OSX cannot download openssl HOT 10
- HashDist fails to download package if not in source cache mirror
- Diamond inheritance not supported
- --with-mpi=1 is added to configure line HOT 2
- keep build folder, sources and object files for possible debugging HOT 3
- check for git in PATH when source is a Git repo
- Retries are not robust enough
- Force to rebuild everything, ignoring installed packages HOT 2
- Not the same environment for a debug and a non-debug build HOT 3
- Error installing FEniCS on MacOSX with hashdist
- hash differs for libadjoint-1.5
- HashDist fenics instalation in a i7 6700 laptop falis due to lapack not finding my CPU architecture HOT 2
- [zmq|ERROR] Command '[u'/bin/bash', '_hashdist/build.sh']' returned non-zero exit status 1 HOT 1
- OSX TSL1.1+ woes
- Support Python3 or fail properly
- Allow to download sources without building them HOT 1
- ConstructorError: while constructing a mapping HOT 1
- module load package in prologue without rebuild all
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hashdist.