Comments (24)
yay! i think the last two comments helped. i've got it soon
==> Fetching dwarfs
==> Downloading http://bananas.debian.net/macos/dwarfs/dwarfs-0.9.9.orig.tar.xz
Already downloaded: /Users/amyczko/Library/Caches/Homebrew/downloads/3a4ad90117afea0b5c8344658ca4a4984055b55d0b46dbe267796f2f4b59d8e3--dwarfs-0.9.9.orig.tar.xz
==> cmake .
==> make
Error: Empty installation
it's installing for me the binaries and manpages. do you need any libs installed or so?
got a better test suggestion?
dwarfs (v0.9.9-30-g2cb5542a5d on branch main [2024-05-07], fuse version 29)
built for arm64, Darwin-23.4.0, AppleClang 15.0.0.15000309
from dwarfs.
now that i got it building on debian (http://bananas.debian.net/debian/dwarfs/), it should be easy to create a homebrew formula with the help of https://github.com/mhx/dwarfs?tab=readme-ov-file#building-on-macos
99% tests passed, 9 tests failed out of 3095
Total Test time (real) = 27.37 sec
The following tests FAILED:
2970 - dwarfs/tools_test.end_to_end/standalone (Subprocess aborted)
2971 - dwarfs/tools_test.end_to_end/universal-tool (Subprocess aborted)
2972 - dwarfs/tools_test.end_to_end/universal-symlink (Subprocess aborted)
2973 - dwarfs/tools_test.mutating_and_error_ops/standalone (Subprocess aborted)
2974 - dwarfs/tools_test.mutating_and_error_ops/universal-tool (Subprocess aborted)
2975 - dwarfs/tools_test.mutating_and_error_ops/universal-symlink (Subprocess aborted)
2976 - dwarfs/tools_test.categorize/standalone (Subprocess aborted)
2977 - dwarfs/tools_test.categorize/universal-tool (Subprocess aborted)
2978 - dwarfs/tools_test.categorize/universal-symlink (Subprocess aborted)
Errors while running CTest
Output from these tests are in: /Users/amyczko/homebrew/dwarfs/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
FAILED: CMakeFiles/test.util
cd /Users/amyczko/homebrew/dwarfs/build && /opt/homebrew/Cellar/cmake/3.29.2/bin/ctest --force-new-ctest-process
ninja: build stopped: subcommand failed.
some tests failed, otherwise looks good, most difficult part will be the system extension with special reboot needed to install... any chance to skip the fuse part and use it without?
ok there are some binaries in build/ i guess those should be installed by the formula, and that'll be a good start...
when i have something, you'll find it at https://github.com/alexmyczko/homebrew-mac
binaries are quite large...
-rwxr-xr-x 1 amyczko staff 3414184 May 7 18:37 dwarfs
-rwxr-xr-x 1 amyczko staff 2437008 May 7 18:47 dwarfs.strip
-rwxr-xr-x 1 amyczko staff 737296 May 7 18:47 dwarfs.strip.upx
from dwarfs.
Many thanks for looking into this!
some tests failed, otherwise looks good
These tests likely fail because macFUSE isn't fully enabled yet. It's exactly the same failures as when running in a docker container without proper privileges.
most difficult part will be the system extension with special reboot needed to install...
That should be taken care of by the macFUSE install. I agree that it's a real pain, but I don't think there's much that we can do here. (Happy to be convinced otherwise, though!)
any chance to skip the fuse part and use it without?
Yeah, I should have made it easier to disable these tests at run-time. Noted for the next release.
In the meantime, the simplest fix would be to just patch CMakeLists.txt
and remove the tools_test
line.
ok there are some binaries in build/ i guess those should be installed by the formula
The install
target should take care of everything that's worth installing, which is essentially all binaries + all manpages.
binaries are quite large...
Do you think so? I'd consider them quite moderate at a few MiBs. :)
Anyway, really appreciate that you're looking into this!
from dwarfs.
any chance to skip the fuse part and use it without?
A much better way would be something like 2cb5542.
from dwarfs.
I think we should still depend on macfuse
, but just not require that it is already fully set up (with reboots and stuff) when running the tests.
from dwarfs.
and the brew formula is just called macfuse? i will try...
Yeah, it's the 4th item in the brew install
command from the instructions. :)
from dwarfs.
hmmm
Cask 'dwarfs' is unreadable: wrong constant name #<Class:0x000000010fcb6af0>
Warning: Treating ./dwarfs.rb as a formula.
Warning: No available formula with the name "macfuse" (dependency of dwarfs). Did you mean afuse?
==> Searching for similarly named formulae and casks...
==> Casks
macfuse ✔ macfuse@dev
To install macfuse ✔, run:
brew install --cask macfuse ✔
from dwarfs.
Meh: https://github.com/orgs/Homebrew/discussions/3163#discussioncomment-2526187
from dwarfs.
Although this seems to suggest that a formula can depend on a cask...
from dwarfs.
Also interesting: Homebrew/homebrew-core#64491
from dwarfs.
Oh, here we go: https://github.com/gromgit/homebrew-fuse
from dwarfs.
This might help: https://github.com/gromgit/homebrew-fuse/blob/main/require/macfuse.rb
And this: https://github.com/gromgit/homebrew-fuse/blob/main/Formula/squashfuse-mac.rb
from dwarfs.
I guess we have to build and be installable without macfuse. Homebrew/homebrew-core#171264 (comment)
That kinda defeats the whole point of DwarFS and is going to be confusing and annoying at the same time. Confusing because likely people installing it won't know they can't use the FUSE part until they've finished installation. And annoying because then they have to re-install the whole thing (which takes a while to build) after installing macFUSE.
My personal (likely biased) opinion is that banning the dependency on macFUSE is... suboptimal.
There are a couple of ways that I can see this proceeding:
- As you suggest, make DwarFS build/install without FUSE. Then it'd depend on whether or not macFUSE is already installed if you get the FUSE driver or not.
- Add the formula to gromgit/homebrew-fuse rather than homebrew-core.
- Have a
dwarfs
formula in homebrew-core, with a:linux
dependency, and adwarfs-mac
formula in homebrew-fuse. - Have
libdwarfs
,dwarfs-tools
,dwarfs
(:linux
) formulae, all in homebrew/core, and adwarfs-mac
formula in homebrew/fuse. The latter 3 formulae would all have a dependency onlibdwarfs
;dwarfs-tools
would be installable without the dependency on macFUSE.
I personally like option (4) the most, but it's going to be a significant amount of work on the codebase. Until we get there, (2)/(3) are probably my preference.
from dwarfs.
- for homebrew-core
- it is already there https://github.com/alexmyczko/homebrew-mac
- exercise for someone else, i will do the debian package, that devuan, raspbian, ubuntu will copy
- no idea
from dwarfs.
- for homebrew-core
- it is already there https://github.com/alexmyczko/homebrew-mac
- exercise for someone else, i will do the debian package, that devuan, raspbian, ubuntu will copy
- no idea
I'm trying to get the code into a state where (4) is possible.
from dwarfs.
As starting point I've created this tarball using git2deb
: http://bananas.debian.net/macos/dwarfs/
from dwarfs.
so download dwarfs.rb
, run brew install ./dwarfs.rb
and it fails:
https://github.com/alexmyczko/homebrew-mac/blob/main/dwarfs.rb
ah my tarball is bad, i'll get that fixed and soon there'll be a formula...
now it's:
-- Checking for module 'flac++>=1.4.2'
-- Found flac++, version 1.4.3
-- Checking for module 'libzstd>=1.5.2'
-- Found libzstd, version 1.5.6
-- Checking for module 'libxxhash>=0.8.1'
-- Found libxxhash, version 0.8.2
-- Checking for module 'fuse>=2.9.9'
-- No package 'fuse' found
-- Checking for module 'fuse3>=3.10.5'
-- No package 'fuse3' found
CMake Error at CMakeLists.txt:354 (message):
No FUSE or FUSE3 library found
-- Configuring incomplete, errors occurred!
will need to check ccmake
options if i can do something about it...
from dwarfs.
now it's:
-- Checking for module 'flac++>=1.4.2' -- Found flac++, version 1.4.3 -- Checking for module 'libzstd>=1.5.2' -- Found libzstd, version 1.5.6 -- Checking for module 'libxxhash>=0.8.1' -- Found libxxhash, version 0.8.2 -- Checking for module 'fuse>=2.9.9' -- No package 'fuse' found -- Checking for module 'fuse3>=3.10.5' -- No package 'fuse3' found CMake Error at CMakeLists.txt:354 (message): No FUSE or FUSE3 library found -- Configuring incomplete, errors occurred!
I'd guess that it's because the macfuse
dependency is missing from the build script?
from dwarfs.
and the brew formula is just called macfuse? i will try...
from dwarfs.
now it's:
-- Checking for module 'flac++>=1.4.2'
-- Found flac++, version 1.4.3
-- Checking for module 'libzstd>=1.5.2'
-- Found libzstd, version 1.5.6
-- Checking for module 'libxxhash>=0.8.1'
-- Found libxxhash, version 0.8.2
-- Checking for module 'fuse>=2.9.9'
-- No package 'fuse' found
-- Checking for module 'fuse3>=3.10.5'
-- No package 'fuse3' found
CMake Error at CMakeLists.txt:354 (message):
No FUSE or FUSE3 library found
-- Configuring incomplete, errors occurred!
I'd guess that it's because the
macfuse
dependency is missing from the build script?
i tried with, without luck
from dwarfs.
yay! i think the last two comments helped. i've got it soon
That is awesome!
==> Fetching dwarfs ==> Downloading http://bananas.debian.net/macos/dwarfs/dwarfs-0.9.9.orig.tar.xz Already downloaded: /Users/amyczko/Library/Caches/Homebrew/downloads/3a4ad90117afea0b5c8344658ca4a4984055b55d0b46dbe267796f2f4b59d8e3--dwarfs-0.9.9.orig.tar.xz ==> cmake . ==> make Error: Empty installation
it's installing for me the binaries and manpages. do you need any libs installed or so?
Binaries and manpages should be sufficient.
got a better test suggestion?
If you could get the following sequence of commands to run via homebrew that would be ideal (for the ninja version you need ninja
as a dependency):
cmake -DWITH_TESTS=ON -GNinja
, or at leastcmake -DWITH_TESTS=ON
ninja
ormake -j$(sysctl -n hw.physicalcpu)
ninja test
/make test
, preferably withCTEST_PARALLEL_LEVEL=$(sysctl -n hw.physicalcpu)
andDWARFS_SKIP_FUSE_TESTS=1
set in the environmentninja install
/make install
(this should install everything needed)
dwarfs (v0.9.9-30-g2cb5542a5d on branch main [2024-05-07], fuse version 29) built for arm64, Darwin-23.4.0, AppleClang 15.0.0.15000309
Once this is working I'll roll a new release so you can reference a release tag rather than a random commit.
from dwarfs.
I guess we have to build and be installable without macfuse. Homebrew/homebrew-core#171264 (comment)
from dwarfs.
Hey, it's been a while!
It took only slightly more than 300 commits to pull this off. :)
Please, if you have time, could you try out dwarfs-0.10.0? This release allows modular builds, i.e. you can in theory provide formulae for libdwarfs
, dwarfs-tools
and dwarfs-fuse
, just as mentioned before.
The release should build fine on macOS, both X64 and ARM64 (I've added these to the CI).
It should be fairly straightforward to build the different modules on macOS:
$ tar xf dwarfs-0.10.0.tar.xz
$ cmake -B build-lib -S dwarfs-0.10.0 -GNinja -DWITH_LIBDWARFS=ON -DWITH_TOOLS=OFF -DWITH_FUSE_DRIVER=OFF
$ cmake --build build-lib
$ sudo cmake --install build-lib
$ cmake -B build-tools -S dwarfs-0.10.0 -GNinja -DWITH_LIBDWARFS=OFF -DWITH_TOOLS=ON -DWITH_FUSE_DRIVER=OFF
$ cmake --build build-tools
$ sudo cmake --install build-tools
$ cmake -B build-fuse -S dwarfs-0.10.0 -GNinja -DWITH_LIBDWARFS=OFF -DWITH_TOOLS=OFF -DWITH_FUSE_DRIVER=ON
$ cmake --build build-fuse
$ sudo cmake --install build-fuse
Of course you can also still build everything, enable tests, etc.
Let me know if you need any help!
from dwarfs.
With some help from our AI overlords, I came up with two formulae.
libdwarfs.rb
class Libdwarfs < Formula
desc "A fast high compression read-only file system for Linux, Windows, and macOS"
homepage "https://github.com/mhx/dwarfs"
url "https://github.com/mhx/dwarfs/releases/download/v0.10.0/dwarfs-0.10.0.tar.xz"
sha256 "c01ae59d4662e4f027a7c8a5934b7aebe6edc4f7affd836fa73e6d861d18bf35"
license "GPL-3"
depends_on "cmake" => :build
depends_on "ninja" => :build
depends_on "googletest" => :build
depends_on "pkg-config" => :build
depends_on "libarchive"
depends_on "brotli"
depends_on "lz4"
depends_on "xz"
depends_on "zstd"
depends_on "gflags"
depends_on "glog"
depends_on "parallel-hashmap"
depends_on "boost"
depends_on "double-conversion"
depends_on "libevent"
depends_on "fmt"
depends_on "utf8cpp"
depends_on "xxhash"
depends_on "flac"
patch do
url "https://github.com/mhx/dwarfs/releases/download/v0.10.0/dwarfs-system-gtest.patch"
sha256 "573656e33a171017046a32500b9ebe50ebd117df16212e6ef8f8d4b034618210"
end
def install
args = std_cmake_args + %W[
-DWITH_TESTS=ON
-DFOLLY_USE_JEMALLOC=OFF
-DWITH_MAN_PAGES=ON
-DWITH_LIBDWARFS=ON
-DWITH_TOOLS=OFF
-DWITH_FUSE_DRIVER=OFF
-DENABLE_PERFMON=ON
-DTRY_ENABLE_FLAC=ON
-DENABLE_RICEPP=ON
-DENABLE_STACKTRACE=OFF
-DPREFER_SYSTEM_GTEST=ON
]
system "sed", "-i", "", "s|PARALLEL_HASHMAP_REQUIRED_VERSION 1.3.8|PARALLEL_HASHMAP_REQUIRED_VERSION 1.0.0|", "CMakeLists.txt"
system "cmake", "--fresh", "-GNinja", "-S", ".", "-B", "build", *args
system "cmake", "--build", "build", "--parallel"
system "ctest", "--test-dir", "build", "--output-on-failure", "--parallel", ENV.make_jobs
system "cmake", "--install", "build"
end
end
dwarfs-tools.rb
class DwarfsTools < Formula
desc "A fast high compression read-only file system for Linux, Windows, and macOS"
homepage "https://github.com/mhx/dwarfs"
url "https://github.com/mhx/dwarfs/releases/download/v0.10.0/dwarfs-0.10.0.tar.xz"
sha256 "c01ae59d4662e4f027a7c8a5934b7aebe6edc4f7affd836fa73e6d861d18bf35"
license "GPL-3"
depends_on "cmake" => :build
depends_on "ninja" => :build
depends_on "googletest" => :build
depends_on "libdwarfs"
def install
args = std_cmake_args + %W[
-DWITH_LIBDWARFS=OFF
-DWITH_TOOLS=ON
-DWITH_FUSE_DRIVER=OFF
-DWITH_MAN_PAGES=ON
-DWITH_MAN_OPTION=ON
-DPREFER_SYSTEM_GTEST=ON
]
system "cmake", "--fresh", "-GNinja", "-S", ".", "-B", "build", *args
system "cmake", "--build", "build", "--parallel"
system "ctest", "--test-dir", "build", "--output-on-failure", "--parallel", ENV.make_jobs
system "cmake", "--install", "build"
end
end
These work fine, as far as I can tell.
The formula for the macFUSE driver should be very similar to dwarfs-tools
, but I can't figure out how to model the macFUSE dependency properly. Feel free to use these as a starting point.
from dwarfs.
Related Issues (20)
- Error while building HOT 2
- Fuse Passthrough
- Are the prebuilt binaries affected by xz/liblzma backdoor? HOT 6
- [Feature request] Allow providing dwarfs with a dedup library HOT 4
- official debian package, allow building without git? HOT 4
- mkdwarfs always crashes with SIGABRT HOT 13
- [MacOS] DwarFS mount not seen through Finder HOT 5
- [Core Dump] Signal 7 (SIGBUS) (code: nonexistent physical address) on making archive of currently running OS (possibly bad use case) HOT 2
- some problem on the README.md files. Please Check up and Fix. HOT 1
- read scalability issues with large archives HOT 9
- Unexpected exception: `inode has no file (any)` HOT 8
- [Feature Request] Mounting multiple archives to the same path HOT 12
- Segfault when using the mold linker HOT 2
- Cannot build v0.9.9 on Ubuntu 22.04 HOT 5
- Vendor fbthrift & folly using vcpkg HOT 2
- exception thrown in worker thread: class dwarfs::runtime_error: lzma_stream_encoder HOT 2
- [Feature Request] Provide non-generic packaging CI for major linux distributions HOT 2
- Document memory behaviour and give tips for dealing with many files HOT 4
- Gentoo ebuild for v0.10.0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dwarfs.