kahing / catfs Goto Github PK

View Code? Open in Web Editor NEW

809.0 20.0 54.0 498 KB

Cache AnyThing filesystem written in Rust

License: Apache License 2.0

Rust 90.93% Shell 6.28% Python 1.55% Gnuplot 0.89% Dockerfile 0.35%

filesystem fuse rust cache-storage cache fuse-filesystem

catfs's People

Contributors

Stargazers

Watchers

Forkers

manvalls cloudxtreme eyedeekay mgodlewski wjt evaluation-alex rivy thomaskurz amesianx supjerk tanshion jqwang1992 yjh0502 isgasho rustcamp mdeps gaul knightxun kaznak liuhaibinniu potatogim rickysarraf pothix absidish holajiawei nextjournal moneytech jonringer ca4ti rheehot sinhasantos tianzhirui grainspring hopkings2008 cnliao strategist922 kz33 jamsinclair materone henrikbengtsson zhutony joe2hpimn dev-jwel udaykkumar qdore clownsw tjni suryatmodulus minghuaw changweige jpedrick basetutu iq-scm linuxpham

catfs's Issues

Invalid 8 Byte Files, Frequent Hangs of Applications, ERROR - "path/file.something" is not a valid cache file, deleting

Hello,

I love catfs. I'm using it to provide caching for a goofys s3 "mount". It makes a huge difference!

I do however have some nagging issues which happen on a reglar basis. I hope I have provided enough information and would like to help resolve this if it's a non-local specific issue.

System:

Gentoo linux
uname -a: Linux deleted 5.12.13-gentoo-x86_64 #1 SMP Thu Jun 24 12:24:13 DELETED 2021 x86_64 Intel(R) Core(TM) i7-7820X CPU @ 3.60GHz GenuineIntel GNU/Linux
16 cores
32GB RAM
Filesystem which the cache is using: 100GB, EXT4-fs (sde1): mounted filesystem with ordered data mode. Opts: errors=remount-ro,user_xattr. Quota mode: disabled.

catfs issues:

When viewing the data of a directory full of .tif files, the first one requested is always 8 bytes, which appears to be the bare minimum of data to get 'file' to report it is a tif image.
Indeed the first 8 bytes of real source file is the same, however, it is far larger than 8 bytes.
When viewing the data of a directory full of .tif files, first request or not, screenful of errors are printed out by catfs. Examples:

2021-06-30 16:53:22 ERROR - "2005_Hurricane_Katrina/aug31JpegTiles_GCS_NAD83/aug31C0902830w295700n.tif" is not a valid cache file, deleting
2021-06-30 16:53:23 ERROR - "2005_Hurricane_Katrina/aug31JpegTiles_GCS_NAD83/aug31C0902830w295830n.tif" is not a valid cache file, deleting
2021-06-30 16:53:23 ERROR - "2005_Hurricane_Katrina/aug31JpegTiles_GCS_NAD83/aug31C0902830w300000n.tif" is not a valid cache file, deleting
2021-06-30 16:53:24 ERROR - "2005_Hurricane_Katrina/aug31JpegTiles_GCS_NAD83/aug31C0902830w300130n.tif" is not a valid cache file, deleting
2021-06-30 16:53:24 ERROR - "2005_Hurricane_Katrina/aug31JpegTiles_GCS_NAD83/aug31C0903000w295530n.tif" is not a valid cache file, deleting
2021-06-30 16:53:25 ERROR - "2005_Hurricane_Katrina/aug31JpegTiles_GCS_NAD83/aug31C0903000w295700n.tif" is not a valid cache file, deleting
2021-06-30 16:53:25 ERROR - "2005_Hurricane_Katrina/aug31JpegTiles_GCS_NAD83/aug31C0903000w295830n.tif" is not a valid cache file, deleting
2021-06-30 16:53:26 ERROR - "2005_Hurricane_Katrina/aug31JpegTiles_GCS_NAD83/aug31C0903000w300000n.tif" is not a valid cache file, deleting

During this time, which can be several minutes, the application trying to use the data is hung and usually needs force killed.
While catfs is "deleting" files, the total size of the cache directory root grows
After these few minutes, everything works, except for that first invalid 8 byte file.
Often but not always after this delay, despite having fetched files before, they are fetched new again (increased inbound network traffic is observed)
Gwenview STDERR on the 8 byte file:
gwenview.libtiff: Error JPEGLib "Not a JPEG file: starts with 0xd5 0x7e"
org.kde.kdegraphics.gwenview.lib: Could not generate thumbnail for file "file:///media/s3-noaa-eri-pds-LOCAL-catfs/2005_Hurricane_Katrina/sep02JpegTiles_GCS_NAD83/sep02C0890130w290900n.tif"
gwenview.libtiff: Error JPEGLib "Not a JPEG file: starts with 0xd5 0x7e"
org.kde.kdegraphics.gwenview.lib: Could not generate thumbnail for file "file:///media/s3-noaa-eri-pds-LOCAL-catfs/2005_Hurricane_Katrina/sep02JpegTiles_GCS_NAD83/sep02C0890130w290900n.tif"
Sometimes after all the other images have been downloaded and thumbnails created, I can double-click on that first file that had been 8 bytes, and it's now downloaded, and will display without error. Other times, it stays 8 bytes.
Reproduction
Mount an aws s3 public bucket with goofys: goofys -o allow_other -o ro noaa-eri-pds /media/s3-noaa-eri-pds-LOCAL
Start catfs appropriately: catfs -o ro --free 1G /media/s3-noaa-eri-pds-LOCAL /home/myusername/.goofys-cache /media/s3-noaa-eri-pds-LOCAL-catfs
Explore the images with a tool such as gwenview that will show thumbnails
Optional: Start gwenview on the command line, outputting STDOUT and STDERR to their own files and then watch these files: gwenview 1> /tmp/gwenview-STDOUT.txt 2> /tmp/gwenview-STDERR.txt & tail -f /tmp/gwenview-STD*

Opens 50,000 pipes to self, hits EMFILE (too many open files), reads incorrect data

I am attempting to use goofys + catfs to mount a local, read-only view of an S3 bucket as follows:

$ /usr/local/bin/goofys -o allow_other --cache -o:allow_other:/var/cache/mirror --dir-mode 493 --file-mode 292 images-dl.endlessm.com /srv/images-dl.endlessm.com_s3

I'm then pointing mirrorbits at a subtree of this directory. It will walk all files in that tree. It's configured to never actually read the contents of any of the files (I have disabled all hashing) but as an implementation detail it will open every file it encounters for reading, and then immediately close it.
The net effect is that every file in the tree is read in quick succession, then closed. There are 27582 files in the subtree in question.

It looks from the logs like like goofys/catfs to do some read-ahead on each file as it is opened. Unfortunately, mid-way through the scan, catfs runs out of file descriptors:

catfs.INFO 2018-07-13 15:30:56 ERROR - read ahead "release/3.4.5/eos-amd64-amd64/base/eos-eos3.4-amd64-amd64.180712-201651.base.img.xz.asc" failed: Too many open files (os error 24) stack backtrace:
catfs.INFO    0:     0x55ee812f12bc - backtrace::backtrace::trace::h7544cb300c8adc70
catfs.INFO    1:     0x55ee812f12f2 - backtrace::capture::Backtrace::new::h0e3ca146bdded087
catfs.INFO    2:     0x55ee81217255 - <catfs::catfs::error::RError<E>>::from::hc6f3000dd77f7864
catfs.INFO    3:     0x55ee8121aabf - catfs::catfs::file::Handle::copy::hb749b1d9c173f302
catfs.INFO    4:     0x55ee811f57dc - <F as threadpool::FnBox>::call_box::h20dacdcfe2491af7
catfs.INFO    5:     0x55ee812bcb41 - std::sys_common::backtrace::__rust_begin_short_backtrace::h69ffcb2ce386e3c5
catfs.INFO    6:     0x55ee81342d3e - panic_unwind::__rust_maybe_catch_panic
catfs.INFO                         at /checkout/src/libpanic_unwind/lib.rs:101
catfs.INFO    7:     0x55ee812c0030 - <F as alloc::boxed::FnBox<A>>::call_box::hfbaa7f762fcf229f
catfs.INFO    8:     0x55ee8133950b - alloc::boxed::{{impl}}::call_once<(),()>
catfs.INFO                         at /checkout/src/liballoc/boxed.rs:772
catfs.INFO                          - std::sys_common::thread::start_thread
catfs.INFO                         at /checkout/src/libstd/sys_common/thread.rs:24
catfs.INFO                          - std::sys::imp::thread::{{impl}}::new::thread_start
catfs.INFO                         at /checkout/src/libstd/sys/unix/thread.rs:90
catfs.INFO    9:     0x7f165f762493 - start_thread
catfs.INFO   10:     0x7f165f28dace - __clone
catfs.INFO   11:                0x0 - <unknown>

Even after the scan has finished and mirrorbits is shut down (so no files should be open), attempting to read most files causes this error. Looking at the output of lsof, catfs has over 100,000 FIFO fds open; they mostly come in r/w pairs with the same NODE.

$ sudo lsof  | grep catfs | grep FIFO | wc -l
109404
$ sudo lsof | head -n1; sudo lsof  | grep catfs | grep FIFO | head -n 10
COMMAND     PID   TID        USER   FD      TYPE             DEVICE SIZE/OFF       NODE NAME
catfs     14161            mirror    2w     FIFO               0,10      0t0     234581 pipe
catfs     14161            mirror   10r     FIFO               0,10      0t0     234783 pipe
catfs     14161            mirror   12w     FIFO               0,10      0t0     234783 pipe
catfs     14161            mirror   13r     FIFO               0,10      0t0     234785 pipe
catfs     14161            mirror   14w     FIFO               0,10      0t0     234785 pipe
catfs     14161            mirror   15r     FIFO               0,10      0t0     234786 pipe
catfs     14161            mirror   16w     FIFO               0,10      0t0     234786 pipe
catfs     14161            mirror   17r     FIFO               0,10      0t0     234790 pipe
catfs     14161            mirror   18w     FIFO               0,10      0t0     234790 pipe
catfs     14161            mirror   19r     FIFO               0,10      0t0     234794 pipe

Reading certain files returns EMFILE; reading others returns content of the correct length but with the contents of the wrong file. For example, $mountpoint/release/3.4.5/eos-amd64-amd64/base/eos-eos3.4-amd64-amd64.180712-201651.base.iso.asc is 833 bytes long (correct) but those bytes are the first 833 bytes of an adjacent file $mountpoint/release/3.4.5/eos-amd64-amd64/base/eos-eos3.4-amd64-amd64.180712-201651.base.iso.

Any suggestions for how to begin debugging this? I'm afraid I have never written a line of Go or Rust in my life!

$ catfs --version
catfs 0.7.0
$ sha256sum $(which catfs)
306466bea43aaa2e14f980479fa49774913043bab35f6b27044e8150419ae97c  /usr/local/bin/catfs

despite what catfs --version says, it's actually https://github.com/kahing/catfs/releases/download/v0.8.0/catfs .

be more resilient to loss of backing device

Consider this scenario:

Mar 13 20:51:26 priyasi systemd[1]: Unnecessary job for /dev/fuse was removed.
Mar 13 20:51:26 priyasi systemd[1]: Unmounting /home/rrs/.cache/catfs/chutzpah...
Mar 13 20:51:26 priyasi catfs[4583]: "/home/rrs/.cache/catfs/chutzpah" unmounted
Mar 13 20:51:26 priyasi catfs[4583]: Unmounted /home/rrs/.cache/catfs/chutzpah
Mar 13 20:51:26 priyasi systemd[3356]: home-rrs-.cache-catfs-chutzpah.mount: Succeeded.
Mar 13 20:51:26 priyasi catfs[4583]: Received TERM, attempting to unmount "/home/rrs/.cache/catfs/chutzpah"
Mar 13 20:51:26 priyasi systemd[1]: home-rrs-.cache-catfs-chutzpah.mount: Succeeded.
Mar 13 20:51:26 priyasi systemd[1]: Unmounted /home/rrs/.cache/catfs/chutzpah.
Mar 13 20:51:26 priyasi systemd[1]: Mounting /home/rrs/.cache/catfs/chutzpah...
Mar 13 20:51:26 priyasi systemd[1]: home-rrs-.cache-catfs-chutzpah.mount: Mount process finished, but there is no mount.
Mar 13 20:51:26 priyasi systemd[1]: home-rrs-.cache-catfs-chutzpah.mount: Failed with result 'protocol'.
Mar 13 20:51:26 priyasi systemd[1]: Failed to mount /home/rrs/.cache/catfs/chutzpah.
Mar 13 20:51:26 priyasi systemd[1]: mnt-chutzpah.automount: Got automount request for /mnt/chutzpah, triggered by 14940 (catfs)
Mar 13 20:51:26 priyasi systemd[1]: Mounting /mnt/chutzpah...
Mar 13 20:51:26 priyasi polkitd(authority=local)[788]: Unregistered Authentication Agent for unix-process:14923:643936 (system bus name :1.1946, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_IN.UTF-8) (disconnected from bus)
Mar 13 20:51:31 priyasi systemd[1]: mnt-chutzpah.mount: Mounting timed out. Terminating.
Mar 13 20:51:31 priyasi mount[14948]: read: Interrupted system call
Mar 13 20:51:31 priyasi systemd[3356]: mnt-chutzpah.mount: Succeeded.
Mar 13 20:51:31 priyasi systemd[1]: mnt-chutzpah.mount: Mount process exited, code=killed, status=15/TERM
Mar 13 20:51:31 priyasi systemd[1]: mnt-chutzpah.mount: Failed with result 'timeout'.
Mar 13 20:51:31 priyasi systemd[1]: Failed to mount /mnt/chutzpah.
Mar 13 20:51:31 priyasi catfs[14940]: Cannot mount: No such device (os error 19) stack backtrace:
Mar 13 20:51:31 priyasi catfs[14940]:    0:     0x555c859ecf35 - catfs::catfs::error::RError<E>::from::h895331c5ecbb2e25
Mar 13 20:51:31 priyasi catfs[14940]:                         at src/catfs/error.rs:52
Mar 13 20:51:31 priyasi catfs[14940]:    1:     0x555c85a17e45 - <catfs::catfs::error::RError<std::io::error::Error> as core::convert::From<std::io::error::Error>>::from::hb6598e8d987b0b29
Mar 13 20:51:31 priyasi catfs[14940]:                         at src/catfs/error.rs:115
Mar 13 20:51:31 priyasi catfs[14940]:    2:     0x555c859d7104 - catfs::catfs::CatFS::new::h6333f2023f3b932d
Mar 13 20:51:31 priyasi catfs[14940]:                         at src/catfs/mod.rs:114
Mar 13 20:51:31 priyasi catfs[14940]:    3:     0x555c85a5afec - catfs::main_internal::h301b7b64c624340c
Mar 13 20:51:31 priyasi catfs[14940]:                         at src/main.rs:237
Mar 13 20:51:31 priyasi catfs[14940]:    4:     0x555c85a580a0 - catfs::main::hfa37ddebbee1dd6d
Mar 13 20:51:31 priyasi catfs[14940]:                         at src/main.rs:40
Mar 13 20:51:31 priyasi catfs[14940]:    5:     0x555c85a5d8df - std::rt::lang_start::{{closure}}::hf3ef3448014deb0d
Mar 13 20:51:31 priyasi catfs[14940]:                         at /usr/src/rustc-1.40.0/src/libstd/rt.rs:61
Mar 13 20:51:31 priyasi catfs[14940]:    6:     0x555c85dea5a2 - _ZN3std9panicking3try7do_call17h405fa073712ab5d5E.llvm.17613216718872221368
Mar 13 20:51:31 priyasi catfs[14940]:    7:     0x555c85df8389 - __rust_maybe_catch_panic
Mar 13 20:51:31 priyasi catfs[14940]:    8:     0x555c85df0138 - std::rt::lang_start_internal::he63ceb3eba03dd55
Mar 13 20:51:31 priyasi catfs[14940]:    9:     0x555c85a5d8b8 - std::rt::lang_start::h80110ac54506f8b6
Mar 13 20:51:31 priyasi catfs[14940]:                         at /usr/src/rustc-1.40.0/src/libstd/rt.rs:61
Mar 13 20:51:31 priyasi catfs[14940]:   10:     0x555c85a5baa9 - main
Mar 13 20:51:31 priyasi catfs[14940]:   11:     0x7faacc3b3bba - __libc_start_main
Mar 13 20:51:31 priyasi catfs[14940]:   12:     0x555c859a61e9 - _start
Mar 13 20:51:31 priyasi catfs[14940]:   13:                0x0 - <unknown>

The backing device (a remote share accessible over sshfs) is not persistent
But the local cache is persistent

Under circumstances where the backing device is lost (network interruption, network change, roaming profile etc), the data should still be served transparently from the cache

`grep -r foo .` causes crash

phone is an sshfs.

I did a grep -r foo . and got:

$ catfs phone phone.cache phone.cached
thread '<unnamed>' panicked at 'index out of bounds: the len is 0 but the index is 0', /home/tange/.cargo/registry/src/github.com-1ecc6299db9ec823/catfs-0.8.0/src/catfs/error.rs:56:30
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at 'index out of bounds: the len is 0 but the index is 0', /home/tange/.cargo/registry/src/github.com-1ecc6299db9ec823/catfs-0.8.0/src/catfs/error.rs:56:30
stack backtrace:
   0:     0x564b76983fe0 - std::backtrace_rs::backtrace::libunwind::trace::h577ea05e9ca4629a
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/../../backtrace/src/backtrace/libunwind.rs:96
   1:     0x564b76983fe0 - std::backtrace_rs::backtrace::trace_unsynchronized::h50b9b72b84c7dd56
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/../../backtrace/src/backtrace/mod.rs:66
   2:     0x564b76983fe0 - std::sys_common::backtrace::_print_fmt::h6541cf9823837fac
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/sys_common/backtrace.rs:79
   3:     0x564b76983fe0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hf64fbff071026df5
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/sys_common/backtrace.rs:58
   4:     0x564b769a052c - core::fmt::write::h9ddafa4860d8adff
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/core/src/fmt/mod.rs:1082
   5:     0x564b76981057 - std::io::Write::write_fmt::h1d2ee292d2b65481
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/io/mod.rs:1514
   6:     0x564b76986440 - std::sys_common::backtrace::_print::ha25f9ff5080d886d
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/sys_common/backtrace.rs:61
   7:     0x564b76986440 - std::sys_common::backtrace::print::h213e8aa8dc5405c0
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/sys_common/backtrace.rs:48
   8:     0x564b76986440 - std::panicking::default_hook::{{closure}}::h6482fae49ef9d963
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/panicking.rs:200
   9:     0x564b7698618c - std::panicking::default_hook::he30ad7589e0970f9
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/panicking.rs:219
  10:     0x564b76986aa3 - std::panicking::rust_panic_with_hook::haa1ed36ada4ffb03
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/panicking.rs:569
  11:     0x564b76986679 - std::panicking::begin_panic_handler::{{closure}}::h7001af1bb21aeaeb
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/panicking.rs:476
  12:     0x564b7698446c - std::sys_common::backtrace::__rust_end_short_backtrace::h39910f557f5f2367
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/sys_common/backtrace.rs:153
  13:     0x564b76986639 - rust_begin_unwind
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/panicking.rs:475
  14:     0x564b7699f851 - core::panicking::panic_fmt::h4e2659771ebc78eb
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/core/src/panicking.rs:85
  15:     0x564b7699f812 - core::panicking::panic_bounds_check::h2e8c50d2fb4877c0
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/core/src/panicking.rs:62
  16:     0x564b768678cb - catfs::catfs::error::RError<E>::from::h1d8d9839a102ee7f
  17:     0x564b76861330 - <catfs::catfs::file::Handle as core::ops::drop::Drop>::drop::h7da3221858c64a46
  18:     0x564b7685ad6c - core::ptr::drop_in_place::h3e2659181536cf18
  19:     0x564b7685af2c - core::ptr::drop_in_place::hb43e9bcb923eae34
  20:     0x564b7685ac7d - <F as threadpool::FnBox>::call_box::h0ea576581ced0b63
  21:     0x564b768a36db - std::sys_common::backtrace::__rust_begin_short_backtrace::h0b88fcd08d2363c9
  22:     0x564b768a3bda - core::ops::function::FnOnce::call_once{{vtable.shim}}::hebe05edcb5c35f61
  23:     0x564b7698942a - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h670c50864ac2cb92
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/alloc/src/boxed.rs:1042
  24:     0x564b7698942a - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h2511952749086d81
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/alloc/src/boxed.rs:1042
  25:     0x564b7698942a - std::sys::unix::thread::Thread::new::thread_start::h5ad4ddffe24373a8
                               at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/sys/unix/thread.rs:87
  26:     0x7f2c74d8b609 - start_thread
  27:     0x7f2c74c97293 - clone
  28:                0x0 - <unknown>
thread panicked while panicking. aborting.
Illegal instruction (core dumped)

Block cache support

Please add support for caching random reads and not only for the entire file.
This would be great to read small parts of huge files (i.e. chia plots).

0.9.0 release

master has many fixes for modern Rust and improves compatibility with 32-bit and macOS. It would be nice to pull in the fix for splice as well.

How the enable remote cache?

Hello, I'am trying to use the catfs to test the storage on local but the cache in the remote.
Right now I can run the cache and the storage all in local, but is there a way to enable cache as the remote?

According to the tests in the README file implies that either the cache or the storage was running in the remote side correct?
I am trying to use NFS in order to use the remote cache, but I don't know whether this is the right way

How to limit number of threads so that catfs can be used in multi-tenant environments?

From experimenting with the devel version of catfs (commit cbd7ab7), I noticed that it uses multi-threading. Looking at htop, it appears that catfs is using all(?) CPU cores by default. Is that correct?

Also, is there a way, e.g. a command-line option, to limit the number of cores that a specific catfs instance will use?

(I know zero Rust, otherwise I'd try to figure this out myself from the source code.)

32 bits build is broken

I'm trying to build catfs on raspberry pi 2 (arm 32bits) and I've got the following errors:

error[E0308]: mismatched types
   --> src/catfs/mod.rs:195:21
    |
195 |                     st.f_blocks,
    |                     ^^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
195 |                     st.f_blocks.into(),
    |                     ^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/mod.rs:196:21
    |
196 |                     st.f_bfree,
    |                     ^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
196 |                     st.f_bfree.into(),
    |                     ^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/mod.rs:197:21
    |
197 |                     st.f_bavail,
    |                     ^^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
197 |                     st.f_bavail.into(),
    |                     ^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/mod.rs:198:21
    |
198 |                     st.f_files,
    |                     ^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
198 |                     st.f_files.into(),
    |                     ^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/mod.rs:199:21
    |
199 |                     st.f_ffree,
    |                     ^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
199 |                     st.f_ffree.into(),
    |                     ^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
  --> src/catfs/rlibc.rs:95:28
   |
95 |         libc::seekdir(dir, loc);
   |                            ^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:137:16
    |
137 |         return self.en.d_ino;
    |                ^^^^^^^^^^^^^ expected u64, found u32

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:140:16
    |
140 |         return self.en.d_off;
    |                ^^^^^^^^^^^^^ expected i64, found i32

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:353:21
    |
353 |             tv_sec: atime.sec,
    |                     ^^^^^^^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:354:22
    |
354 |             tv_nsec: atime.nsec as i64,
    |                      ^^^^^^^^^^^^^^^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:357:21
    |
357 |             tv_sec: mtime.sec,
    |                     ^^^^^^^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:358:22
    |
358 |             tv_nsec: mtime.nsec as i64,
    |                      ^^^^^^^^^^^^^^^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:446:53
    |
446 |         let res = unsafe { libc::ftruncate(self.fd, size as i64) };
    |                                                     ^^^^^^^^^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:455:59
    |
455 |         let res = unsafe { libc::posix_fallocate(self.fd, offset as i64, len as i64) };
    |                                                           ^^^^^^^^^^^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:455:74
    |
455 |         let res = unsafe { libc::posix_fallocate(self.fd, offset as i64, len as i64) };
    |                                                                          ^^^^^^^^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:489:76
    |
489 |             unsafe { libc::pread(self.fd, as_mut_void_ptr(buf), buf.len(), offset) };
    |                                                                            ^^^^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:498:82
    |
498 |         let nbytes = unsafe { libc::pwrite(self.fd, as_void_ptr(buf), buf.len(), offset) };
    |                                                                                  ^^^^^^ expected i32, found i64

error[E0308]: mismatched types
   --> src/catfs/inode.rs:112:18
    |
112 |             ino: st.st_ino,
    |                  ^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
112 |             ino: st.st_ino.into(),
    |                  ^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/inode.rs:116:22
    |
116 |                 sec: st.st_atime,
    |                      ^^^^^^^^^^^ expected i64, found i32
help: you can cast an `i32` to `i64`, which will sign-extend the source value
    |
116 |                 sec: st.st_atime.into(),
    |                      ^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/inode.rs:120:22
    |
120 |                 sec: st.st_mtime,
    |                      ^^^^^^^^^^^ expected i64, found i32
help: you can cast an `i32` to `i64`, which will sign-extend the source value
    |
120 |                 sec: st.st_mtime.into(),
    |                      ^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/inode.rs:124:22
    |
124 |                 sec: st.st_ctime,
    |                      ^^^^^^^^^^^ expected i64, found i32
help: you can cast an `i32` to `i64`, which will sign-extend the source value
    |
124 |                 sec: st.st_ctime.into(),
    |                      ^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/inode.rs:128:22
    |
128 |                 sec: st.st_ctime,
    |                      ^^^^^^^^^^^ expected i64, found i32
help: you can cast an `i32` to `i64`, which will sign-extend the source value
    |
128 |                 sec: st.st_ctime.into(),
    |                      ^^^^^^^^^^^^^^^^^^

error: aborting due to 22 previous errors

For more information about this error, try `rustc --explain E0308`.
error: Could not compile `catfs`.

To create a 32 bits chroot:

DIR=ubuntu32
RELEASE=bionic
mkdir $DIR
sudo debootstrap --arch i386 $RELEASE $DIR http://archive.ubuntu.com/ubuntu/
sudo mount -t proc proc $DIR/proc
sudo mount -t devtmpfs dev $DIR/dev
sudo mkdur $DIR/src
sudo mount --bind $(pwd) $DIR/src
sudo setarch i386 chroot $DIR /bin/bash
apt-get install curl build-essential libfuse-dev pkg-config
curl -sSf https://static.rust-lang.org/rustup.sh | sh
cd /src
cargo build

libc::ENOATTR deprecation

The newer libc warns:

warning: use of deprecated constant `libc::ENOATTR`: ENOATTR is not available on Linux; use ENODATA instead
   --> src/catfs/file.rs:274:61
    |
274 |                 if my_errno != libc::ENODATA && my_errno != libc::ENOATTR {
    |                                                             ^^^^^^^^^^^^^
    |
    = note: `#[warn(deprecated)]` on by default

I added this to handle macOS behavior. Some context: rust-lang/libc#1356 (comment)

caching rules?

What is the specific rule/algorithm for caching? Is it by time/space? And could it be customized?

Not working with SeaDrive

I tried catfs with SeaDrive because it could be a great way to selectively sync part of my drive.
Unfortunately it doesn't seem to work. When I click on one of the folder, it just keeps displaying them again in a kind of loop. Don't know if it is related to catfs or seadrive through.

Thread 'main' panicked at 'index out of bounds: the len is 0 but the index is 0'

I can reproduce this crash most times (but not always) with the following steps:

create a large media file in (in my case, a 1.5GB mp4)
run cat file.mp4
press Control+C *multiple times) to cancel the cat command

Of course it doesn't make much sense to do this, I was just testing stuff. And if the error occurs here, it might also occur in other cases.

2018-06-18 07:14:34 ERROR - "file.mp4" is not a valid cache file, deleting
thread '<unnamed>' panicked at 'index out of bounds: the len is 0 but the index is 0', src/catfs/error.rs:56:30
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::begin_panic
   6: std::panicking::begin_panic_fmt
   7: rust_begin_unwind
   8: core::panicking::panic_fmt
   9: core::panicking::panic_bounds_check
  10: <catfs::catfs::error::RError<E>>::from
  11: catfs::catfs::file::Handle::copy
  12: <F as threadpool::FnBox>::call_box
thread '<unnamed>' panicked at 'index out of bounds: the len is 0 but the index is 0', src/catfs/error.rs:56:30
stack backtrace:
   0:     0x560f168c7f6b - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::h79684051ac41db78
   1:     0x560f168cddce - std::sys_common::backtrace::print::h9921892c7d251199
   2:     0x560f168c1bc0 - std::panicking::default_hook::{{closure}}::hb1ef6f271daa5072
   3:     0x560f168c16f6 - std::panicking::default_hook::h2b67894d489b9efc
   4:     0x560f168c200b - std::panicking::rust_panic_with_hook::h41c9d3408267e857
   5:     0x560f168c1e5e - std::panicking::begin_panic::h831f3212b8c92f84
   6:     0x560f168c1dca - std::panicking::begin_panic_fmt::h6b11c1b1e86a67d8
   7:     0x560f168c1d62 - rust_begin_unwind
   8:     0x560f168e5190 - core::panicking::panic_fmt::h53107fe7defdc8ff
   9:     0x560f168e5133 - core::panicking::panic_bounds_check::h085a39af45a0b18e
  10:     0x560f167c93b7 - <catfs::catfs::error::RError<E>>::from::h27d490962b8b7e54
  11:     0x560f167b0836 - <catfs::catfs::file::Handle as core::ops::drop::Drop>::drop::h1938f76c4e2338fe
  12:     0x560f167aa27b - core::ptr::drop_in_place::h67e4fafbf0527b3c
  13:     0x560f167aa37b - core::ptr::drop_in_place::h945f3a163c1a7185
  14:     0x560f167aa147 - <F as threadpool::FnBox>::call_box::h5aefb8f0e430b8fc
  15:     0x560f168bda13 - std::sys_common::backtrace::__rust_begin_short_backtrace::heceb445b827e0f72
  16:     0x560f168dbc6e - __rust_maybe_catch_panic
  17:     0x560f168bad72 - <F as alloc::boxed::FnBox<A>>::call_box::h283e02a755a98629
  18:     0x560f168caf57 - std::sys_common::thread::start_thread::hbfbd91005df9c030
  19:     0x560f168c2588 - std::sys::unix::thread::Thread::new::thread_start::h8f1e91b7a8bfa5fb
  20:     0x7f97c456d6da - start_thread
  21:     0x7f97c407e88e - __clone
  22:                0x0 - <unknown>
thread panicked while panicking. aborting.

SSHFS version 2.8
FUSE library version: 2.9.7
fusermount version: 2.9.7
using FUSE kernel interface version 7.19
catfs 0.8.0
cargo 0.26.0
Ubuntu 18.04
Linux 4.15.0-23-generic

read syscall intermittently fails with ECANCELED (Operation canceled)

I have a brand-new, read-only catfs filesystem. I am using it to accelerate a python script that reads many small files from NFS. After the script reads 1,000-20,000 files, the read system call fairly reliably returns -ECANCELED. Python throws an exception and the script terminates. catfs does not log anything when this occurs.

ECANCELED is not mentioned as a possible error in the read(2) or read(3p) man pages. Does catfs actually intend to return this error? If so, what does it mean, and what are programs expected to do about it?

A quick grep shows this line of code as a potential source of the error:

catfs/src/catfs/file.rs

Line 499 in daa2b85

io::Error::from_raw_os_error(libc::ECANCELED),

Hello! and thanks for the response over at pcachefs

catfs was actually one of the very first ones I tried!

I am running ubuntu 18.04.1 and here is how I installed catfs:

sudo apt-get install cargo git libfuse-dev fuse python-fuse
git clone https://github.com/kahing/catfs.git
cd catfs
sudo cargo install catfs

Here is how I tried it out using a sshfs mount /media/remote with catfs :

sshfs -o reconnect,noatime,ServerAliveInterval=15,ServerAliveCountMax=3,cache_timeout=14400,entry_timeout=1200,large_read,max_read=65536,cache=yes,kernel_cache,workaround=nodelaysrv,cache_stat_timeout=600 admin007@myremotefs:/ /media/remote

~/.cargo/bin/catfs /media/remote /media/cache /media/video

and then launched kodi media player (which is configured to read from /media/video), kodi just gave me a spinning progress indicator while trying to play a file from the catfs /media/video mount, and I could see monitoring bandwidth that the file was transferring, so it seemed to be trying to transfer the entire file before playback. I did not have this issue with mcachefs or pcachefs.

Currently I am using mcachefs because of the ones I tried it performed the fastest.

Is there anything else I should try? maybe I am just missing a small option or setting to get this party going :)

Downloads full file

For small files that's fine but for large, multi-gigabyte files it's inefficient to download the entire thing on access because certain operations might only require a quick open/close of the file to read the first few megabytes (for instance reading media metadata using mediainfo or ffmpeg).

Using --gid flag throws "operation not permitted"

I've compiled my own catfs binary from the latest master branch

When mounting with the following:

catfs --gid 1001 --uid 1000 /test-source /test-cache /test

I get the following error:

ERROR - Cannot mount: Operation not permitted (os error 1)    0: catfs::catfs::error::RError<E>::from
   1: catfs::main
   2: std::sys_common::backtrace::__rust_begin_short_backtrace
   3: std::rt::lang_start::{{closure}}
   4: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/core/src/ops/function.rs:259:13
      std::panicking::try::do_call
             at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/panicking.rs:403:40
      std::panicking::try
             at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/panicking.rs:367:19
      std::panic::catch_unwind
             at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/panic.rs:129:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/rt.rs:45:48
      std::panicking::try::do_call
             at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/panicking.rs:403:40
      std::panicking::try
             at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/panicking.rs:367:19
      std::panic::catch_unwind
             at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/panic.rs:129:14
      std::rt::lang_start_internal
             at /rustc/09c42c45858d5f3aedfa670698275303a3d19afa/library/std/src/rt.rs:45:20
   5: main
   6: __libc_start_main
   7: _start

I'm either likely using this software wrong or have some permissioning issues 😅. Let me know if there's something I need to fix on my end or if you need me to provide additional information 🤓

Edit: I am trying to allow other users, or if possible, users of a certain group also access the cached volume.

Thanks for both goofys and catfs these are really cool tools and appreciate your work on them 😊

optimistically write-through

catfs should write-through each IO if possible. On random write, writing to src could fail (ie: goofys), then we can switch back to flush entire file on close.

thread '<unnamed>' panicked at 'ino 0x0000000000005dc6 refcnt 1 deref 2',

I consistently get this panic when using catfs (with a backing sshfs). The accesses that were going on at the time were concurrent reads.

This is on fedora linux 30.

thread '<unnamed>' panicked at 'ino 0x0000000000005dc6 refcnt 1 deref 2', src/catfs/inode.rs:325:13
stack backtrace:
   0:     0x5567456c48d3 - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::h50ebfb8734a81144
                               at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1:     0x5567456c009b - std::sys_common::backtrace::_print::hc7fdae4fb6b58d2d
                               at src/libstd/sys_common/backtrace.rs:71
   2:     0x5567456c3516 - std::panicking::default_hook::{{closure}}::hc55d0892611a29ff
                               at src/libstd/sys_common/backtrace.rs:59
                               at src/libstd/panicking.rs:197
   3:     0x5567456c32a9 - std::panicking::default_hook::h3c8a3df5d3469668
                               at src/libstd/panicking.rs:211
   4:     0x5567456c3c1f - std::panicking::rust_panic_with_hook::h24c9a1c35b1f49cc
                               at src/libstd/panicking.rs:474
   5:     0x5567456c37a1 - std::panicking::continue_panic_fmt::h8ed9632bdd4b9299
                               at src/libstd/panicking.rs:381
   6:     0x5567456c36ee - std::panicking::begin_panic_fmt::h000a944fb20c1b89
                               at src/libstd/panicking.rs:336
   7:     0x55674523a9f8 - catfs::catfs::inode::Inode::deref::ha80f5d680059df59
                               at src/catfs/inode.rs:325
   8:     0x55674527b818 - catfs::catfs::CatFS::forget::he60bc4926a9150db
                               at src/catfs/mod.rs:497
   9:     0x5567452b10b2 - <catfs::pcatfs::PCatFS as fuse::Filesystem>::forget::heaa21df26af5b057
                               at src/pcatfs/mod.rs:114
  10:     0x55674529efae - fuse::request::Request::dispatch::h8a112763bca046d5
                               at /home/yiding/.cargo/registry/src/github.com-1ecc6299db9ec823/fuse-0.3.1/src/request.rs:152
  11:     0x5567452a6b2c - fuse::request::dispatch::h04448e37e6d4b14e
                               at /home/yiding/.cargo/registry/src/github.com-1ecc6299db9ec823/fuse-0.3.1/src/request.rs:34
  12:     0x55674523fbb0 - fuse::session::Session<FS>::run::hb63d616c2d340913
                               at /home/yiding/.cargo/registry/src/github.com-1ecc6299db9ec823/fuse-0.3.1/src/session.rs:79
  13:     0x55674529c0c6 - catfs::main_internal::{{closure}}::he583d75e88ea67ec
                               at src/main.rs:252
  14:     0x556745240162 - std::sys_common::backtrace::__rust_begin_short_backtrace::h701448e976ab0c85
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/sys_common/backtrace.rs:136
  15:     0x5567452b37f4 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::hda067f3acb6e46b0
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/thread/mod.rs:470
  16:     0x556745259cd4 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h68ddc649c0763ecb
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/panic.rs:309
  17:     0x556745259c37 - std::panicking::try::do_call::h88297ddea77f2617
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/panicking.rs:293
  18:     0x5567456c7109 - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:87
  19:     0x556745259b18 - std::panicking::try::h9ebff07a775c52f5
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/panicking.rs:272
  20:     0x556745259d64 - std::panic::catch_unwind::h9fefc47ce2c1cb0d
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/panic.rs:388
  21:     0x5567452b32db - std::thread::Builder::spawn_unchecked::{{closure}}::h33b8858a911913a8
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libstd/thread/mod.rs:469
  22:     0x556745208ab2 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h5c4bbb01fcb79527
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/libcore/ops/function.rs:231
  23:     0x5567456b7a0e - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h4cac16ae2114a837
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/liballoc/boxed.rs:702
  24:     0x5567456c67df - std::sys::unix::thread::Thread::new::thread_start::h2adc1b80820f790e
                               at /rustc/3c235d5600393dfe6c36eeed34042efad8d4f26e/src/liballoc/boxed.rs:702
                               at src/libstd/sys_common/thread.rs:14
                               at src/libstd/sys/unix/thread.rs:80
  25:     0x7fc8442705a1 - start_thread
  26:     0x7fc844183162 - __clone
  27:                0x0 - <unknown>

Loop Device Mounting Behavior

I am trying to mount SquashFS images over goofys with catfs caching (i.e., --cache "--free:1%:/cache"). I experience no issues when catfs is disabled; however, enabling catfs causes some very weird behavior. The following sequence of events consistently happens when attempting to mount a SquashFS file:

The first attempt to mount fails with the following error message:

$ sudo mount /goofys/public_datasets/image/imagenet-2012.squash /image-net
mount: /dev/loop0: can't read superblock

And the kernel prints these error messages in /var/log/syslog:

Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.312455] blk_update_request: 6 callbacks suppressed
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.312458] blk_update_request: I/O error, dev loop0, sector 296022272
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.316717] blk_update_request: I/O error, dev loop0, sector 296022272
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.320672] buffer_io_error: 2 callbacks suppressed
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.320675] Buffer I/O error on dev loop0, logical block 148011136, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.325583] blk_update_request: I/O error, dev loop0, sector 296022274
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.329396] Buffer I/O error on dev loop0, logical block 148011137, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.334262] blk_update_request: I/O error, dev loop0, sector 296022276
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.338123] Buffer I/O error on dev loop0, logical block 148011138, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.343457] blk_update_request: I/O error, dev loop0, sector 296022278
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.347361] Buffer I/O error on dev loop0, logical block 148011139, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.352658] blk_update_request: I/O error, dev loop0, sector 0
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.356387] SQUASHFS error: squashfs_read_data failed to read block 0x0
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.356477] blk_update_request: I/O error, dev loop0, sector 296022272
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360511] blk_update_request: I/O error, dev loop0, sector 296022272
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360523] Buffer I/O error on dev loop0, logical block 148011136, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360703] blk_update_request: I/O error, dev loop0, sector 296022274
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360704] Buffer I/O error on dev loop0, logical block 148011137, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360877] blk_update_request: I/O error, dev loop0, sector 296022276
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.360878] Buffer I/O error on dev loop0, logical block 148011138, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.361046] Buffer I/O error on dev loop0, logical block 148011139, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.362751] Buffer I/O error on dev loop0, logical block 148011136, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.363037] Buffer I/O error on dev loop0, logical block 148011137, async page read
Jun  4 21:02:39 ip-172-29-10-99 kernel: [1638976.403495] squashfs: SQUASHFS error: unable to read squashfs_super_block

I have posted the goofys output (with --debug_fuse enabled) for this operation here.

All subsequent attempts to mount similarly fail.

If I remount goofys (without clearing the cache directory), the SquashFS mount succeeds; however, the mount command does not return until the entire SquashFS has been downloaded to disk (which can take an extremely long time for large SquashFS images). I have posted the the first part of the goofys output (with --debug_fuse enabled) for this operation here. As you can see, the debug output shows continual ReadFile requests for 131072 bytes (the default SquashFS block size) for perpetuity until the file is completely downloaded. I don't understand why the file system ops would change so drastically in this case and why the entire file has to be downloaded before file I/O can proceed -- this mitigates the usefulness of goofys for this application.

For reference, I have posted the goofys output (with --debug_fuse enabled) for a successful mount (with catfs disabled) here -- you can see the very limited file I/O that normally occurs for a SquashFS mount and the mount command returns in less than a second. All subsequent file I/O into the SquashFS proceeds satisfactorily.

Any ideas on what causes this weird behavior with catfs?

catfs always runs in foreground

./catfs /mnt/gsdata /tmp/catfsdata /mnt/data

Hangs in foreground, but works (mounted folder exists)

Version: downloaded as 0.8.0, reported 0.7.0 (see #12 )

Caching folder from mounted goofys S3 folder.

How to run it in background?

prefetch_canceled randomly failed

Seems racy. Travis logs:

---- prefetch_canceled stdout ----
1392thread 'prefetch_canceled' panicked at 'assertion failed: `(left == right)`
1393  left: `"f"`,
1394 right: `"file1\n"`', tests/integration_tests.rs:516:13
1395stack backtrace:
1396   0: backtrace::backtrace::libunwind::trace
1397             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
1398   1: backtrace::backtrace::trace_unsynchronized
1399             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
1400   2: std::sys_common::backtrace::_print_fmt
1401             at src/libstd/sys_common/backtrace.rs:77
1402   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
1403             at src/libstd/sys_common/backtrace.rs:59
1404   4: core::fmt::write
1405             at src/libcore/fmt/mod.rs:1052
1406   5: std::io::Write::write_fmt
1407             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/io/mod.rs:1426
1408   6: std::io::impls::<impl std::io::Write for alloc::boxed::Box<W>>::write_fmt
1409             at src/libstd/io/impls.rs:156
1410   7: std::sys_common::backtrace::_print
1411             at src/libstd/sys_common/backtrace.rs:62
1412   8: std::sys_common::backtrace::print
1413             at src/libstd/sys_common/backtrace.rs:49
1414   9: std::panicking::default_hook::{{closure}}
1415             at src/libstd/panicking.rs:204
1416  10: std::panicking::default_hook
1417             at src/libstd/panicking.rs:221
1418  11: std::panicking::rust_panic_with_hook
1419             at src/libstd/panicking.rs:472
1420  12: rust_begin_unwind
1421             at src/libstd/panicking.rs:380
1422  13: std::panicking::begin_panic_fmt
1423             at src/libstd/panicking.rs:334
1424  14: integration_tests::prefetch_canceled
1425             at tests/integration_tests.rs:516
1426  15: integration_tests::prefetch_canceled::{{closure}}
1427             at tests/test_suite/mod.rs:6
1428  16: core::ops::function::FnOnce::call_once
1429             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libcore/ops/function.rs:232
1430  17: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
1431             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/liballoc/boxed.rs:1015
1432  18: __rust_maybe_catch_panic
1433             at src/libpanic_unwind/lib.rs:86
1434  19: std::panicking::try
1435             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/panicking.rs:281
1436  20: std::panic::catch_unwind
1437             at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/panic.rs:394
1438  21: test::run_test_in_process
1439             at src/libtest/lib.rs:539
1440  22: test::run_test::run_test_inner::{{closure}}
1441             at src/libtest/lib.rs:452

On a clean Ubuntu box, fails to build with "the trait `std::default::Default` is not implemented for `std::path::PathBuf`"

sudo apt install cargo pkg-config libfuse-dev && cargo install catfs

error[E0277]: the trait bound std::path::PathBuf: std::default::Default is not satisfied
--> .cargo/registry/src/github.com-1ecc6299db9ec823/catfs-0.4.0/src/evicter/dir_walker.rs:23:23
|
23 | cur_path: Default::default(),
| ^^^^^^^^^^^^^^^^ the trait std::default::Default is not implemented for std::path::PathBuf
|
= note: required by std::default::Default::default

error: aborting due to previous error

error: failed to compile catfs v0.4.0, intermediate artifacts can be found at /tmp/cargo-install.A640JAvfiIKV

Caused by:
Could not compile catfs.

NFS support and thread safety

When using with goofys, can we point catfs to an NFS volume, so that the cache can be shared between multiple users on multiple machines? Are there thread safety issues with multiple concurrent writes? Requesting a note in the README regarding this.

error when compiling on macOS

Is catfs supposted to run on macOS?

Errors below occured when I try to install it with cargo.

error[E0425]: cannot find function `fdopendir` in module `libc`
  --> src/catfs/rlibc.rs:56:29
   |
56 |     let dh = unsafe { libc::fdopendir(fd) };
   |                             ^^^^^^^^^ did you mean `opendir`?
help: possible candidate is found in another module, you can import it into scope
   |
6  | use catfs::rlibc::fdopendir;
   |

error[E0425]: cannot find function `pipe2` in module `libc`
   --> src/catfs/rlibc.rs:178:30
    |
178 |     let res = unsafe { libc::pipe2(p.as_mut_ptr(), libc::O_CLOEXEC) };
    |                              ^^^^^ did you mean `pipe`?

error[E0425]: cannot find function `splice` in module `libc`
   --> src/catfs/rlibc.rs:207:30
    |
207 |     let res = unsafe { libc::splice(fd, off_from_ptr, other, off_to_ptr, len, 0) };
    |                              ^^^^^^ not found in `libc`
help: possible candidate is found in another module, you can import it into scope
    |
6   | use catfs::rlibc::splice;
    |

error[E0425]: cannot find value `AT_EMPTY_PATH` in module `libc`
   --> src/catfs/rlibc.rs:279:66
    |
279 |     let res = unsafe { libc::fstatat(dir, s.as_ptr(), stp, libc::AT_EMPTY_PATH) };
    |                                                                  ^^^^^^^^^^^^^ not found in `libc`

error[E0425]: cannot find function `utimensat` in module `libc`
   --> src/catfs/rlibc.rs:342:30
    |
342 |     let res = unsafe { libc::utimensat(dir, s.as_ptr(), times.as_mut_ptr(), flags as i32) };
    |                              ^^^^^^^^^ did you mean `utimes`?
help: possible candidate is found in another module, you can import it into scope
    |
6   | use catfs::rlibc::utimensat;
    |

error[E0425]: cannot find function `posix_fallocate` in module `libc`
   --> src/catfs/rlibc.rs:435:34
    |
435 |         let res = unsafe { libc::posix_fallocate(self.fd, offset as i64, len as i64) };
    |                                  ^^^^^^^^^^^^^^^ not found in `libc`

error[E0308]: mismatched types
   --> src/catfs/mod.rs:195:21
    |
195 |                     st.f_blocks,
    |                     ^^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
195 |                     st.f_blocks.into(),
    |                     ^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/mod.rs:196:21
    |
196 |                     st.f_bfree,
    |                     ^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
196 |                     st.f_bfree.into(),
    |                     ^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/mod.rs:197:21
    |
197 |                     st.f_bavail,
    |                     ^^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
197 |                     st.f_bavail.into(),
    |                     ^^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/mod.rs:198:21
    |
198 |                     st.f_files,
    |                     ^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
198 |                     st.f_files.into(),
    |                     ^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/mod.rs:199:21
    |
199 |                     st.f_ffree,
    |                     ^^^^^^^^^^ expected u64, found u32
help: you can cast an `u32` to `u64`, which will zero-extend the source value
    |
199 |                     st.f_ffree.into(),
    |                     ^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/catfs/file.rs:598:56
    |
598 |         self.src_file = File::openat(dir, path, flags, mode)?;
    |                                                        ^^^^ expected u32, found u16
help: you can cast an `u16` to `u32`, which will zero-extend the source value
    |
598 |         self.src_file = File::openat(dir, path, flags, mode.into())?;
    |                                                        ^^^^^^^^^^^

error[E0560]: struct `catfs::libc::dirent` has no field named `d_off`
  --> src/catfs/rlibc.rs:89:17
   |
89 |                 d_off: 0,
   |                 ^^^^^^ `catfs::libc::dirent` does not have this field
   |
   = note: available fields are: `d_ino`, `d_seekoff`, `d_reclen`, `d_namlen`, `d_type`, `d_name`

error[E0308]: mismatched types
  --> src/catfs/rlibc.rs:92:25
   |
92 |                 d_name: [0i8; 256], // FIXME: don't hardcode 256
   |                         ^^^^^^^^^^ expected an array with a fixed size of 1024 elements, found one with 256 elements
   |
   = note: expected type `[i8; 1024]`
              found type `[i8; 256]`

error[E0609]: no field `d_off` on type `catfs::libc::dirent`
   --> src/catfs/rlibc.rs:120:24
    |
120 |         return self.en.d_off;
    |                        ^^^^^ unknown field
    |
    = note: available fields are: `d_ino`, `d_seekoff`, `d_reclen`, `d_namlen`, `d_type`, `d_name`

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:158:48
    |
158 |     let res = unsafe { libc::mkdir(s.as_ptr(), mode) };
    |                                                ^^^^ expected u16, found u32

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:168:55
    |
168 |     let res = unsafe { libc::mkdirat(dir, s.as_ptr(), mode) };
    |                                                       ^^^^ expected u16, found u32

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:352:56
    |
352 |     let res = unsafe { libc::fchmodat(dir, s.as_ptr(), mode, flags as i32) };
    |                                                        ^^^^ expected u16, found u32

error[E0308]: mismatched types
   --> src/catfs/rlibc.rs:459:50
    |
459 |         let res = unsafe { libc::fchmod(self.fd, mode) };
    |                                                  ^^^^ expected u16, found u32

error[E0308]: mismatched types
  --> src/catfs/inode.rs:39:15
   |
39 |     match t & libc::S_IFMT {
   |               ^^^^^^^^^^^^ expected u32, found u16

error[E0308]: mismatched types
  --> src/catfs/inode.rs:40:9
   |
40 |         libc::S_IFLNK => fuse::FileType::Symlink,
   |         ^^^^^^^^^^^^^ expected u32, found u16
   |
   = note: expected type `u32`
              found type `u16`

error[E0308]: mismatched types
  --> src/catfs/inode.rs:41:9
   |
41 |         libc::S_IFREG => fuse::FileType::RegularFile,
   |         ^^^^^^^^^^^^^ expected u32, found u16
   |
   = note: expected type `u32`
              found type `u16`

error[E0308]: mismatched types
  --> src/catfs/inode.rs:42:9
   |
42 |         libc::S_IFBLK => fuse::FileType::BlockDevice,
   |         ^^^^^^^^^^^^^ expected u32, found u16
   |
   = note: expected type `u32`
              found type `u16`

error[E0308]: mismatched types
  --> src/catfs/inode.rs:43:9
   |
43 |         libc::S_IFDIR => fuse::FileType::Directory,
   |         ^^^^^^^^^^^^^ expected u32, found u16
   |
   = note: expected type `u32`
              found type `u16`

error[E0308]: mismatched types
  --> src/catfs/inode.rs:44:9
   |
44 |         libc::S_IFCHR => fuse::FileType::CharDevice,
   |         ^^^^^^^^^^^^^ expected u32, found u16
   |
   = note: expected type `u32`
              found type `u16`

error[E0308]: mismatched types
  --> src/catfs/inode.rs:45:9
   |
45 |         libc::S_IFIFO => fuse::FileType::NamedPipe,
   |         ^^^^^^^^^^^^^ expected u32, found u16
   |
   = note: expected type `u32`
              found type `u16`

error[E0277]: no implementation for `u32 & u16`
  --> src/catfs/inode.rs:39:13
   |
39 |     match t & libc::S_IFMT {
   |             ^ no implementation for `u32 & u16`
   |
   = help: the trait `std::ops::BitAnd<u16>` is not implemented for `u32`

error[E0308]: mismatched types
   --> src/catfs/inode.rs:131:31
    |
131 |             kind: to_filetype(st.st_mode),
    |                               ^^^^^^^^^^ expected u32, found u16
help: you can cast an `u16` to `u32`, which will zero-extend the source value
    |
131 |             kind: to_filetype(st.st_mode.into()),
    |                               ^^^^^^^^^^^^^^^^^

error[E0308]: mismatched types
   --> src/evicter/mod.rs:104:50
    |
104 |         DiskSpace::Percent(p) => ((st.f_blocks * st.f_frsize) as f64 * p / 100.0) as u64,
    |                                                  ^^^^^^^^^^^ expected u32, found u64

error[E0277]: cannot multiply `u64` to `u32`
   --> src/evicter/mod.rs:104:48
    |
104 |         DiskSpace::Percent(p) => ((st.f_blocks * st.f_frsize) as f64 * p / 100.0) as u64,
    |                                                ^ no implementation for `u32 * u64`
    |
    = help: the trait `std::ops::Mul<u64>` is not implemented for `u32`

error[E0308]: mismatched types
   --> src/evicter/mod.rs:108:37
    |
108 |     let x = desired - (st.f_bfree * st.f_frsize) as i64;
    |                                     ^^^^^^^^^^^ expected u32, found u64

error[E0277]: cannot multiply `u64` to `u32`
   --> src/evicter/mod.rs:108:35
    |
108 |     let x = desired - (st.f_bfree * st.f_frsize) as i64;
    |                                   ^ no implementation for `u32 * u64`
    |
    = help: the trait `std::ops::Mul<u64>` is not implemented for `u32`

error: aborting due to 32 previous errors

error: failed to compile `catfs v0.8.0`, intermediate artifacts can be found at `/var/folders/8j/gcq2wfcj41jbn45x071shl4h0000gn/T/cargo-install.75RhyeDLPNLv`

how to set desired permission on the cached files ?

hi,

i am trying to set a 0644 permission on the cached file, but it seems its hard coded to 0640? how to fix this ?

Downloaded as 0.8.0, reported 0.7.0

wget https://github.com/kahing/catfs/releases/download/v0.8.0/catfs
chmod a+x ./catfs
./catfs -h

catfs 0.7.0
Cache Anything FileSystem
...

ls -la catfs

-rwxrwxr-x 1 ubuntu ubuntu 6453456 Jan 9 06:49 catfs

Seems to be just a typo, as 0.7.0 downloaded has another size.

how to run your benchmark in the VM not in docker?

problem described as the title

handle ENOSPC

perhaps by forcing evicter to run?

error: non-binding let on a synchronization lock

Due to a new lint in rust 1.65 introduced by rust-lang/rust#97739, there is a new compilation error:

error: non-binding let on a synchronization lock
   --> src/catfs/file.rs:570:13
    |
570 |         let _ = self.page_in_res.0.lock().unwrap();
    |             ^   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this binding will immediately drop the value assigned to it
    |             |
    |             this lock is not assigned to a binding and is immediately dropped
    |
    = note: `#[deny(let_underscore_lock)]` on by default
help: consider binding to an unused variable to avoid immediately dropping the value
    |
570 |         let _unused = self.page_in_res.0.lock().unwrap();
    |             ~~~~~~~
help: consider immediately dropping the value
    |
570 |         drop(self.page_in_res.0.lock().unwrap());
    |         ~~~~~                                  +

(Ignore the line numbers as they correspond to an older version, but I was able to verify it still occurs in v0.9.0 and the code triggering the error still exists.)

I was planning to fix this, but I can't figure out what the purpose of that line is. Is this mutex being used to gate access to more things than its internal data?

Add macOS Travis builder

Now that #45 has merged, CI should include it. Note that this may be difficult since Homebrew is removing support for FUSE.

Pass-through FUSE mount options

Mount options to FUSE appear to be hard-coded (eg: default_permissions)

This makes it difficult to expose a catfs mount for use in a docker container from the host; docker tries using it's overlayfs as root, but fails since the catfs mount isn't mounted with allow_other or allow_root.

Here's an example of what I'm trying to run with:

RUST_LOG=catfs=debug,fuse=debug,integration_tests=debug \
    /usr/local/bin/catfs /mnt/source /data/cache /mnt/dest \
    -o rw,allow_other,umask=002,gid=1000,uid=1000,syslog,dev,suid

## Output from `catfs`
2017-09-17 07:28:54 DEBUG - catfs "/mnt/source" "/data/cache"
2017-09-17 07:28:54 INFO  - Mounting /mnt/dest
2017-09-17 07:28:54 DEBUG - INIT(1)   kernel: ABI 7.26, flags 0x1ffffb, max readahead 131072
2017-09-17 07:28:54 DEBUG - INIT(1) response: ABI 7.8, flags 0x31, max readahead 131072, max write 16777216

##  Output from `mount`
/dev/fuse on /mnt/dest type fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,default_permissions)

catfs installation trouble: failed to run custom build command for `fuse v0.3.1`

I hope you can help, on an up to date Alma Linux system after # cargo install catfs

error: failed to run custom build command for `fuse v0.3.1`

Caused by:
  process didn't exit successfully: `/tmp/cargo-installu2WIRp/release/build/fuse-d71146d524bd1323/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-env-changed=FUSE_NO_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG
  cargo:rerun-if-env-changed=PKG_CONFIG
  cargo:rerun-if-env-changed=FUSE_STATIC
  cargo:rerun-if-env-changed=FUSE_DYNAMIC
  cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
  cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_PATH
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-linux-gnu
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_linux_gnu
  cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
  cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR

  --- stderr
  thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: `"pkg-config" "--libs" "--cflags" "fuse" "fuse >= 2.6.0"` did not exit successfully: exit status: 1
  error: could not find system library 'fuse' required by the 'fuse' crate

  --- stderr
  Package fuse was not found in the pkg-config search path.
  Perhaps you should add the directory containing `fuse.pc'
  to the PKG_CONFIG_PATH environment variable
  Package 'fuse', required by 'virtual:world', not found
  Package 'fuse', required by 'virtual:world', not found
  ', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/fuse-0.3.1/build.rs:10:76
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: failed to compile `catfs v0.8.0`, intermediate artifacts can be found at `/tmp/cargo-installu2WIRp`

[edit] I saw 'could not find system library 'fuse' required by the 'fuse' crate'

but but..

dnf install fuse

Last metadata expiration check: 3:36:31 ago on Wed 20 Jul 2022 12:19:02 PM CEST.
Package fuse-2.9.7-15.el8.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!

Also installed:

fuse.x86_64                                                       2.9.7-15.el8                                               @baseos
fuse-common.x86_64                                                3.3.0-15.el8                                               @baseos
fuse-libs.x86_64                                                  2.9.7-15.el8                                               @baseos
fuse3.x86_64                                                      3.3.0-15.el8                                               @baseos
fuse3-libs.x86_64                                                 3.3.0-15.el8                                               @baseos
nbdfuse.x86_64                                                    1.6.0-5.module_el8.6.0+2880+7d9e3703                       @appstream
s3fs-fuse.x86_64                                                  1.91-1.el8                                                 @epel
btrfs-fuse.x86_64                                                 0-5.20211113git8635fbc.el8                                 epel
fuse-afp.x86_64                                                   0.8.1-35.el8                                               epel
fuse-devel.i686                                                   2.9.7-15.el8                                               baseos
fuse-devel.x86_64                                                 2.9.7-15.el8                                               baseos
fuse-dislocker.x86_64                                             0.7.3-2.el8                                                epel
fuse-encfs.x86_64                                                 1.9.5-5.el8                                                epel
fuse-libs.i686                                                    2.9.7-15.el8                                               baseos
fuse-overlayfs.x86_64                                             1.8.2-1.module_el8.6.0+2878+e681bc44                       appstream
fuse-zip.x86_64                                                   0.7.2-1.el8                                                epel
fuse3-devel.i686                                                  3.3.0-15.el8                                               baseos
fuse3-devel.x86_64                                                3.3.0-15.el8                                               baseos
fuse3-libs.i686                                                   3.3.0-15.el8                                               baseos
glusterfs-fuse.x86_64                                             6.0-56.4.el8                                               baseos
gvfs-fuse.x86_64                                                  1.36.2-11.el8                                              appstream

split out RError as a separate package

Code reuse: Would you mind dual licencing the code as Apache 2 / MIT?

I am currently designing a file system agnostic automated tiered storage solution in rust, which I plan to release under the same MIT/Apache2 dual licence rustlang uses.

I could safe quite a lot of time If I was able to reuse some catfs code, but to do so catfs would need to be under the same dual license.

Do you mind dual licensing catfs as Apache2/MIT? I would highly appreciate it.

Error installing catfs into AWS graviton server

Hello everybody.

I need your help, please.
I have been trying install Catfs into AWS Graviton instance but i was found that error. I wonder If you could help me with this.

   Compiling catfs v0.8.0
error[E0308]: mismatched types
  --> /home/carlos.zrm/.cargo/registry/src/github.com-1ecc6299db9ec823/catfs-0.8.0/src/catfs/rlibc.rs:92:26
   |
92 |                 d_name: [0i8; 256], // FIXME: don't hardcode 256
   |                          ^^^ expected `u8`, found `i8`
   |
help: change the type of the numeric literal from `i8` to `u8`
   |
92 |                 d_name: [0u8; 256], // FIXME: don't hardcode 256
   |                          ^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0308`.
error: failed to compile `catfs v0.8.0`, intermediate artifacts can be found at `/tmp/cargo-installtpQEXa`

Caused by:
  could not compile `catfs`

Versions

Cargo version

rustc 1.54.0 (a178d0322 2021-07-26)

Linux version

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.2 LTS"
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"

CPU version
AWS instance m6g.medium

Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          1
On-line CPU(s) list:             0
Thread(s) per core:              1
Core(s) per socket:              1
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       ARM
Model:                           1
Model name:                      Neoverse-N1
Stepping:                        r3p1

writeback support

the "simple" implementation would be logging all the dirty files and copy back in the background, while making sure they are never evicted. Ideally provide ways to let the user know how much dirty data are yet to be replicated

Hang when encountering link to own directory

I use (or try to, not entirely set-up yet) catfs together with mergerfs. My directory structure is:

/mnt/disk1-3 merged into /mnt/uncached_storage
/mnt/cache is an ssd
/mnt/storage is /mnt/uncached_storage with /mnt/cache as the cache.

When setting things up for another application I mistakenly created a link to a directory inside that same directory. For example:

ln -s /mnt/storage/Documents /mnt/storage/Documents/Documents

When trying to ls /mnt/storage/Documents then, it hangs completely.

metadata cache

Not working with fstab

Hello guys and thank you for this great package! I tried adding pretty much exactly the same command to my /etc/fstab, but it refuses to work with it, getting this:

error: The following required arguments were not provided:
    <mountpoint>

USAGE:
    catfs <from> <to> <mountpoint> -o <option>...

For more information try --help

v0.8.0

Ignoring unknown FUSE operation 39

Tried running catfs with gooyfs and is seeing this error message repeatedly.
Seems like this is under a feature flag for fuse crate and no longer being updated.

fstab syntax not workign

Hey, when trying to moutn via fstab I'm getting an error:

root@vodnl:~# mount /mnt/cephfs                                                                                                                                                                                                                                                                                                                
error: The following required arguments were not provided:
    <mountpoint>

USAGE:
    catfs <from> <to> <mountpoint> -o <option>...

FStab line :

catfs#/mnt/cephfs_real#/mnt/storage/cephfs_cache        /mnt/cephfs         fuse    allow_other,--uid=1001,--gid=1001,--free=1%,_netev,noatime   0       0

Re-enable fsx tests

I removed these to unblock CI in c940bd8.

check_dirty randomly failed

Seems racy. Logs:

---- check_dirty stdout ----
thread 'check_dirty' panicked at 'assertion failed: !xattr::get(&foo_cache, "user.catfs.src_chksum").unwrap().is_some()', tests/integration_tests.rs:393:13
stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:505
   1: integration_tests::check_dirty
             at ./tests/integration_tests.rs:393
   2: integration_tests::check_dirty::{{closure}}
             at ./tests/test_suite/mod.rs:6
   3: core::ops::function::FnOnce::call_once
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/ops/function.rs:227
   4: core::ops::function::FnOnce::call_once
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/core/src/ops/function.rs:227

Travis CI fails with beta and nightly

Example:

https://travis-ci.org/kahing/catfs/builds/640443369

The following required arguments were not provided: <mountpoint>

./catfs /mnt/ampache-base#/mnt/ampache-cache /mnt/ampache -o  allow_other,--uid=1001,--gid=1001,--free=1%
error: The following required arguments were not provided:
    <mountpoint>

This is likely fixed in un-released 0.9.0.

Does not follow stated behavior in README: "Entire file is cached if it's open for read"

I am using goofys + catfs and observing that if I read the first N bytes of a file, it seems to cache only those bytes and not the whole file as stated in the README.

Mount command:

goofys --region us-west-2 --file-mode 0440 -o allow_other --cache /tmp/test_dir_cache s3_bucket_name:test_dir /tmp/test_dir_mount

Example of reading first N bytes (same behavior happens when reading from a python script:

head --bytes 100 /tmp/test_dir_mount/large_file.mp4

The size of the large file here is 158M:

ls -halp /tmp/test_dir_mount/large_file.mp4

However the size of the cached file after the read is 128K:

ls -halp /tmp/test_dir_cache/large_file.mp4

I have tested this with multiple files and reads from 10 bytes up to 100M, resulting in the same behavior of only caching the read part. If instead the last N bytes are read, the entire file is cached.

This seems to contradict the README, which states Entire file is cached if it's open for read, even if nothing is actually read..

My desired behavior would be to have a flag that toggles these two behaviors, and appropriate documentation. As it currently stands, one of our use cases is failing because it depended on caching the entire file on touching it.

Finally, please comment if this should be opened as a goofys issue instead.

System:

goofys version 0.24.0-45b8d78375af1b24604439d2e60c567654bcdf88
catfs 0.9.0 (also tried 0.8.0 and 0.7.0)
Linux 4.15.0-142-generic #146~16.04.1-Ubuntu SMP Tue Apr 13 09:27:15 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

version `GLIBC_2.18' not found (required by catfs)

I'm trying to use catfs along with goofys both latest versions as of now on centOS 7.5 , however, when i try to mount with cache enabled I receive this error:

Invalid value "/tmp/cache" for --cache: catfs: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by catfs)

the version exist with centos: glibc-2.17-222.el7.x86_64

any idea how to solve this?

File appears truncated

Scenario: a remote file changes.
Problem: While the cache content is good and up-to date, the view presented at the mountpoint (third argument to catfs) is truncated. The file is there, but it's only partially present if I display it via cat. Other times a bunch of NULL bytes appear at the end.

So the cache directory content is fine, but through the mountpoint I get garbage.

Add tests for files > 2 GB

This should exercise pwrite64, etc. code paths on 32-bit platforms like Raspberry Pi.