We have a lot of sleep calls that could be faster.

Write a tool to detect when a mount is ready. (Current benchmarks do this by busy looping on umount.) Use its exit status to determine when it's okay to proceed on the mount and proceed checking that process has completed.

It'll also be useful to have a way to pad a few milliseconds on, i.e., unmount means the process may not be done yet. The current utils/timeout script is okay, but (a) it's noisy, and (b) it'd be nice to not fix it on ffs when we start. Maybe a waitfor command that takes a time limit before killing the process?

Rather than taking a mountpoint as an argument, use an optional --mount flag.

If the input is over STDIN, you have to specify a --mount.

If the input is from a file foo.EXT, then try to make a directory foo. If that file already exists, give up and tell the user about --mount. If it doesn't, great: do that. When things unmount, double check the directory is empty and then remove it.

Need to revise all the tests, and then test a few configurations of this feature.

When running on the citylots JSON file, parsing takes a long, long time. Way longer than I expected---since I was told to "expect in the ballpark of 500 to 1000 megabytes per second deserialization". I suspect I'm doing something wrong.

Relatedly, lazy loading only saves time on loading, not parsing. Can we use serde_json::RawValue and similar tricks to parse but defer construction? I'm not sure every format supports this.

Keep a free list, and reuse inode numbers.

An idea from @angelhof: notice directories where every file has numerical name suffixes (e.g., role0, role1, or simpliciter 0, 1, 2, ...) and automatically treat them as lists.

The best way to do this is to use the auto type for directories, too, and new directories have that type assigned.

• choose target ba64e76
• actually output ba64e76
• fsync causes output (see #4) doesn't seem to actually work; see 867cb6b for broken test

Would be good to build a .deb file for this.

https://github.com/mmstick/cargo-deb seems promising.

Make a little doc/gh-pages directory. Include one of those cute screencast GIFs.

Just load/unload data directly on the filesystem.

Progress indicator. https://docs.rs/indicatif/0.16.2/indicatif/

Multithreading? (Is that ever a performance win?)

Pack

BFS?

Unpack

Flag to follow symlinks, possibly even out of the root.

-i flag for prompting on weirdness.

A --live flag to sync the JSON out on every write.

A --watch flag to notice updates to the JSON file and rebuild the FS.

With lazy/incremental, this would be complicated.

Trying to mount a JSON file within the virtual filesystem fails (which is ok, I guess):

raphael@tukk ~/Downloads> ./ffs -i ffs1.json
raphael@tukk ~/Downloads> cat ffs1.json
{"option":"test"}
raphael@tukk ~/Downloads> cp ffs1.json ffs1/
raphael@tukk ~/Downloads> ls ffs1/
ffs1.json option
raphael@tukk ~/Downloads> ./ffs -i ffs1/ffs1.json
fusermount: bad mount point ffs1/ffs1: Permission denied
thread 'main' panicked at 'called Result::unwrap() on an Err
value: Os { code: 2, kind: NotFound, message: "No such file or
directory" }', src/main.rs:38:41
note: run with RUST_BACKTRACE=1 environment variable to display a
backtrace

This actually creates the subdirectory ffs1/ffs1 but does not mount anything there. Unmounting the filesystem results in an additional entry "ffs1: {}" in the json file. I'd suggest that a failed command should not leave behind any new files/members.

• Instrument all Filesystem trait methods c910624, improved in 2c01cde
• Clean stderr output for error! and warn! bcd2609
• Quiet mode cb388da

Started working in https://github.com/mgree/ffs/tree/homebrew.

Really, no idea what I'm doing. Not sure how to depend on a cask (macfuse) and I have the sinking feeling that you can't. Oy.

Via https://crates.io/crates/winfsp ?

then saved backed into JSON as "space" 👎🏻

There are plenty of interesting options on https://man7.org/linux/man-pages/man8/mount.fuse3.8.html.

• default_permissions
• allow_other
• auto_unmount always on? a02d674
• kernel_cache

• access, fallocate
• multi-user stuff
• error code coverage
• TOML 476f3a1 and 8cac6d2
• run testing in parallel 2ec8fa6

• fsync (need output) 9d0ae6d
• setattr (needed for basic bash use, cp, etc)
  • chmod 5f46ac4
  • chown feature in c7ab088 testing in c79cd35 with an incredible saga in between
  • truncate 7dc761a
  • utimens 9a58dee

Request: https://twitter.com/wtfpdf/status/1409883313888804868

Promising lead on a Rust library: https://github.com/J-F-Liu/lopdf with Document::load and Document.*

$> touch x.json
$> ffs x.son
thread 'main' panicked at 'JSON: Error("EOF while parsing a value", line: 1, column: 0)', src/format.rs:187:53
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

Investigate weird unmounts on macOS, producing:

WARN fuser::mnt::fuse2: umount failed with Os { code: 22, kind: InvalidInput, message: "Invalid argument" }

See actions log. Linux seems fine.

• binary data via base64 2f483ea
• TOML 688b78a
• YAML 94917ed
• XML moved to #15
• Generic framework (detect/parse/unparse) c9198f9
• choose output format independently of input format c9198f9
• autodetection of formats
  • extension
  • speculation moved to #16
• CLI options for specifying formats d441958

• Options for naming of ListDirectory elements (prefix/suffix? format string?)
• Metadata
  • extensions
  • dotfiles
  • xattrs in #30

🔢

Lazy loading (scan but don't parse into structures until demanded). #50

Ahead-of-time mappings (literally build/unbuild FS).

Use 9p, per Jay McCarthy. Or use the Bento FS thing?

Timing: use https://crates.io/crates/tracing-timing?

Microbenchmarks

• read
• write

Macrobenchmarks

• tasks?

Compare against

• FS (various hdd configs) as baseline
• ramdisk as memory FS baseline
• jq
• js
• python
• native rust

For example i3blocks supports multiple signals which you can trigger using pkill -SIGRTMIN+10 i3blocks, which then causes an action.

Could you add atleast two signals, one for re-reading INPUT file and another for writing OUTPUT file.
This would open ffs for uses in scripts and would allow updating INPUT and/or OUTPUT files without needing to restart ffs.

The JSON object { ".": 5 } will create a file named dot holding the number 5... but it will dump back to JSON without restoring the name.

The best solution is probably to track an "unmunged" name, and files that don't undergo a rename call go back exactly how they were.

This unmunged name is metadata, so the appropriate mechanism for working with it is up in the air (see #2).

Several format renderers can put the output in a nice, human-readable format. Why not treat yourself? We all deserve nice things.

Per cberner/fuser#153 (comment), better to rely on the explicit contract than on the the current implementation in fuser calling mem::drop.

• completions for bash, fish, zsh 1af11e1
• manpage 93ecc0f
• cargo crate 1e4ec41
• releases from workflow 96a3126

On a malformed JSON file:

: mgree@rocinante:~/talks/20211025plos [master] ; ~/ffs/target/release/ffs -i demo.json 
thread 'main' panicked at 'JSON: Error("expected value", line: 7, column: 12)', src/format.rs:322:45
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

https://twitter.com/laurencetratt/status/1451198313085030407 seems to indicate a way forward. OO!

Saves you some trouble (e.g., passing {} in on STDIN).

Depends on #2 in part.

Use a short buffer and some heuristic (xdg-mime or infer, but really probably custom heuristics) to detect file types.