cannot pass mount options like direct_io / max_readahead about fuse

[jacobsa]

Can you elaborate on what you're trying to do with these/why you need them? Setting arbitrary fuse options is dangerous in general—for at least some of them, the file system daemon code needs to know that they are set and cooperate accordingly.

[kahing]

I am actually writing my own file system (https://github.com/kahing/goofys). I tested with gcsfuse as well to make sure it's not something I am doing incorrectly.

[jacobsa]

Sorry, I'm not sure what you mean. What are you testing, and what do you think you might have done incorrectly?

[kahing]

I am not doing anything incorrectly. I want to pass additional fuse options to my own fuse based filesystem through this fuse library. I checked with gcsfuse as well to make sure that the limitation is within this fuse library, and not the way I am using it.

[jacobsa]

Sorry, I was confused; I thought this bug was in the gcsfuse project. Could you give me some code for what you're trying to do?

[jacobsa]

And also please describe which options you're trying to use, and why.

[kahing]

the fuse options parsing and passing code is copied from gcsfuse. I want to experiment with -o big_writes and -o max_readahead and how it affects performance.

[jacobsa]

Could you please post a minimized snippet that shows what you mean? I'm familiar with the gcsfuse code, but bringing that all into scope is just asking for us to misunderstand each other, to not be able to understand this thread in six months, etc.

[lrowe]

goofyfs is passing options down as MountConfig.Options, so they end up in the fusermount command arguments, e.g.: {"fusermount", "-o", "default_permissions,fsname=bucket-name,direct_io", "--", "/mnt"}

That fails with mount helper error: fusermount: mount failed: Invalid argument because many fuse options (including direct_io) are implement by the shared library lib/fuse.c rather than in util/fusermount.c.

[jacobsa]

Thanks for the details. Yes, I come to the same conclusion: despite some confusion in the fuse documentation, some of these options are implemented in the library <-> kernel protocol, and some in fusermount.

Which brings me back to my question: if you want support for particular options in this package, please describe which options and why you want/need them.

[lrowe]

The direct_io option would allow for reads larger than the 128KB / 32 page limit imposed by FUSE_MAX_PAGES_PER_REQ. (See: http://sourceforge.net/p/fuse/mailman/message/11633623/)

However I believe it may be necessary to implement buffering in the fuse driver as tools like gunzip read in fixed 32KB chunks.

[jacobsa]

Direct I/O does allow for that, but I'm trying to understand the underlying problem: why do you need to read more than 128 KiB at a time?

When you say "fuse driver" in your second sentence, I assume you mean in your file system implementation. If so, I agree—direct I/O isn't a panacea for high-latency file systems, since users are still going to size their buffers as if they were talking to local disk.

[kahing]

I initially thought jacobsa/fuse was not passing the mount options correctly to kernel, but if the problem is only with options that are implemented outside of the fuse kernel driver, then the onerous is on me to show the benefits.

[jacobsa]

Yeah, I think this is a matter of needing to write code to make it happen, not just passing along strings.

I don't mean to just blindly push back, by the way—I'm genuinely curious why you want this. I experimented with this and several other performance-related fuse options awhile back, and settled on what seemed to be best for my use. But I'm not at all confident I got it right, especially since the documentation for these things is so sparse.

[kahing]

Right I wanted the fuse options precisely so that I could experiment with them. I am not sure to what extents they would help.

[jacobsa]

I see. Your best bet is then to fork and modify the code, then send a pull request if it's actually helpful. (And I would love to hear about the results.)