Goroutine race breaks guarantees on write -> flush barrier about fuse

[jacobsa]

So what to do about this? Some options:

• Do nothing. Don't make the "writes before flushes" guarantee to the file system implementer. This sucks, let's not do it.
• Process all requests serially. This is nice and simple, but could be very slow.
• Set up a barrier: wait for responses to all write requests for a given handle to complete before passing on a flush for that handle, potentially blocking the pipeline behind. Document this publicly.
• Change the API so we don't call blocking functions but rather just pass on requests, either as a channel or a read method or asynchronous function calls.

[jacobsa]

This can be made very easy to reproduce by sticking a time.Sleep(time.Second) at the top of the bazilfuse.WriteRequest case in server.go. It makes FlushFSTest.Mmap_MunmapBeforeClose fail every time.

The same is not true of FlushFSTest.CloseReports_ReadWrite, where it appears that each write(2) call is blocking on the fuse write request to finish. This surprises me; I thought writes were supposed to happen asynchronously via the page cache.

[jacobsa]

Closing the loop: I think perhaps the reason that we wait for write(2) is because we don't respond to Init with bazilfuse.InitWritebackCache, corresponding to FUSE_WRITEBACK_CACHE in fuse_kernel.h. This was apparently added in kernel 3.15, so I can't verify on my 3.13 machine.

[jacobsa]

Current favored solution:

• Kill the FileSystem interface.
• Instead, have an Op interface with methods Context() context.Context and Respond(error).
• Concrete implementations are structs with both request and response fields.
• Take this opportunity to put them in a fuseops sub-package. For example, fuseops.ReadOp.
• Have a Connection type that exposes a read method for a stream of Ops, directly reflecting /dev/fuse.
• Move existing method and struct comments onto the op structs.
• Mount function accepts serveConn func(*Connection).
• Kill MountedFileSystem.WaitForReady; instead just in Mount (and document this).
• Audit package and README comments, update where necessary.

[jacobsa]

This is blocked by #4; we want tests that we can run many times to confirm we've fixed the issue.

[jacobsa]

All done with 336525b.