whatwg / fs Goto Github PK

Migrated from WICG/file-system-access#173 (see #2)

It should be easy to read only files or subfolders from a parent folder, and have a recursive option to return the contents of subfolders. Example:

try {
 for await(let file_reference of folder_reference.read({ files: true, folders: false, recursive: true })) {
  console.log(file_reference)
 }
} catch(error) {
 console.error(error)
}

🤖 This is an automatic issue report for Web IDL syntax error. 🤖

webidl2.js says:

Syntax error at line 4 in fs,4, since `dictionary FileSystemReadWriteOptions`:
 long long at = 0;
                 ^

Error: Required member must not have a default

Please file an issue at https://github.com/saschanaz/webidl-updater/issues/new if you think this is invalid or should be enhanced.

I'm a little confused by the fact that closing is mentioned many times as a concept, it appears there is a close() method that I'm calling successfully under Chrome, and yet I don't see any actual explicit documentation of a close() method.

Our web app downloads large application specific static data files. They range in the 10kb to several hundred kb size. We use a variety of approaches to cache this data. We would prefer to store them in OPFS, and, to be able to share them between multiple tabs of our app. For example, if the user opens two tabs to our app, we want to have a single copy of the data. We would like to download the data once, and then be able to read using the OPFS access handles from multiple tabs. We prefer access handles for their good performance.

Looks like https://fs.spec.whatwg.org/#filesystemdirectoryhandle-resolve-example uses some non-standard (Chrome-only) showOpenFilePicker. Would be good to remove use of such features in the specs, even in examples.

Besides read and write which are sync methods, there are async methods truncate, getSize, flush and close, returning promises.

Does it make sense for the dedicated interface available only in dedicated workers to have async methods ?

The language for getting FileHandles indicates you check the 'query access' for the file using readwrite if create is true, and read if create is false. (though it also says you don't need to implement this if you don't implement any dependent specs).

a) Does this mean you can't write to a file unless you pass create: true?
b) "not implementing any dependent specs" seems incredibly vague. They should be called out directly, or even better yet it should indicate under what circumstances it's needed to check the 'query access' in more concrete form, and then this rule can be applied to any current or future dependent specs.

We had a call to discuss #21 synchronously. Apologies for not announcing it here ahead of time as per https://whatwg.org/working-mode#meetings, though rest assured that nothing from the meeting is binding.

• #21, rendered: https://whatpr.org/fs/21.html

• Randell: odd that async AccessHandle is missing

  • Emanuel: needs more community discussion, but I’d be happy to move forward with the tentative proposal we have.
  • Austin: it’s something we want to add. We might also want to deprecate sync AccessHandle…
  • Randell: it’s a bit odd to specify them and then immediately deprecate.
  • Emanuel: I suspect that’ll take a long time, especially given where Wasm is at.

• Randell: on sync AccessHandle, why are some accessors async?

  • Emanuel: context is that there’s sorta two sides. Wasm wants synchronous. Web API designers prefer less synchronous APIs.
  • Randell: sounds reasonable for compat with the future async AccessHandle.
  • Anne: does it queue a task?
  • Austin: it should.
  • ACTION: Anne to raise this as a spec comment.

• Randell: how is quota supposed to be handled?

  • Austin: in our impl the write would fail due to quota limits.
  • Emanuel: write() can fail due to quota limits if you write beyond the file size. And truncate() can fail if you increase the file size.
  • Randell: how do you deal with partial writes?
  • Emanuel: in principle quota is checked upfront and we reject upon reaching the limit, if it’s within the limit but the write failed anyway, number of successfully written bytes are returned.
  • Randell: I think it’s important that if you have written something, you return a number. Otherwise the application has to assume the file is hosed.
  • ACTION: Emanuel to make sure this is in the spec.
  • ACTION: Google to ensure it’s what they implemented.

• Randell: I think the locking concern got resolved.

• Emanuel: there were a couple of changes we were discussing internally I’d like to bring up.

  • Reading and writing takes a non-optional options parameter. I would like to make that parameter optional.
  • ACTION: Emanuel to fix the spec.

• Emanuel: how to deal with SAB and races

  • Anne: not my area of expertise, but from what I recall this might be UB in C++. Henri Sivonen would be a good person to ask.
  • ACTION: Everyone to ask around internally to sort out what’s best here.

• Randell: AOB?

  • Randell: is there anything we might run into?
  • Emanuel: I don’t think anyone relies on a scary corner case. I think it should work once you implement it.

• Randell: how much quota do applications expect?

  • Austin: in Chrome we allow any site up to 60% of your disk.
  • Randell: I believe we are more conservative.

• Anne: not sure we raised this, but what about “valid file name”? How does that work for OPFS?

  • There’s also a problem with long paths on Windows.
  • Randell: thus far we hash things and store names in a side table.
  • Austin: that matches our implementation.
  • ACTION: Randell to file an issue on file names.

• Anne: publish minutes in an issue? [Agreement]

• Austin: there’s also PRs for remove/move and need locking.

  • Randell: exclusive locks I assume?
  • Austin: yup.
  • Randell: if we add move, we should also specify the implications for existing handles to that file.
  • Anne: I think it requires making it more explicit that handles are essentially paths.
  • Austin: need to talk to Marijn.
  • ACTION: Austin to talk to Marijn and sort out behavior of handles.

Any issues that apply to the text in this standard filed at https://github.com/WICG/file-system-access/issues should be moved over.

Migrated from WICG/file-system-access#319 (see #2)

@ArthurSonzogni

You may want to reference this from the BFCache meta bug:
whatwg/html#5880
+CC @rakina

I was wondering what kind of interaction this feature will have with the Back-forward cache. I see the execution context can obtain a lock for a file. Then the document might be (?) able to enter the BackForwardCache without releasing it. Maybe this could cause some problems?
For instance there are other features like WebLock which prevents the BFCache from being used. Maybe the same should happen here:
https://source.chromium.org/chromium/chromium/src/+/main:content/browser/renderer_host/back_forward_cache_impl.cc;l=265;drc=e5616aeff413a17175a96056b3bf1fbc9ca0ade7;bpv=1;bpt=1
Maybe you did it already. I see kWebFilesystem. Not sure if this is related.

More generally, you might want to clarify if there are any BFCache specific behavior about this feature.

@rakina

On the kWebFilesystem thing, I think @hajimehoshi and @fergald did add WebFileSystem support for BFCache, but I'm not sure about the details - maybe the can comment on what Chrome is doing and if there's anything to be added on the spec side.

@fergald

My understanding is that WebFileSystem had nothing that prevented bfcaching (there are no connections to bring up/down or locks). I see that it's unblocking is still controlled by a flag but we should clean that up and leave it unconditionally unblocked.

I don't think there's any spec change (unless we want to state positively that it is compatible with bfcache) but we should probably add a WPT that checks if a page using WFS is cached. That would make it clear which browsers support it.

@mkruisselbrink

On the chrome side currently using the file system access API with local files on the file system will cause the page to be prevented from entering bfcache, although I think that is primarily since we haven't reased about interaction with usage tracking etc for our current permissions implementation (where lifetime of permissions is dependent on the presence of tabs with pages loaded).

For the newly proposed access handle API, which include file locking, I do suspect we'll need explicit spec-level integration with bfcache. We don't want pages in the bfcache to be able to keep files locked after all.

This behavior is currently not well specified

Hey 👋🏾,
so recently I see the new writable stream interface that the API exposes and immediately thought about streaming responses to files kind of like how it's done in Node e.g.

const http = require('http');
const fs = require('fs');
const path = require('path');

const file = fs.createWriteStream('download.jpg');
http.get('http://i3.ytimg.com/vi/J---aiyznGQ/mqdefault.jpg', function (response) {
    response.pipe(file);

    file.on("finish", () => file.close());
});

In the web to get a similar effect you'd have to do:

const newHandle = await window.showSaveFilePicker({suggestedName: "video.mp4"});
const writableStream = await newHandle.createWritable();

fetch('./video.mp4')
.then(async (response) => {
    const reader = response.body.getReader();
    let isDone = false;
    let position = 0;
    
    progressContainer.style.display = 'unset';
    return new Promise((resolve) => {
        async function download(deadline) {
            while(!isDone && deadline.timeRemaining() > 0) {
                const {done, value} = await reader.read();
                isDone = done;
                

                if (isDone) {
                    return resolve(true);
                }
                writableStream.write({type: "write", position, data: value});
                position += value.length;
            }

            requestIdleCallback(download);
        }

        requestIdleCallback(download);
    });

}).then(() => {
    writableStream.close();
});

Here we use requestIdleCallback to make sure we yield to the browser and not block the main thread. You can probably see where I'm going with this, since this could be simplified by doing something like:

// ...
const newHandle = await window.showSaveFilePicker({suggestedName: "video.mp4"});
const writableStream = await newHandle.createWritable();

fetch('./video.mp4')
.then(async (response) => {
    return await response.body.pipeTo(writableStream);
}).then(() => {
    writableStream.close();
});

OR something a bit easier or more relevant to the scope of this API:

const newHandle = await window.showSaveFilePicker({suggestedName: "video.mp4"});
const writableStream = await newHandle.createWritable();

fetch('./video.mp4')
.then(async (response) => {
    return writableStream.write(response.body);
}).then(() => {
    writableStream.close();
});

When reviewing I noticed multiple places that did something along the lines of:

throw an {{InvalidStateError}}

That should really be:

throw an "{{InvalidStateError}}" {{DOMException}}

Migrated from WICG/file-system-access#101 (see #2)

We should at least support the last-modified time and size, though we could consider other information such as the creation time.

For a file, it's currently not possible to get the size without reading in the file into memory via getFile().
For a directory, it's currently not possible to get the number of files in the directory without iterating through it (requested in WICG/file-system-access#215).

The WPT tests seem to assume you must pass 'at', and if you don't all reads start at 0. This is ... odd, especially for implementing the equivalent to read(), which maintains an internal position in the file, and reads from there. It means for almost all reads (or writes), you must pass at, which means you must track the expected position in the file - likely in a wrapper around syncaccesshandle.read()/write(). Emscriptem certainly can track the location in the file and insert at:nnnn always, but this seems like an odd choice for an API (and not great for performance, due to the extra parsing and validation and the basically forced extra kernel call to seek -- we could avoid the seek() call if we also track the position and elide if if it's a no-op, but that's not worth it either).

I imagine this is legacy decisions from the non-OPFS File System API

Though maybe not legacy -- SyncAccessHandles came later. Can (should?) we change this? If you're writing a bunch of data, it's a whole lot more (unnecessary) bookkeeping and overhead.

   position += handle.write(data1, {at: position});
   position += handle.write(data2, {at: position});

But you can't quite do that, since if there's an error and a partial write, you won't notice it. To be pedantic about catching errors, you need to do:

try {
  written = handle.write(data1, {at: position});
  if (written != data1.length()) { /* error */} else { postion += written;}
  written = handle.write(data2, {at: position});
  if (written != data2.length()) { /* error */} else { postion += written;}
} catch(e) { ... }

With classic posix-like read/write (with an assumed position update), it would be

  handle.write(data1);
  handle.write(data2);

or

try {
  if (handle.write(data1) != data1.length) { /* error */}
  if (handle.write(data2) != data2.length) { /* error */}
} catch(e) { ... }

There is no method to request the parent of FileSystemFileHandle.
Some use cases:

1. There is no ability to remove the file from the handle, so it's required to access the parent in order to call FileSystemDirectoryHandle.removeEntry(). This will allow the files opened with PWA (via file_handlers) to be removed from within PWA.
2. There is no ability to access other files in the same directory. For example consider the photo opened with PWA (image viewer) and the user wants a) to navigate to the next/previous photo in the directory or b) to browse/list all the other photos within same directory.

Migrated from WICG/file-system-access#121 and WICG/file-system-access#244 (see #2)

@foolip:

https://fs.spec.whatwg.org/#api-filesystemfilehandle-getfile says in domintro:

If the file on disk changes or is removed after this method is called, the returned File object will likely be no longer readable.

This isn't the normative text, but when the normative algorithm is written down will there really be any uncertainty to justify "likely" or can it be defined exactly what happens and when.

For a File that is no longer readable, does that mean that it's somehow neutered? Would lastModified reflect the change/removal time, or how would one tell that a File object is in this state?

@mkruisselbrink:

The note there is mostly trying to describe what is already (not well) defined in FileAPI. I.e. this is the behavior for all File objects that represent actual files on disk. Unfortunately this is also not very well specified in FileAPI (see for example w3c/FileAPI#47, but also w3c/FileAPI#75). I definitely intend to fix that, but will need to fix the FileAPI side of things before I can fix this side.

@mkruisselbrink:

We should define behavior of FileSystemFileHandle.getFile() when the underlying file no longer exists. Currently the spec is pretty much silent on this, we should probably make it clear/explicit that this should reject with a NotFoundError.

The web-platform tests check (in a few instances) that async operations like truncate fail if there's another async operation still unresolved.

a) there's nothing in the spec about this
b) I don't see that there's a need for this. So long as they're queued for operation and so they'll complete in order, multiple async operations should work. The main question would be "what happens to queued operations if an earlier operation fails/throws"'; this would have to be answered in the spec. I'd presume the answer would be "all queued operations would be rejected with XXXX if an earlier queued operation fails". (InvalidState?)

"resolve" doesn't hint too much what it is actually doing. One needs to read the algorithm to see what it actually returns.

These should probably be changed to use HTML comments? They currently render as part of the standard.

WPT tests for messaging use queryPermission(); this isn't defined here. It's in https://wicg.github.io/file-system-access/#api-filesystemhandle-querypermission

The OPFS tests shouldn't be using queryPermission, I believe

This issue was originally discussed here: https://github.com/whatwg/fs/pull/21/files#r879343956

The spec in general runs steps in parallel without enqueuing a task. It should be updated as mentioned in the comment chain above to use the right approach.

Update: there will be two File System sessions

- A breakout session will be held Wednesday, Sept 14th at 11:15-12:15 PDT (18:15-19:15 UTC). See details and subscribe here

- A WICG incubation meeting will be held Friday, Sept 16th at 13:30-14:30 PDT (20:30-21:30 UTC). See details and subscribe here

I've requested time for a breakout session about this API at TPAC. I'd like to use this issue to gather agenda items/topics people would like to talk about

Possible agenda items:

• Alignment on SyncAccessHandle implementation and standardization
  • Supporting multiple writers #34
  • Making all methods sync #7
  • Other features?
• Gauging interest in an async alternative to SyncAccessHandle

Possible agenda items which relate to the non-OPFS portion of the API (tracked in WICG/file-system-access):

• File System Change Events (very early design proposal)
• Supporting access to the file system without an open window
  • Enabling the "background sync" use case
  • Allowing long-running file operations to complete after tab closure

With ongoing work developing V8 stack-swapping & works like https://github.com/WebAssembly/js-promise-integration/blob/main/proposals/js-promise-integration/Overview.md , it's quite possible the current work we've done to build a very special purpose high-speed sync interface catering to webassembly might not have the performance upside they do today.

We should try to understand where the platform is headed & what to expect will happen in the future, and use this information to reassess whether #21 's (and #7 's) current choice of preferring FileSystemSyncAccessHandle makes sense, and consider switching back to FileSystemAsyncAccessHandle with all methods being async & looking more like a normal web interface.

The performance concerns that pushed us to FileSystemSyncAccessHandle in the first place may not hold for another year. And then we'd be left with an api that doesn't look normal for no real good reason, other than historical accident. Let's try to gain some confidence that won't be the case.

While preparing #1 I noticed that many promises are resolved and rejected from in parallel steps. Instead this has to be done from tasks. https://html.spec.whatwg.org/#queue-a-global-task seems most appropriate here, not entirely sure if this needs a new task source. Perhaps we should have one for storage in general?

Right now, it appears that this proposal does not support in-place overwrites. Furthermore, all access is buffered, and buffers are owned by ECMAScript code, rather than by the user agent.

Unfortunately, this is not the best way to get top performance. I will use Linux’s io_uring as it is the only high performance API I am remotely familiar with, but the general pattern should be similar for other such APIs as it is largely dictated by hardware. To get high performance, one needs:

• Direct I/O (bypassing the page cache).
• Asynchronous I/O (already present in the current proposal).
• Support for in-place and append writes.
• The ability to submit requests in parallel from multiple CPU cores.

For maximum performance, pre-registered buffers are required.

If this is not planned, it would be useful to say that such applications are out of scope.

Migrated from WICG/file-system-access#325 (see #2)

This issue was originally discussed here: https://github.com/whatwg/fs/pull/21/files/0cff51afca4e80753bf7b38bdf8c62c1ee37ffd9#r860572377

The question is, how should user agents react when there is an active access handle attached to a file, and that file is modified by an external process (e.g. the user runs rm -rf on the storage profile while the browser is running)? This case is currently a TODO in the Access Handles spec.

There are quite a few differences between the web-platform tests and the current spec -- which isn't really surprising, since the spec is still being modified. Some error conditions disagree, I think. There are also tests for move() support, which isn't in the spec.

There also are extensive tests for '.' and '..', and inherent assumptions that both are automatically defined when a directory is created, but that isn't really in the spec (perhaps it's vaguely alluded to by saying filenames are platform dependent, but that's very weak -- and as we discussed, it would be nice to see that go away). The spec needs to be more precise about what the filesystem tree navigation should look like. (Probably it helps use by wasm programs if '.' and '..' are automatically defined when a directory is created, I presume)

See step 1 of https://fs.spec.whatwg.org/#write-a-chunk

Let input be the result of converting chunk to a FileSystemWriteChunkType. If this throws an exception, then return a promise rejected with that exception.

it refers to https://infra.spec.whatwg.org/#javascript-string-convert which seems wrong, should it refer to https://webidl.spec.whatwg.org/#dfn-convert-ecmascript-to-idl-value instead?

Migrated from WICG/file-system-access#326 (see #2)

@mkruisselbrink

I've been thinking a bit about what options might make sense to be able to pass to the createAccessHandle and createSyncAccessHandle methods, primarily to make sure we won't paint ourselves into a corner if we initially ship only one "flavor" of access handles (i.e. exclusively locked, in-place writable access handles), as currently described in the AccessHandle proposal.

I'm currently thinking there might be three separate axes on which it could make sense to change the behavior of access handles:

File locking behavior (don't lock the file at all, hold a shared (aka read) lock on the file, or hold an exclusive lock on the file)
Should the handle operate on the file in-place, or atomically on a swap/temporary version of the file (either starting out blank or created by copying the previously existing file)
Should the handle provide read-only access or allow both reading and writing.
I'm not sure if all possible combinations of these make sense, but I think quite a few of them probably do. For example:

• Exclusive locking, in-place, read-write access matches the current AccessHandle proposal.
• Shared locking, swap file, read-write access sort of matches the current createWritable API, with the extra benefit that you'd be able to read back the uncommitted changes.
• Shared locking, in-place, read only access has been requested in WICG/file-system-access#325
• Exclusive locking, swap file, read-write access would help WICG/file-system-access#286
• Non locking, in-place, read only access would address WICG/file-system-access#157

We would only allow in-place, readwrite access for files in the origin private file system, for the same reasons we don't have an in-place mode for createWritable today (on the other hand, in-place read-only mode should be just fine for arbitrary file handles).

We probably want to disallow non-locking or shared-locking in-place read-write access, as that would enable changes from one frame to be immediately visible in other frames, resulting in potentially high-frequency timers etc. Requiring a page to first obtain a exclusive lock before being able to write in-place would give us the opportunity to make sure this can't be exploited as a super-low-latency communication mechanism.

One question I'm not sure about is if we'd also want to enable a particular AccessHandle to transition between different locking modes. Is there benefit to being able to upgrade a shared lock to an exclusive lock without having to re-open the access handle (or the reverse, drop from exclusive down to shared/no-lock)? My hope would be that we wouldn't need that, but I'm not an expert in how people use file locking.

Another question is what locking a file actually means for files outside the origin private file system. Enforcing the same locking semantics for other access via the File System Access API in the same browser instance would be a start, but probably we would also want to lock the actual files on disk with whatever file locking mechanism the underlying OS supports. Afaik on posix file locks are advisory only, but on windows shared and exclusive file locks do exist. I think we should aim for a best-effort do whatever is typical on the underlying FS, while enforcing well specified semantics for other access using the File System Access API.

Finally, some strawman IDL proposals:

enum FileSystemLockMode { "none", "shared", "exclusive", };
enum FileSystemAccessMode { "read", "readwrite", };
enum FileSystemWriteMode {"atomic", "atomic-from-copy", "in-place"};

dictionary FileSystemFileHandleCreateAccessHandleOptions {
  FileSystemAccessMode access = "readwrite";
  FileSystemWriteMode mode = "atomic";
  FileSystemLockMode lock;
};

I.e. the default would be similar to what createWritable does today, while if you want the current Access Handle proposal's behavior you'd have to pass {mode: "in-place"}. Not entirely sure what the default for lock should be. For in-place readwrite mode "exclusive" is the only valid option, so that should probably be it. On the other hand for read-only access it might make more sense to default to "shared" if no lock mode is provided.

In an initial implementation of the current Access Handle proposal in chrome we would implement this as

enum FileSystemWriteMode {"in-place"};

dictionary FileSystemFileHandleCreateAccessHandleOptions {
  required FileSystemWriteMode mode;
};

which more or less matches the current Access Handle proposal anyway (the only difference with the current proposal being that mode would be required), while this can relatively easily be backwards compatibly extended to cover the larger feature set described above.

@tomayac

Super nit remark: I know where we come from with this, but the below…

enum FileSystemAccessMode { "read", "readwrite", };

…is the odd one out that’s spelled without a dash ‘-‘.

While the WPT tests appear to test a few things with deleted File/Directory handles (like that values() on a deleted directory returns NotFoundError), there's no spec text that I see to specify what happens to an existing File/DirectoryHandle when it's removeEntry()'d from it's parent. Ditto SyncAccessHandles.
Also: what happens if an async method is still in progress?

This came up in web-platform-tests/wpt@b20e834#r85726061

Ideally API errors map uniquely to DOMExceptions, which make them actionable by the site. For example, if NoModificationAllowedError always indicates that a file is locked and NotAllowedError always indicates the site needs to requestPermission()on the handle, the site can respond differently to these errors.

Currently, developers have to figure out this mapping on their own, or read the whole spec and infer this mapping. It would be nice to include a table mapping API errors to DOMExceptions that developers can use as a reference. Is there precedent for having this type of table in a spec?

That being said, it could be tricky to add a mapping that truly guarantees an error corresponds to a given API error (especially outside of the OPFS, where there's a whole host of things that could happen on the underlying OS). At the very least, here's a list of the mappings I'm aware of as a starting point...

• NotAllowedError
  • No permission at the API level (i.e. need to requestPermission())
• NoModificationAllowedError
  • Cannot acquire a lock to the handle
  • Handle lacks read or write permission at the OS level (for example, trying to get write permission on a read-only file). This should only be relevant outside of the OPFS
• InvalidModificationError
  • Attempting to overwrite a file via move()
• QuotaExceededError
  • Operation will exceed quota
• TypeMismatchError
  • The handle type (file/dir) does not match what's on disk
• AbortError
  • Using a not-yet-launched feature (i.e. directory move())
  • Some internal errors
• InvalidStateError
  • Attempting to close an already-closed writable or SyncAccessHandle
  • Some internal errors

Some errors are mostly relevant outside of the OPFS:

• AbortError (in addition to above)
  • Picker selection cancelled
  • File save/move blocked by Safe Browsing
• InvalidStateError (in addition to above)
  • Trying to show a picker in a detached frame
• SecurityError
  • Trying to show picker in sandboxed iframe/opaque origin/third party context/without user gesture

it would be pretty sweet if it where possible to write a FileSystemFileHandle or FileSystemDirectoryHandle to the clipboard as a way of copying something from the sandboxed navigator.storage.getDirectory into the users own folder of choice

await navigator.clipboard.write([ 
  await navigator.storage.getDirectory()
  // ...more fileHandle(s)
])

Migrated from WICG/file-system-access#107 (see #2)

This could be used by code that wants to create a unique temporary file/directory. The code would generate names by adding a sequence number to a fixed prefix ("temp001"), and stop when it's able to create a new file/directory (as opposed to opening an already existing one).

As per Web IDL instead of

The isSameEntry(other) method, when invoked, must run these steps:

we would write

The isSameEntry(other) method steps are:

these days.

(Migrated from WICG/file-system-access#371.)

Currently, a FileSystemFileHandle gives you a FileSystem(Sync)AccessHandle by calling FileSystemFileHandle. create(Sync)AccessHandle(). This means you end up with code like below:

// Before:
const accessHandle = await fileHandle.createSyncAccessHandle();
accessHandle.write(…)

I wonder if maybe we do something else instead: similar to FileSystemFileHandle.getFile(), we could just have FileSystemFileHandle.get(Sync)HighPerformanceFile() or something, which then exposes all the things as listed in the IDL.

// After:
const hiPerfFile = await fileHandle.getSyncHighPerformanceFile();
hiPerfFile.write(…);

This would mean you get a special kind of high performance file object rather than a handle on a handle. It wouldn't be symmetric to File objects, but conceptionally it feels at least a little cleaner to me.

The webidl for read() and write() should indicate that they throw

Migrated from WICG/file-system-access#68 (see #2)

For example, the getFileHandle() and getDirectoryHandle() methods can reject, but how and with what exception?

TPAC highlighted a pretty substantial architectural difference between the Chromium and Firefox implementations of this API. Chromium's implementation maps a FileSystemHandle to a file path (which I had attempted to codify a while back), while Firefox's implementation suggests "they are references to objects resolved at creation time."

This has gone unnoticed until this point because no features have exposed this difference to the web.

But there are some significant web-observable implications to this choice, most notably around directory moves. Consider the following code, which moves a directory containing a child we have a handle to:

// Create file at /old/file.txt.
const dir_entry = root.getDirectoryHandle('old', { create: true });
const file_entry = dir_entry.getFileHandle('file.txt', { create: true });

// The file previously pointed to file at /old/file.txt now resides at /new/file.txt.
await dir_entry.move('new');

What happens next?

// PATH-BASED: The promise rejects with NotFoundError.
// REF-BASED:  The call succeeds and returns the File.
await file_entry.getFile();

// PATH-BASED: The promise rejects with NotFoundError.
// REF-BASED:  The promise resolves successfully and creates a new writable stream.
await file_entry.createWritable();

// PATH-BASED: The promise resolves to `null`.
// REF-BASED:  The promise resolves to 'file.txt' (same as before the move).
await dir_entry.resolve(file_entry);

// PATH-BASED: The isSameEntry() promise resolves to `false`.
// REF-BASED:  The isSameEntry() promise resolves to `true`.
const other_file = await dir_entry.getFileHandle('file.txt');
await other_file.isSameEntry(file_entry);

// PATH-BASED: The directory's ID has changed, along with its path.
// REF-BASED:  The directory's ID remains unchanged since before the move.
await dir_entry.getUniqueId();
// PATH-BASED: The file's ID remains unchanged, along with its path.
// REF-BASED:  The file's ID remains unchanged since before the move.
await file_entry.getUniqueId();

// This method is proposed in https://github.com/whatwg/fs/issues/38
// PATH-BASED: The promise presumably will resolve to `null`.
// REF-BASED:  The promise presumably will resolve to a copy of `dir_entry`.
await file_entry.getParent();

// PATH-BASED: getFile() is once again valid and returns the file at /old/file.txt.
//             The isSameEntry() promise resolves to `true`.
// REF-BASED:  getFile() returns the file at /new/file.txt.
//             The isSameEntry() promise resolves to `false`.
const old_dir = await root.getDirectoryHandle('old', { create: true });
const other_file = await old_dir.getFileHandle('file.txt', { create: true });
await file_entry.getFile();
await other_file.isSameEntry(file_entry);

Directory moves very blatantly expose this difference. It will be basically impossible to specify directory moves in a way that's consistent across platforms without being much more specific here. This also has implications for at least the getUniqueId()method and removed handles. Directory moves would also expose a difference in resolve() and a discussed-but-not-yet-formally-proposed getParent() method, among other things.

Looking at the code above, I tend to agree that a ref-based approach leads to outcomes which are more likely to align with user expectations, at least regarding directory moves. It would be nice if moving a directory didn't invalidate all child handles, for example. Meanwhile, there are some instances where a path-based approach arguably makes more sense. What happens to a FileSystemHandle when its underlying file is removed?

​​// Create a file.
const file_entry = root.getFileHandle('file.txt', { create: true });

// Remove the file.
await file_entry.remove();

// PATH-BASED: We can still write to a removed file.
// REF-BASED:  ????
await file_entry.createWritable();

// PATH-BASED: The isSameEntry() promise resolves to `true`.
// REF-BASED:  The isSameEntry() promise resolves to `false`, presumably?
const other_file = root.getFileHandle('file.txt', { create: true });
await other_file.isSameEntry(file_entry);

That being said, it seems reasonable to specify that a handle can still be written to if it's removed via this API, while handles removed by other means (such as by clearing site data or via an OS file manager) could be invalidated.

In summary...

One option is to never specify features that expose this implementation difference, such as directory moves. Unfortunately this is a pretty fundamental difference which I suspect will be hard to paper over as the API continues to evolve. To me, this is a very unsatisfying option. Consider a web IDE which just wants to rename src/foo/ to src/bar/, but is forced to recursively copy all contained files.

Another theoretical option is to accept that there will be cross-browser differences. Sure, moving a directory will invalidate all child handles, but you can re-acquire all the handles within the new directory. However, going down this path is bound to expose many more subtle cross-browser differences. For example, Chromium locks all ancestor directories when a file has an open writable to ensure its path does not change while it is being written to. Having an open writable will block moving a parent directory, which would be a confusing restriction in a reference-based design. This option would be bad for developers and the impacted users, but just listing it here for completeness.

The other option is to specify a requirement that a FileSystemHande is a reference (in the same way I had attempted to specify it’s a path here). That framework could be used to specify new methods such as move(), remove() and getUniqueId() in a way that’s consistent across browsers. This is our preferred option, but supporting this in Chromium would require a pretty substantial re-architecture that I'm hesitant to commit to without clear indication from other browsers and the developer community that handles should be based on references rather than paths...

@szewai does WebKit have an opinion here?

https://github.com/whatwg/fs/blob/main/w3c.json is no longer needed (and it's confusing some of our horizontal review tools).

Separately, you'll probably want to update some of the md files (LICENSE, etc.) as well.

w3c/whatwg-coord#12

The WPT tests assume a property 'locked' is exposed by a WritableFileStream. This does not appear anywhere in the spec, and I don't see any reason for it to be exposed. The solution here is probably to remove references to 'locked' from WPT

Otherwise lock acquisition will race with other lock-acquiring methods. See #9 (comment)

Can multiple async operations be pending for WritableFileStream? I presume so.
If so, what happens to following operations if one has an error? (Right now I presume nothing happens to them; they run as normal.)
We could, for example, abort all queued tasks for the stream on error.

I find the directory picker to be maybe to picky on what i can and can't choose b/c it can contain executable or system files?

Maybe instead of flat out restricting me to pick any folder i want how about instead allowing showDirectoryPicker to succeed on mostly anything and instead make the read, write, requestPermission access on that handle to throw instead? it would be like creating some opaque handle that you would not be able to do much with it.

In that case maybe you can at least go into some some of the directory or read some of the file or allowing anyone do save multiple files in a newly created directory, it feels quite silly that i can't select the nearly empty download folder to save multiple files which i can't select, But i can select my hole git folder where i have a buch of bad voodoo stuff that can execute a bunch of things, like what gives?

if i could pick the download folder (that i'm not able to select for some unknown reason) an i get a directory handle, then if i create a new folder within that directory and do: downloadDirHandle.getDirectoryHandle('status-report-${Date.now()}', { create: true }) then surly that newly created folder should at least be safe to read and write files to?

The text for truncate() on a WritableFileStream doesn't match the non-normative text, such as the discussion of the cursor position.

All references currently are in the non-normative text. PR #76 can provide a template

There was a request for symbolic links support (creating & reading links) in WICG/file-system-access#377, but it was closed as being risky to OPFS. I'd like to re-open this request again here.

First, fs work seems like it applies beyond OPFS, & fs ought to be able to encompass one of the most common filesystem capabilities. Second, the security concerns seem like they ought potentially be handleable. Perhaps the api for creating a symlink is attached to a directory, and we limit it to only creating symlinks to it's children.

the webidl has
[[EnforceRange](https://webidl.spec.whatwg.org/#EnforceRange)] required [unsigned long long](https://webidl.spec.whatwg.org/#idl-unsigned-long-long) at;

However the WPT test of negative seek targets for SyncAccessHandles looks for dom NotSupportedError; the webidl spec says it should throw a TypeError. Likely this is simply a bug in the WPT test

There's only a single line of normative definition for resolve() (which per other discussion here should be renamed). The non-normative language is fairly complete.