treojs / idb-schema Goto Github PK

I know this is idb schema after all, but what do you think of allowing JSON export of data along with the schema? It certainly seems like a reusable feature...

(Beyond JSON alone, the "structured cloning algorithm" can allow for other types of data which could be reasonably cloned too, but if you're amenable to the feature request, I think JSON would be a nice start...)

Thanks!

Hi,

Sorry for the long post, but I thought this may interest you.

I'm planning on submitting a PR to add support in db.js for additional JSON-based schema-building formats in addition to the type supported now.

I was wondering whether you were interested in seeing the proposals below added into idb-schema. An advantage of this is that we could allow users to more easily mix and match formats, e.g., if they built the first two versions programmatically using idb-schema's current approach but then moved to a simpler JSON-based format or vice versa. Or multiple formats could perhaps even be combined within a single version. Each has its advantages and disadvantages.

The programmatic approach is unique in offering the full expressivity of addCallback, while the JSON formats can benefit from being declarative and potentially more succinct.

The first JSON type is already available in db.js (though I want to see it keyed to version so that an extended migration path is possible, e.g., if the user was on version 1 but needed to be brought through versions 2 and 3 as supported in idb-schema).

{
    store1: {
        key: {keyPath: 'id', autoIncrement: true},
        indexes: {
            firstName: {},
            answer: {unique: true}
        }
    }
}

To expand this for supporting versions would be a simple matter of allowing:

{
    1: {
        store1: {
        // ...
    },
    2: {
        store2: {}
    }
}

For each new version, we can compare the stores on the supplied object with the existing stores in the database, and if not in use, we can (optionally) delete them, whereas if there is a change, we can delete unused or changed indexes and create new ones, or rebuild any keyPath/autoIncrement changes. (This is mostly how it works now.)

The advantage of this approach over the other approaches is that it will show the whole schema in code exactly as it will be built upon each version. The disadvantage is the same (that it will become unwieldy with many versions) and also that it doesn't have a means of indicating that a store should be renamed or copied (though I guess it could be expanded to add movedFrom or copiedFrom properties). It might also be enhanced to act similarly to JSON Merge Patch (see below) to allow use of null to explicitly indicate that a given store, store feature, or index was to be deleted, and otherwise not delete. If we supported both of these changes, then I think the third type below would also automatically be supported as a subset of this (modified) first type.

Maybe we could even allow callbacks here too to get all the capabilities in one (though it would no longer by JSON):

{
    1: [function myCallback (upgradeneededEvent, librarySpecificObject) {}, {
        store1: {
        }
    ]

And/or we might allow add/put/delete/clear (with optional keys (including MongoDB-style) where relevant) (as well as an equivalent JSON for our query syntax) within the syntax to avoid the need for callbacks (similar to your idb-batch format).

The second type is one I wish to add based on JSON Patch though it is based on the first type.

3: [
     { "op": "remove", "path": "store2" },
     { "op": "add", "path": "store1/indexes", "value": { "lastName": { } } },
     { "op": "replace", "path": "store1/indexes/answer/unique", "value": false },
     { "op": "move", "from": "storeOldName", "path": "storeNewName" },
     { "op": "copy", "from": "people/indexes", "path": "otherPeople/indexes" }
]

The advantage here is that besides being declarative, a standard, and only needing to express changes, is that it can indicate the renaming of a store (i.e., move) so as to indicate that the original content should be preserved (though under a different name). It can also allow declarative copying (e.g., to replicate indexes on another store).

The third type is JSON Merge Patch (Note that this spec doesn't work well if the null value is needed in the altered JSON, but for our schema purposes, there are no current arguments which act in a special manner with null).

This approach would also be declarative, a standard, and only need to express changes (including being explicit about whether it was deleting or altering), but it would not allow renaming (or copying) of stores. Its advantage over JSON Patch would be that it is more succinct. Here is the equivalent (minus the move/copy)

3: {
    "store2": null,
    "store1": {
        "indexes": {
            "lastName": { }
        },
        "answer": {
            "unique": false
        }
}

What do you think?

I've been working on adding idb-schema to db.js and have it working and passing our tests, but with a slightly ugly user-facing workaround. With an added API to idb-schema, the need for this workaround can be avoided and I think may be useful to other libraries.

The issue is that when callback is used to build the upgradeneeded callback, the callbacks added earlier by addCallback will be called with the upgradeneeded event, but I'd like to pass our own object as the first argument to allow the user to make db.js-style queries rather than forcing the user to be required to utilize the upgradeneeded event.

Would you be open to a PR which supports a callback argument supplied to the Schema constructor (perhaps on an options object instead or as a method called directly after Schema()) which, if present, will itself be passed the upgradeneeded event and the user's callback so that wrapper can determine what to supply to the user's callback?

User code:

const schema = new Schema(function (e, cb) {
    cb(dbJSWrapper(e), e);
}).version(1)
  .addStore('books')
  .addCallback((dbJSObject, e) => {
    // do something custom
  })

In the db.js code:

request.onupgradeneeded = e => {
    const idbschema = new IdbSchema();
    userOptions.schemaBuilder(idbschema); // Allow the db.js user to build their schema with `addCallback()`, etc.

    idbschema.callback()(dbJSObject, e); // Pass our own custom object for making db.js queries
};

Because of the problem with transactions within upgradeneeded expiring, if exposing one's library's promise-based API (mentioned in #7), these promises can break within that context, so a workaround may be to send multiple addCallback calls, and it would be nice to have a little syntactic sugar to do this more elegantly.

I'm sorry for a kind of support question, but is there some easy way to browserify the es6 modules syntax without setting up a grunt file, etc.

https://github.com/defunctzombie/zuul
Firefox, Chrome, IE10-11, Safari 6-latest