morgul / dustbin Goto Github PK

This module should (optionally) be able to be used with RequireJS.

It is incredibly important that RequireJS be optional. It should always be usable as a normal javascript include.

It can be useful to know what key an item was stored in to make later retrieval easier, as well as allow for reference by other objects. Perhaps stored as a '_key' property, or using an interal '_id' property.

It may be useful to store objects wrapped to allow meta data to avoid needing to dictate any form of structure upon the stored data.

Implement map/reduce functionally, inspired by Riak.

This gist has some ideas.

I'm proposing we add support for links between objects in Dustbin. This could be very useful to managing data relationships.

Link Format:

A link would be an object of this structure:

{
    tag: [{bucket: "some_bucket", key: "some_key"}];
}

We would add links to the $metadata (proposed in #4 ), which would simply be the above object with multiple tags.

Working with Links

The first thing we would need is an api for adding/removing links:

// Add a link to obj1 from obj1 to obj2 with the tag 'tag'.
dustbin.link(obj1, 'tag', obj2);

// Add a link to obj1 from obj1 to the object at bucket/key with the tag 'tag'.
dustbin.link(obj1, 'tag', 'bucket', 'key');

// Adds links to obj1 from ob1 to obj2, obj3 and obj4 with the tag 'tag'.
dustbin.addLinks(obj1, 'tag', obj2, obj3, obj4);

// Adds links to obj1 from ob1 to the objects at 'bucket/ket1' and 'bucket/key2' with the tag 'tag'.
dustbin.addLinks(obj1, 'tag', ["bucket", "key1"], ["bucket", "key2"]);

// Remove a link from obj1 with the tag 'tag', that points to obj2.
dustbin.unlink(obj1, 'tag', obj2);

// Remove a link from obj1 with the tag 'tag', that points to the object 'bucket/key'.
dustbin.unlink(obj1, 'tag', 'bucket', 'key');

// Sets the links on object obj1 to the `link` object passed in.
dustbin.setLinks(obj1, {tag: [{bucket: "bucket", key: "someKey"}]});

These should cover most of the use cases. We still need the ability to work with the links:

// Get all links associated with the given tag. Returns a list of resolved objects.
dustbin.getLinks(obj1, 'tag');

// Link walk object 1
dustbin.walk(obj1, ["bucket", "tag"], [undefined, "someTag", true]);

// Link walk object 1, verbose syntax
dustbin.walk(obj1, {bucket: "bucket", tag: "tag"}, {tag: "someTag", keep: true});

This is pretty much the exact same link walking as Riak, with the exception that instead of using "_", I'm using undefined to mean "any". I believe that simply not specifying one of the options is more straightforward than an arbitrary (if commonly used) string convention.

In the end, the API is less important than answering the question, "Will this be useful?". While I can see use cases, I'm not certain that they're worth adding this support in.

If we're not sure if we want this, but thing it might be useful in some places, I'd like to write it as an extension to Dustbin; i.e. if you include dustbin.js and dustbin-links.js the DustBin object would have it's prototype extended to support links. This way, only the people who want them will need to include the additional code, and any overhead it might induce.

Implement a query api, inspired by django's query api.

Basically, how I would like this to work is like this:

dustbin.query({someKey: "valueToMatch", someOtherKey: "someOtherValueToMatch"});

We can support the field api from django like this:

dustbin.query({someKey__lt: 5, someOtherKey__contains: "partial string"});

Alternatively, we could do this using a jugglingdb like api:

dustbin.query({lessThan: {someKey: 5}, contains: {someOtherKey: "partial string"}});

I'm not sure which I like better.

dustbin.query() should return an array of buckets. Either just the names, or names with metadata (like number of objects stored).

When storing an object using 2 arguments (bucket and object), a new key is always created for the object even if the meta data has one, and bucket matches.