textileio / js-threaddb

This project has been moved to https://github.com/textileio/js-textile

Javascript implementation of Textile's ThreadDB

This project has been moved to https://github.com/textileio/js-textile.

js-threaddb's Issues

Should we persist either the auth or the public key

This could make it easier for consumers to access past access tokens, and potentially let's us do smarter caching of things like user identities locally. Caching identities locally is useful, because it reduces the likelihood that someone will create a bunch of random identities during testing etc.

Add support for write/read validators/filters

This is a new feature of go-threads, it is potentially awesome for thread-db users. In theory, this feature is pretty easy to support, though we should think carefully about how to expose it in the APIs we have here... perhaps we shouldn't?

Missing license information

We should use the same license we use elsewhere.

After state in changes middleware should be optimized

There are some suggestions in comments that should be explored. For example, we could just use undefined[] for deletes, and req.values for add/put if not we aren't waiting for auto-incrementing indexes to come back from IndexedDB (which, maybe we don't need to worry about because we're always using ulids?). If that is the case, then we can just always compute the :after: state using jsonpatch.applyPatch, or we could even just lazily compute the after state when we actually need them?

Add early-stage development steps to README

Just to help folks get started with development and testing

Cleaner access to threads-client

Right now, we abstract away access to the remote daemon via threads-client, which is great. Ideally, this library shouldn't require that as a dependency at all eventually. In the mean time, however, we need a cleaner way to actually create a threads-client and apply the right gRPC metadata. This is related to #2.

Fix up these messy types in Document + Collection

There is some pretty complex type-system things going on in these two classes to support direct manipulation of instances and then calling things like save, delete, etc. The main issue is that these are generic interfaces that contain generic interfaces, which is messy. This should be possible to cleanup, so that we get proper type inference without all those scary "as" directives.

Versions in IndexedDB are annoying for us.

Is there a way to abstract versions away by using the hash of the indexes or something? I tried this before, but we'd need strictly increasing version numbers, which doesn't really work with hashing. Some auto-incrementing version system would be useful.

Better error reporting to mask out dexie stuff

Since we're wrapping Dexie, we should avoid passing Dexie errors out to the callers, and prefer to catch common errors and throw those ourselves. The same goes for gRPC errors and common errors from our remote calls. These should each have their own specified Error in the right module here. For example, in remote, we have an Errors type that contains our expected remote errors (and one generic one). We should do a similar thing for local db errors, and throw those. This makes it easier for the caller to catch specific errors, rather than inspecting their messages etc.

Track "head" of remote updates

We need some way to know if our local state is in sync with the remote state. A nice way to do this is to simply track the latest "head" of the remote that we know about. Something like the hash of latest remote thread record. This isn't yet exposed in any nice way on the go side, so this ticket would require some updates over there as well. Otherwise, we have to query the remote thread to find out which instances we're missing/have to push.

Make this repo public

We can probably do this any time now.

Integrate with existing Hub connectors

Right now, this lib works with a remote go-threads daemon as long as you are able to explicitly provide the right credentials. For local development and running against a pure go-threads daemon (not the Hub), this is pretty simple, you just need an authentication token (which we have APIs and clients for). But to work against the Hub, this will be cumbersome, so we need to update the gRPC calls to utilize our existing infrastructure/connectors.

cc @andrewxhill for this one.

Update default serviceHost to Hub once ready

Right now we have it defaulting to localhost, which is fine for early testing and development, but not for end users.

Create docs and publish them

We should try to find a simple, canonical way to build the docs, and then deploy them here for now, until we are ready to move to the main docs pages for textile "things".

Full text search

Hi @carsonfarmer / Textile team,

I was just playing with this database in a browser locally and trying to figure out how I would do text search. Unless I'm missing it in the docs, I don't see any operators in the dexie-mongoify library that supports text search (i.e. the $text operator).

I ask partly because if this is a feature that is lacking, I might try my hand at contributing. I have a one-off document search mechanism in my project (using Dexie) that I could use as reference. I plan to ditch my existing solution for js-threaddb when it's ready, so I thought I might try to contribute if there is somewhere I can be helpful.

EDIT: here's the full-text search example from the Dexie repo that I used. It searches all words in a string, but only using "starts with". So you can't search for a string in the middle of a word, and it definitely doesn't have any "fuzzy search" capabilities. I'm going to keep looking around for a good full-text search solution. I'll probably pursue making a PR to add the $text operator to dexie-mongoify.

EDIT 2: I also found ydn-db-fulltext from searching around. Please let me know if you have any other leads on existing solutions for the fulltext search in IndexedDB problem :-).

Add "watch" API to remote object

This is for more real-time applications that are online. It would use the clients listen API. It should support auto-reconnecting.

need dexies useLiveQuery()

but it requires >[email protected]

Support deleteRange for track changes

Right now, we assume that a local db operation that deletes a whole range of instances is just a local operation, so should not be replicated to the remote. This might not be the case, as the caller might really want to clear our a bunch of instances. Should we allow this? If yes, we'll have to come up with a clean way to push a range deletion to the remote.

Enable Dependabot on this repo

Use the new "native" GitHub feature.

remove vscode settings from repo

These shouldn't have been checked in in the first place!

Cleanup pull semantics to include only those ids for the given collection(s)

Right now, it looks like we're always returning all updated ids, even if they weren't included in the updated collection. This normally isn't an issue, unless there are leftover changes from a previous pull that weren't applied.