fluidinfo / fluidinfo.js Goto Github PK

Example usage:

fi = fluidinfo({username: "test", password: "test"});
var handler = function(data) {
  // handle stuff
};
var vals = {
  "ntoll/rating": 7,
  "ntoll/comment": "I like sandworms"
};
fi.tag({values: vals, about: "some about value", success: handler});

Again, needs thought / discussion.

Because of a bug in Mozilla (viz. https://bugzilla.mozilla.org/show_bug.cgi?id=608735) the getAllResponseHeaders() function doesn't work on CORS requests (it just returns null). As a result I hacked together something to work around this so we always had the headers from Fluidinfo available for the purposes of debugging. Unfortunately, it pollutes the JS console with errors and warnings. Since this only affects Mozilla (all other modern browsers work correctly) we should use the specified getAllResponseHeaders function and return an empty dictionary in the case of Mozilla.

It's now possible to omit the tag parameter when sending a GET /values request.

It's now possible to omit the tag parameter when sending a DELETE /values request.

Given that we'll be creating JS objects to represent Fluidinfo objects I think these "instances" should come with some sensible utility functions as standard:

getTag("a/path");
setTag("a/path");
deleteTag("a/path");
updateTags({specification object});
reset(); // clears the caches

Needs thought / discussion.

This is not really a bug, but it's a behavior that could be better defined and a suggestion.

If I call the update function (or tag, which calls update) with a tag value that is undefined, the payload that is sent to Fluidinfo looks like this:

{"queries":[["fluiddb/about=\"fred45\"",{"sally/rating":{}}]]}

In this case, sally/rating had an undefined value in the options.values passed to update. So Fluidinfo is being asked to update nothing at all.

I think the code should either throw an error or convert the undefined value to be a null. To me the latter makes more sense. In which case we need a test for undefined in the update function which sets a null value.

Somewhat related: the update function could keep track of how many values it is supposed to update, and if it's zero just call the onSuccess function immediately without making a network call. This would allow client code that doesn't know how many values it is actually updating to be more efficient when the number is actually zero. I can open a separate ticket on this if people think it's worth fixing.

The library needs to make the serialisation of payloads from JS objects -> JSON transparent to the user.

We currently have code that looks like this:

      var HEADERS = ["Content-Type", "Content-Length", "Location", "Date",
        "WWW-Authenticate", "Cache-Control", "X-FluidDB-Error-Class",
        "X-FluidDB-Path", "X-FluidDB-Message", "X-FluidDB-ObjectId",
        "X-FluidDB-Query", "X-FluidDB-Name", "X-FluidDB-Category",
        "X-FluidDB-Action", "X-FluidDB-Rangetype", "X-FluidDB-Fieldname",
        "X-FluidDB-Type", "X-FluidDB-Argument", "X-FluidDB-Access-Token",
        "X-FluidDB-New-User", "X-FluidDB-Username"];
      var h = "";
      for(h in HEADERS){
        var header = HEADERS[h];

I'm not sure that that's the best way to loop through the array (my JS skills are almost zero), but in any case it's producing a weird message in the JS console, which ends like this:

Refused to get unsafe header "X-FluidDB-Access-Token"
Refused to get unsafe header "X-FluidDB-New-User"
Refused to get unsafe header "X-FluidDB-Username"
Refused to get unsafe header "function (val) {
    for (var i = this.length; i--;) {
        if (this[i] == val) {
            return true;
        }
    }
    return false;
}"

so I think the for h in HEADERS is picking up some property of the array (which looks like a find function) and trying to get the value of the function (as a string!) as a header.

The error message goes away when I instead use the more normal(?)

      for (var i = 0; i < HEADERS.length; i++) {
        var header = HEADERS[i];

So I'm going to make that change in this ticket.

JSONP is a "hack" that allows cross site calls without using CORS (although I believe its capabilities are limited). It has been suggested that it'd be useful for fluidinfo.js to support JSONP calls (see https://twitter.com/#!/renedudfield/status/102723431413452800).

I think this is a good idea since JSONP works on a wider range of browsers than CORS does. Also, Fluidinfo appropriately supports JSONP calls to it (see http://doc.fluidinfo.com/fluidDB/api/http.html#support-for-jsonp).

I suggest a flag is passed at time of instantiation to indicate the library should use JSONP for cross-site calls rather than the default CORS method:

  fi = fluidinfo({username: "username", password: "password", jsonp: true});

If I remember correctly, any fix will also need access to the DOM in order to make the JSONP hack work.

Finally, it should be opaque to the user how we're making the call to Fluidinfo (either CORS or JSONP). Their main concern is that the damn thing works. :-)

Fluidinfo will have a new endpoint: /recent This needs to be reflected in the public methods of fluidinfo.js.

Pasted below is the (edited) specification described on the Fluidinfo developers mailing list:

1. We will add the following endpoints to the fluiddb service:

  /recent/about/{about value}
  /recent/objects/{object id}
  /recent/users/{username}

2. Each of this endpoints will return a JSON list with objects
   representing recent activity. The list will look like this:

  [
   {
     "username": "terrycojones",
     "tag": "terrycojones/like",
     "object_id":  "3b57f6b7-c239-481a-9595-beeffa2958c3",
     "about": "Recent Activity",
     "value": "foo",
     "timestamp": "2012-01-26T16:00:09Z"
    },
    ...
  ]

3. Each endpoint will return a limited amount of recent activity
   objects. 10 objects would be good for now. We make that number an
   argument in the future.

Example usage:

  fi = fluidinfo({username: "test", password: "test"});
  var values = ["ntoll/rating", "ntoll/description"]
  var where = 'has fluiddb/about = "foo"';
  fi.delete({values: values, where: where, onSuccess: function(result){},
    onError: function(result){}});

Allow the constructor to be passed an OAuth access token so that we can make OAuth2 calls to Fluidinfo.

Currently, the user simply passes in the URL to the get, post, put, delete and head functions under .api. Unfortunately this opens up the possibility that the user will pass in something that's wrong. Far better is to follow fluidinfo.py's approach and allow them to supply a list of arguments to append to the URL and let fluidinfo.js correctly handle things (escaping and so on).

I propose we change the way the options object is used to specify the call to Fluidinfo:

• Replace the url value with path
• Add an args dictionary which will is used to build the path arguments:

  var path = "values";
  var args = { tag = ["foo/bar", "baz/qux"],
    query = "has ntoll/rating > 7"}
  fi.api.get({path: path, args: args});

Will result in the following URL:

https://fluiddb.fluidinfo.com/values?tag=foo/bar&tag=baz/qux&query=has%20ntoll%2Frating%20>%207

Notice how the args have been correctly escaped AND the tag list has been turned into the several instances of the tag argument.

We have a bunch of code like this:

    /**
     * Easily tag a specified object
     */
    session.tag = function(options) {
        if (options.about === undefined && options.id === undefined) {
            throw {
                name: "ValueError",
                message: "Supply either an 'about' or 'id' specification."
            };
        }
        if (options.about) {
            options.where = 'fluiddb/about="' + options.about + '"';
        } else if (options.id) {
            options.where = 'fluiddb/id="' + options.id + '"';
        }
        this.update(options);
    };

Notice that we're mutating the options instance provided by the
user. It'd be better if we never did this. If the user tries to
reuse the same options instance twice they may end up with
surprising behaviour because we've modified it. We should make a copy
when we need to manipulate the provided options.

Currently, the test suite comprehensively ensures the library is sending the right sort of thing to Fluidinfo but doesn't make sure the right event handlers fire and the resulting response is processed properly.

OAuth2 calls only work over https. We could be checking for that in the code.

This is to make it easy for a developer to reference the opaque value in code. For example, if the opaque value is an image then the url value should be used as the <image> tag's src attribute.

This is definitely a feature I keep having to re-invent whenever I want to embed images, docs, audio etc that's stored in Fluidinfo.

For completeness, here's what I expect the result to look like:

[
    { id: "SOMEUUID",
      about: "foo",
      original: { ... the original object returned from Fluidinfo in all its verbose glory ...},
      "fluiddb/about": "foo",
      "ntoll/rating": 7,
      "ntoll/picture": { 
        "value-type" : "image/png",
        "size" : 179393,
        "url": "https://fluiddb.fluidinfo.com/objects/SOMEUUID/ntoll/picture"
      }
    },
    ... etc...
  ]

Example usage:

fi = fluidinfo({username: "test", password: "test"});
var handler = function(data) {
  // do stuff
};
var tags = ["fluiddb/about", "ntoll/rating", "terrycojones/rating", "esteve/comment"];

fi.getObject({about: "some about value", select: tags, success: handler});

Obviously, you should be able to specify either the fluiddb/about value or the UUID in the options dict passed into the function.

Needs thought and discussion.

The current tests (using the Jasmine BDD test framework) are not comprehensive, complete or documented in any way (as appropriate).

Steps to reproduce:

• Try to use the code from a Firefox extension. ;b

More seriously, it appears that splitting the headers on \r\n
doesn't work in this context. Splitting on \n does work.

It looks like getting headers from an XHR request isn't as simple as first thought. You can't just reference xhr.responseHeaders. Instead you could use xhr.getAllResponseHeaders(); but this doesn't appear to work on Firefox. :-(

The solution is to have a constant array of expected headers and attempt to extract them from the response with xhr.getResponseHeader. This works on both Chrome and Firefox and is clearly defined in the W3C specification of an XmlHttpRequest object (here: http://www.w3.org/TR/XMLHttpRequest/#the-getresponseheader-method). For example:

  xhr.getResponseHeader("content-type");
  "application/json"

Example usage:

  fi = fluidinfo({username: "test", password: "test"}); // must be logged in for this to work
  fi.createObject({about: "foo", onSuccess=function(result){}, onError=function(result){});

The query (at least at the moment)

$ curl 'http://fluiddb.fluidinfo.com/recent/about/9801

comes back with

[{"username": "terrycojones", "about": "9801", "timestamp": "2012-02-11T19:58:10.142469", "value": null, "tag": "terrycojones/neat-number", "object_id": "c2784b35-d6b6-4168-913d-8ef62b0a041d"}, {"username": "terrycojones", "about": "9801", "timestamp": "2012-02-11T19:56:08.318588", "value": "http://math.stackexchange.com/questions/102682/what-is-special-about-the-numbers-9801-998001-99980001", "tag": "terrycojones/reciprocal", "object_id": "c2784b35-d6b6-4168-913d-8ef62b0a041d"}, {"username": "fluiddb", "about": "9801", "timestamp": "2012-02-11T19:55:45.326760", "value": "9801", "tag": "fluiddb/about", "object_id": "c2784b35-d6b6-4168-913d-8ef62b0a041d"}]

In which the value of terrycojones/neat-number is null.

This causes an error to be thrown by line 749 of fluidinfo.js:

if (obj.value["value-type"] !== undefined) {

because obj.value is null.

While the raw XHR object should always be available to the event handlers (we don't want to hide anything from the end users of the library), we should also make it easy to find what's of most interest.

I suspect this will require a little more thought, but I propose passing a Fluidinfo response object that looks something like this:

{
  status: 200,
  headers: {
    "Content-Type": "application/json",
    "etc...": "etc..."
  },
  data: {foo: "bar"}, // appropriately cast into Javascript values/structures as required
  request: xhr // the original XMLHttpRequest object for the purpose of completeness
}

I think this would be a big win in ease-of-use as well as being relatively trivial to implement and test.

Comments, suggestions and critique most welcome.

It could be prettier, and we like pretty, mmm, pretty.

Now that Fluidinfo has a /renew-oauth-token endpoint, we need to use
it, in conjunctions with renewal tokens returned during OAuth Echo
calls, to generate new access tokens, when needed.

Just because it's a nice thing to do...

The code throws if you pass options {instance: 'http://localhost:9000/'} to the constructor. That's because colon is not in the allowed chars in urlRegex. It would probably be better to be more relaxed about what's allowed, e.g. var urlRegex = /^(http|https):\/\/.+\/$/;

Since sinon mocks away the XHR request in the jasmine based test suite we've noticed a class of bugs related to specific implementation details of the XHR object in different browsers. We need to be able to test that the library works on at least Firefox and Webkit. Selenium seems to be the obvious choice to facilitate this.

If I do the following query:

$ curl -v 'http://fluiddb.fluidinfo.com/values?tag=fluiddb%2Fabout&query=has%20terrycojones/9801'

I get a 404 because there is no such tag (terrycojones/9801). This causes an error in fluidinfo.js
Uncaught TypeError: Cannot read property 'id' of undefined at line 528:

        var processResult = function(raw) {
            var result = [];
            var data = raw.data.results;
            var objectID;
            for (objectID in data.id){

because raw in an HTTP error object:

Object
data: ""
headers: Object
rawData: ""
request: XMLHttpRequest
status: 404
statusText: "Not Found"
__proto__: Object

The options.onError function should have been called, not the onSuccess.

The only two changes are: object_id field name has been replaced with just id. And timestamp has been replaced by updated-at

[ { "about" : "odpo",
    "id" : "fa5cf18e-2d89-44bd-9226-21af8bd187da",
    "tag" : "ceronman/test_tag2",
    "updated-at" : "2012-02-13T17:22:58.389364",
    "username" : "ceronman",
    "value" : [ "One", "Two", "Three" ]
  },
  ...
]

Example usage:

fi = fluidinfo({username: "test", password: "test"});
var handler = function(data) {
  // do stuff
};
var vals = {
  "ntoll/rating": 7,
  "ntoll/description": "I like it!"
};
fi.update({values: vals, where: "has terrycojones/rating < 2", success: handler});

Obviously needs thought and discussion.

I've had a look into this and I think, in the short term, it'd be relatively easy to implement a comprehensive and well tested XHR based function to handle all the calls down the wire rather than rely on jQuery to do it.

If we assume that we're not targeting server-side JS (i.e. node.js) in this first release then I think it's good to make the library framework-agnostic.

Some useful articles I've read whilst researching this:

http://blog.sociomantic.com/2010/08/native-javascript-ninjutsu-ajax-with-xhr/

and

http://www.quirksmode.org/js/xmlhttp.html

I'm happy to take on this work. I suspect it's only 6-8 hours worth of work (tops) given the comprehensive examples cited above.

Just so we have nice docs :-)

From the internet...

"0, not being a valid HTTP status code, is used by browsers to indicate
success for a non-http xhr response (for example, using the file://
protocol)"

This seems to confirm it: https://developer.mozilla.org/En/Using_XMLHttpRequest#section4

So I'm not sure what was happening with issue #54 but I think the correct thing is to assume that <300 is a successful response. In the case of issue #54 then we should be trapping the onError callback of the XHR object and reacting appropriately.

Sometimes it's useful to be able to get the username of the currently logged-in user for the purposes of building paths to tags (such as username/rating). Rather than have to faff about with remembering to store this information somewhere in your own application it should be possible to get at it with as an attribute:

  var fi = fluidinfo({username: "ntoll", password: "secret"});
  var u = fi.username // username will now equal "ntoll"

I'm not sure how/why this is happening yet, but when line 423 is reached in fluidinfo.js it is possible that options.onError either doesn't exist or exists but is not a function. See screenshot at http://jon.es:/var/www/jon.es/other/fluidinfojs-error.png

This is occurring during some local testing, and I probably don't have things set up 100% right, but the library should be robust in the face of that. I'll dig for more details now and add them to the ticket.

Bar has reported Firebug complaining of the following (I can't reproduce on my version):

assignment to undeclared variable fluidinfo
[Break On This Error] fluidinfo = function(options) {
fluidinfo.js (line 20)

reference to undefined property this.server.requests[0].requestHeaders.Authorization
[Break On This Error] expect(this.server.requests[0].requestHeaders['Authorization'])
fluidinfoSpec.js (line 135)

assignment to undeclared variable expected
[Break On This Error] expected = "https://fluiddb.fl...m/objects/fakeObjectID/username/tag";
fluidinfoSpec.js (line 151)

assignment to undeclared variable actual
[Break On This Error] actual = this.server.requests[0].url;
fluidinfoSpec.js (line 152)

assignment to undeclared variable j
[Break On This Error] for(j=0; j<args[arg].length; j++) {
fluidinfo.js (line 230)

assignment to undeclared variable i
[Break On This Error] for(i=0; i<primitiveTypes.length; i++) {
fluidinfo.js (line 154)

reference to undefined property this.server.requests[0].data
[Break On This Error] expect(this.server.requests[0].data)
fluidinfoSpec.js (line 27)

reference to undefined property this.server.requests[0].data
[Break On This Error] expect(this.server.requests[0].data)
fluidinfoSpec.js (line 27)

reference to undefined property this.server.requests[0].data
[Break On This Error] expect(this.server.requests[0].data)
fluidinfoSpec.js (line 27)

assignment to undeclared variable memberType
[Break On This Error] memberType = typeof(value[i]);
fluidinfo.js (line 167)

assignment to undeclared variable result
[Break On This Error] result = [];
fluidinfo.js (line 417)

Because having good documentation is important!

I'm not sure exactly what's triggering this, but I send bad credentials to Fluidinfo from the options page (chrome-extension://oaigjjdgkdnmnckmkodplpihbjokhbln) of a Chrome extension and the library calls the onSuccess handler even though the request fails. I can see it failing by attaching an onerror handler to the createXMLHTTPObject result. When the failure happens, the response has status 0. So I think that at least we should be testing that the response status is > 0.

I want to be able to pass an instance like http://127.0.0.1:8888/ or http://ec2.blah.amazon.com:8888/ as an instance. Thanks!

Because Doug Crockford (no less) says so!

See page 52 of "Javascript: The Good Parts":

"We start by making a function that will produce objects. We will give it a name that starts with a lowercase letter because it will not require the use of the new prefix."

Example usage:

fi = fluidinfo({username: "test", password: "test"});
results = fi.query({select: ["fluiddb/about", "ntoll/rating"], where: "has terrcojones/rating>7"});

Rather than just allow the results to be the JSON that comes down the wire I suggest it's more useful that the result is a simplified array of objects that'd look something like this:

[
  { id: "SOMEUUID",
    "fluiddb/about": "foo",
    "ntoll/rating": 7,
  }
]

Each object should probably have a function to get at the original JSON representation too. If the returned value is for an opaque type it should just be copied to the new object "as is".

This definitely needs thought so please discuss in comments! :-)

Specifically, the use of two and four space indents are mixed. Also,
spaces before parentheses in some places are missing.

Specifically, the use of two and four space indents are mixed. Also,
spaces before parentheses in some places are missing.

I'm using the fluidinfo.js code in the Firefox extension I'm working
on... it's basically the first time I'm using the library and I'm
finding it quite awkward to figure out how to do things. I'm ending
up having to dig through object browser code to find examples and read
the fluidinfo.js source to determine what kind of methods are
available and how they work.

I'll update the README.md as I discover things to help make this
process a bit easier for the next person that has to use the library.

It's a Javascript keyboard and IE is picky on that :(

... so the library has it for ever but it isn't polluting the global namespace.

my bad…

Some browsers return "Content-Type" others. "content-type" and so on. This interferes with the decoding of results that rely on content-type = "application/json".