cscheid / rserve-js

Javascript implementation of the Rserve protocol

A javascript implementation of RServe over WebSockets

RServe is a protocol for communication with a remote R session. This package allows you to connect to a running RServe server that is serving over websockets. This way node.js (and javascript programs in general) can communicate with an instance of R.

RServe-js allows a web browser to communicate directly with a running R process on the other side of the wire. This means it's the equivalent of a chainsaw: there are ways to use it safely,

Quick tour

$ git clone https://github.com/cscheid/rserve-js.git

Start Rserve in web-sockets mode:

$ cd rserve-js/tests
$ r_files/start_no_ocap

Run some javascript that connects to port 8081:

$ node

> r = require('../main.js')
{ Robj:
  ...
  write_into_view: [Function] }

> r = r.create()
{ ocap_mode: false,
  ...
  resolve_hash: [Function] }

> r.running
true

> r.eval('rnorm(10)', function(err, a) { if (err === null) console.log(a); })
undefined

{ type: 'sexp',
  value:
   { type: 'double_array',
     value:
      { '0': -1.5626166190555,
        '1': -0.16678360090204197,
        '2': 1.362470594733813,
        '3': 0.2462241937647293,
        '4': -0.6439588002729958,
        '5': 1.6695940797441013,
        '6': -0.8298271898727629,
        '7': -0.14431491982950537,
        '8': -0.05561817220786299,
        '9': -1.5889826020213365,
        BYTES_PER_ELEMENT: 8,
        get: [Function: get],
        set: [Function: set],
        slice: [Function: slice],
        subarray: [Function: subarray],
        buffer: [Object],
        length: 10,
        byteOffset: 24,
        byteLength: 80 },
     attributes: undefined } }

Security considerations

NB: Rserve, in the mode described above, should only be run in trusted networks! eval, in that example above, is truly eval:

> // RUNNING WITH SCISSORS
> r.eval('readLines(pipe("ls /etc"))', function(err, x) { if (err === null) console.log(x); })

  { type: 'sexp',
  value:
   { type: 'string_array',
     value:
      [ ...
        'apache2',
		... ],
     attributes: undefined } }		

Thankfully, Rserve provides a mode which only exposes a fixed set of entry points. These are known (for historical reasons) as object capabilities.

Object capabilities

There's a demo of object-capability support in tests/ocap_tests.js and tests/oc.init.R. Roughly speaking, in object-capability mode, the server involves an initialization function that returns an object. This object is sent and is accessible by the Javascript side.

Critically, the serialization process converts any R functions and closures to opaque objects, which in Javascript are converted to things that behave exactly like asynchronous function calls (eg. XmlHttpRequest). These callable objects are known as capabilities, since each of them is a very specific feature accessible by the remote system. Since capabilities are functions, they return new values. And since both Javascript and R have functions as first-class values, capabilities are also first-class in this system. Capabilities can return other capabilities, and so a system can be designed to provide, by default, a very limited set of features, which can be increased when appropriate.

rserve-js's Issues

Rserve refers to global variable Rserve

So this works:

> Rserve = require('rserve')
> ...

But this doesn't

> r = require('rserve')
> ...

Oops.

Maximal node version for ocap support

I'm interested in setting up the ocap support, and the example script ocap_tests.js happens to run fine with 0.10.x versions (e.g. OK with 0.10.18 in my case), but apparently fails with 0.11.x (0.11.13 in my case). The error occurs line 13 of ocap_tests.js - funs is undefined at this point, for a reason lying deep in the ocap function I expect...

It's OK for me to use 0.10.x for now, but I guess at some point dependencies will push me towards 0.11.x.

Error: Parse Error just connecting

When executing this node.js code:

Rserve = require('rserve');
var s = Rserve.create({
    host: 'http://127.0.0.1:6311',
    on_connect: test,
    on_close: function(msg) {
        console.log("Socket closed. (!?)");
        console.log(msg);
    },
    debug:{
        message_out: function(data) {
            console.log("OUT!", data);
        },
        message_in: function(data) {
            console.log("IN!", data.data);
        }
    }
});

function test(){
    console.log('connected');
    s.eval('123123', function(){
       console.log("callback!!");
    });
}

I get this error on the node.js console:

events.js:71
        throw arguments[1]; // Unhandled 'error' event
                       ^
Error: Parse Error
    at Socket.socketOnData (http.js:1430:20)
    at TCP.onread (net.js:404:27)

The output of Rserve is:

connection accepted.
sending ID string.

header read result: 16
DUMP [16]: 47 45 54 20 2f 20 48 54 54 50 2f 31 2e 31 0d 0a
discarding buffer because too big (awaiting 1414012975 bytes)
Connection closed by peer.
done.

Node exits upon simple R errors

Evaluating instances of several classes of bad R commands with rserve-js causes node to exit without further information:

> r=require('rserve')
{ Robj: 
   { null: [Function],
     clos: [Function],
...
  determine_size: [Function],
  write_into_view: [Function] }

> rc=r.create()
{ ocap_mode: false,
  running: false,
...
  wrap_ocap: [Function],
  resolve_hash: [Function] }
> rc.eval('a;',function(d){console.log(d);})
undefined
> 
/my_project_path/node_modules/rserve/main.js:764
    var handle_error = opts.on_error || function(error) { throw new Rserve.Rse
                                                                ^
Error: ERROR FROM R SERVER: 127 65538 127 16 [object ArrayBuffer]

Some other commands which produce a similar node crash, and their corresponding output in R:

Ref to undefined variable: "a" -->  Error: object 'a' not found
Mismatched parens: "plot(;"    -->  Error: unexpected ';' in "plot(;"
Misplaced $: "error$$$$error"  -->  Error: unexpected '$' in "error$$"

I've experimented with wrapping these commands in "tryCatch" and "eval(parse(" to attempt to catch the error when it occurs in R, but node still exits upon encountering these errors. And from the rserve-js source it appears this behavior comes from in rserve-js rather than being somehow internal to node, or an artifact of the Rserver connection.

Is there any way to prevent rserve-js from crashing node?

Also FYI some platform/version info: on Ubuntu 12.04 with node v0.10.22 and rserve-js 1.0.2, tried two different configs (for unrelated reasons): R 3.0.2 / Rserve 1.8-0, and R 2.14.1-1 / Rserve 1.7-3

Thanks.

Error while assigning java script variable to r variable

I am confused whether i can assign javascript variable to R variable.

var disease="fever";
client.assign("'x', disease"); or
client.assign("x", disease);
client.eval("print(x)")
Both doesnot work

this the error i get
Error: ignoring SIGPIPE signal
*** stack smashing detected ***: /usr/local/lib/R/site-library/Rserve/libs//Rserve terminated
Am i doing it wrong or whether this kind of facility is not provided in the Rserve-js

Move to promises

Another breaking change, but I'm seriously inclined to move all javascript ocaps in rserve-js to be Promises A+ promises.

Catch WebSocket connection errors

Is there a particular reason why we don't catch errors resulting from new WebSocket(...) in Rserve.create()? It's a pretty easy fix to add the line

socket.onerror = handle_error;

so that node.JS doesn't crash just because of a connection error.

Minor issue when deploying in an Express app

After successfully running the quick tour, I went on using rserve-js in the context of an express app.

I used the following snippet:

var r = require('rserve');
r = r.create();

However, if I put console.log(r.running) immediately after, I get "false": I tentatively explain this by assuming the R client is actually running as the result from an asynchronous call.

This becomes a (minor) problem when I want to init an R session prior to REST call route definition: having r.running to false causes (logically) r.eval to crash.

I circumvented with the following snippet:

(function init() {
  if(r.running) {
    r.eval('setwd(".."); source("initScript.R")', function() {
      console.log("Initialized.");
    });
  } else {
    setTimeout(init, 100);
  }
})();

But maybe there's a more elegant solution? I didn't notice a callback argument to r.create(), but I may have missed something?

Other minor point: the version installable via npm (https://www.npmjs.org/package/rserve) crashes. The crash is caused by the following snippet, present in the git version, and missing in the npm version:

opts = _.defaults(opts || {}, {
    host: 'http://127.0.0.1:8081',
    on_connect: function() {}
});

Using the git version in place of the npm version fixes the issue.

Error running Quick Tour

I tried following the Quick Tour directions. Everything worked up until the r.eval line:

> r.eval('rnorm(10)', function(err,a) { if (err === null) { console.log(a) } else {console.log(err) }})
undefined
> [ 'ERROR FROM R SERVER: ERR_inv_cmd 65538 67 16 [object ArrayBuffer]',
  67 ]
[ 'ERROR FROM R SERVER: ERR_inv_cmd 65538 67 16 [object ArrayBuffer]',
  67 ]

I'm on node 4.2, OS X 10.10, and R 3.1.3

Error messages on rclouddocs/Introduction to wdcplot

I got this error and Error in fun(...) atempt to apply non function when running "Inroduction to wdcplot" after executing a number of notebooks. In a new tab it worked.

Error: Mismatched anonymous define() module: [object Object]
http://requirejs.org/docs/errors.html#mismatch
at makeError (https://rcloud.research.att.com/lib/js/require.js:166:17)
at intakeDefines (https://rcloud.research.att.com/lib/js/require.js:1221:36)
at Object.localRequire as require
at requirejs (https://rcloud.research.att.com/lib/js/require.js:1737:24)
at Function.req.config (https://rcloud.research.att.com/lib/js/require.js:1745:16)
at eval (dcplot.js:3:11)
at eval (dcplot.js:43:3)
at rcloud.setup_js_installer.install_js (https://rcloud.research.att.com/js/rcloud_bundle.js:279:34)
at WebSocket.socket.onmessage (https://rcloud.research.att.com/

null always passed to r.eval's callback

I'm new to rserve-js, and just tried to go through the quick tour for a start.
Everything runs fine, until I issue:
r.eval('rnorm(10)', function(a) { console.log(a); })

Instead of something similar to the JSON object shown in the README, I always get null... Even when I try some trivial R expression, such as '1'.

Does someone have a clue on my problem? I run Ubuntu 12.04 (64bit), R 3.0.3, and node v0.11.11. I tried several older versions of node, with the same result.

Support the passing of objects returned by eval() straight to set()

Rserve.create().eval() accepts a callback function with signature (error, robj). Rserve.create().set() should be able to accept such a robj (without any transformations) so that set() effectively is the complement to eval(). This would enable applications to pass objects that require attributes, such as data frames and matrices, to R by constructing a valid SEXP robj.

set() should also accept all results of robj.json() as well, although the result in R may not be exactly the same as the what is generated by eval(). E.g. because of the type ambiguity of arrays that are all NA, set('x', eval(as.numeric(NA))) will save a logical vector to 'x'.

I'm working on a pull request for this now but I won't submit it until pull request #32 is either accepted or denied because there could be a lot of conflicts depending on the outcome.

factors / nominal variables

How can we support R factors?

I can see the factor names are coming across the wire and are in the attributes, somewhere, but I don't quite see how to retrieve them.

I also am not sure whether to replace the int keys with strings, and hope the JavaScript VM uses a dictionary, or to manually annotate with a dictionary.

large payload is not supported

Parameters passed with DT_LARGE as well as SEXPs encoded with XT_LARGE are not supported. This limits the total accepted payload to just under 16Mb (for some versions of Rserve to 12Mb). Also the protocol allows for large encoding 8Mb and up.