anvaka / ngraph.offline.layout

Performs offline layout of large graphs and saves results to the disk

ngraph.offline.layout

Performs offline 3D layout of large graphs and saves results to the disk. This is somewhat experimental.

The results are saved into data folder by default.

usage

// Assume you have a huge graph (instance of ngraph.graph):
var graph = require('ngraph.generators').grid(10000, 10000);
var createLayout = require('ngraph.offline.layout');
var layout = createLayout(graph);
layout.run();

This will run the 3d force-based layout for 500 iterations. Each 5th iteration is saved into ./data/{ITERATION_NUMBER}.bin file.

The layout can be configured with options argument:

// run only 100 iterations
var layout = createLayout(graph, {
  iterations: 100, // Run `100` iterations only
  saveEach: 10, // Save each `10th` iteration
  outDir: './myFolder', // Save results into `./myFolder`
  layout: require('ngraph.forcelayout3d') // use custom layouter
});

After all iterations are completed, the final positions.bin file will be saved into outDir. This file consists of Int32's written in Little Endian format. Each node of the graph is given three integers in the output file. The order of positions matches the order of graph traversal for graph.forEachNode() method.

If the outDir contains data from previous run the layouter will attempt to resume based on the last saved iteration. If you don't want this, you can tell it to overwrite existing files:

var overwrite = true;
layout.run(overwrite);

install

With npm do:

npm install ngraph.offline.layout

license

MIT

ngraph.offline.layout's Issues

'Value is out of bounds' when writing int.

The positions generated by forcelayout3d are almost surely not going to be ints.

This causes the call to writeInt32LE to throw an exception
https://github.com/anvaka/ngraph.offline.layout/blob/master/index.js#L103

I'm not sure about other use cases, but for mine adding "|0" to truncate each dimensions value is fine.

Alternatively writeFloatLE/writeDoubleLE, but your docs say 'int' specifically so I don't know if that's relied upon somewhere.

Offline layout diverges from ngraph.pixel online layout

Hej,

I am prototyping a visualization for 100k+ nodes. Not sure I will actually get there but this is another problem.

I am trying to use the offline layout to avoid layout computations while using ngraph.pixel. However, I see different results in layouting. When using ngraph.pixel the graph nicely expands after a few iterations; ngraph.offline only generates a blob of nodes, even after ~20k iterations. Both strategies use ngraph.forcelayout3d for the layout computation.

I've verified that the positions I load (ArrayBuffer -> Int32Array) correspond to the positions computed using the offline layout.

Are there any adjustments to the physics engine necessary? I am loading the positions after setting renderer.isStable(true) using graph.forEachNode().

TypeError: value is out of bounds

I'm trying to generate a positions.bin file but so far not succeeding:

$ nodejs -v
v0.10.40
$ nodejs
> var graph = require('ngraph.generators').grid(100, 100);
undefined
> var createLayout = require('ngraph.offline.layout');
undefined
> var layout = createLayout(graph);
undefined
> layout.run();
Step 1
Step 2
Step 3
Step 4
Step 5
Saving:  data/5.bin
TypeError: value is out of bounds
    at TypeError (<anonymous>)
    at checkInt (buffer.js:784:11)
    at Buffer.writeInt32LE (buffer.js:916:5)
    at /tmp/node_modules/ngraph.offline.layout/index.js:91:11
    at Object.objectKeysIterator [as forEachNode] (/tmp/node_modules/ngraph.generators/node_modules/ngraph.graph/index.js:496:11)
    at saveIteration (/tmp/node_modules/ngraph.offline.layout/index.js:88:11)
    at Object.run (/tmp/node_modules/ngraph.offline.layout/index.js:52:9)
    at repl:1:8
    at REPLServer.self.eval (repl.js:110:21)
    at repl.js:249:20

Example of how read positions in the frontend

Hello, thanks for your amazing work on graphs.
I'm using ngraph.offline.layout on a server to precompute the positions, however, I have trouble figuring out how to decode the binary data in the frontend.
Should I use dataview ?

Could you add a little example in readme ?
thanks

Deprecated function fs.existsSync(path)

Hello,
I cannot use ngraph.offline.layout using npm because the function fs.existsSync(path) used in index.js is deprecated.

Setting initial positions

Hi,

Thanks for putting this library together, it's very useful.

I'd like to be able to pass in some approximate starting positions (3D or 2D) for each node, but I can't see a way to do it. Is this possible ?

I think it's possible with your C++ version, but that's probably a bit heavy duty for my purposes.

Thanks

incompatible ngraph.graph / forcelayout versions?

Got a NaN node mass while following the pm tutorial, had to edit node_modules/ngraph.forcelayout/index.js: return 1 + (links.length || links.size) / 3.0;; appears to be fixed by matching newest ngraph.graph with ngraph.force.layout

TypeError: layout.getGraphRect is not a function

I am getting this error when trying to use the offline layout here is my code snippet

let offlayout = Viva.Graph.Layout.offline(this.graph, {
      terations: 100, // Run `100` iterations only
      saveEach: 10, // Save each `10th` iteration
      outDir: './cache', // Save results into `./cache`
      layout: require('ngraph.forcelayout3d') // use custom layouter
    });

i also tried to do this

let layout = Viva.Graph.Layout.forceDirected3d(this.graph, {
      springLength: 30,
      springCoeff: 0.0008,
      dragCoeff: 0.01,
      gravity: -1.2,
      theta: 1
    });
    let offlayout = Viva.Graph.Layout.offline(this.graph, {
      terations: 100, // Run `100` iterations only
      saveEach: 10, // Save each `10th` iteration
      outDir: './cache', // Save results into `./cache`
      layout: require('ngraph.forcelayout3d') // use custom layouter
    });

Same problem