imohitmayank / jaal

Your interactive network visualizing dashboard

Hi there, it's Mohit Mayank @ mywebsite 👋. I'm the Lead Data Scientist @ Outplay

• 🔭 I’m currently working on creating Cognitive solutions for Enterprise problems!
• 🌱 I’m currently learning everything 🤣
• 👯 I’m looking to collaborate in Natural Language Processing and Graph Neural Networks.

Connect with me:

Languages and Tools:

---

📕 Latest Blog Posts

• Introduction to Graph Neural Networks with DeepWalk
• Ranking algorithms — know your multi-criteria decision solving techniques!
• Co-variance: An intuitive explanation!
• Interactive Q learning
• The Invariant principle — Introduction and examples

---

🎯 Latest Projects/Talks

• The Art behind Fractals
• Chaos Game
• Interactive Q Learning
• Koch Snowflakes
• Landscape 3D

---

jaal's Issues

InvalidCallbackReturnValue error when trying to run the app

Getting the following error message when trying to launch the application:

Dash is running on http://127.0.0.1:8050/

 * Serving Flask app "test" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:8050/ (Press CTRL+C to quit)
127.0.0.1 - - [02/Feb/2021 13:36:54] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [02/Feb/2021 13:36:55] "GET /_dash-dependencies HTTP/1.1" 200 -
127.0.0.1 - - [02/Feb/2021 13:36:55] "GET /_dash-layout HTTP/1.1" 200 -
No trigger
[2021-02-02 13:36:56,269] ERROR in app: Exception on /_dash-update-component [POST]
Traceback (most recent call last):
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\dash.py", line 1041, in add_context
    response, cls=plotly.utils.PlotlyJSONEncoder
  File "C:\Program Files\Python_3_7_4\lib\json\__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\_plotly_utils\utils.py", line 59, in encode
    encoded_o = super(PlotlyJSONEncoder, self).encode(o)
  File "C:\Program Files\Python_3_7_4\lib\json\encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "C:\Program Files\Python_3_7_4\lib\json\encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\_plotly_utils\utils.py", line 134, in default
    return _json.JSONEncoder.default(self, obj)
  File "C:\Program Files\Python_3_7_4\lib\json\encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type type is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\_compat.py", line 39, in reraise
    raise value
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\dash.py", line 1078, in dispatch
    response.set_data(func(*args, outputs_list=outputs_list))
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\dash.py", line 1044, in add_context
    _validate.fail_callback_output(output_value, output)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\_validate.py", line 261, in fail_callback_output
    _validate_value(val, index=i)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\_validate.py", line 256, in _validate_value
    toplevel=True,
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\_validate.py", line 210, in _raise_invalid
    bad_val=bad_val,
dash.exceptions.InvalidCallbackReturnValue: The callback for `<Output `graph.data`>`
returned a value having type `type`
which is not JSON serializable.


The value in question is either the only value returned,
or is in the top level of the returned list,

and has string representation
`<class 'dash.exceptions.PreventUpdate'>`

Can't display a big graph

Hello!
Thanks for this wonderful package! I've been playing with random graphs and it works great. However after trying to display a "big graph" (44 nodes and ~300 edges) something weird happens. It seems unable to stabilize the graph and it doesn't stop moving. Do you know how to fix this?
Thank you!

how to save jpeg or png publication quality file

Hello - I've been using your tool which is great however I can't work out how to save the output to an image file. I saw there was an enhancement post on this but couldn't see what the solution was. I can't use a screenshot for a publication.

Thanks!

Jaal not working due to dash_bootstrap_components bump!

On running the 0.1.1 version of Jaal, we get following error message,

AttributeError: FormGroup was deprecated in dash-bootstrap-components version 1.0.0. You are using 1.0.1. For more details please see the migration guide: https://dash-bootstrap-components.opensource.faculty.ai/migration-guide/

Potential solution is to put make sure to use older version in python package requirements (dash-bootstrap-components<1)

How i can use jaal with flask

I am building application on flask for network visualization .I want to integrate jaal to my web app .Is there any way to achieve this .As jaal is using dash i know we can integrate dash with flask but for jaal i dont understand the way .if you can provide me any sample so i can do it

When filtering edge hide objects not filtered

when you have a large number of objects it would be very helpful to remove the objects that don't meet the filter criteria.

Add basic live graph properties changes options.

Launch in Dash

This is a reallly fun project, have you perhaps already developed a way to integrate this within a dash app (embed it somehow), if so a tutorial would be appreaciated, kudos for the project.

[Feature Request] Display label/title instead of id on nodes

Hi there,
I wondered if it is possible to display the title or a label under the nodes instead of the id. In my case the ids are urls wich are quite long. So it would be great to show the (shortened) website title instead wich right now is passed in the node_df and thus shown as tooltip. I tried something like this:

Jaal(edge_df, node_df).plot(directed=True, vis_opts={'nodes': {'label': 'node_df[title]'}}) # not working
Jaal(edge_df, node_df).plot(directed=True, vis_opts={'nodes': {'label': 'title'}}) # not working either

I also tried to pass the labels in a label column in the node_df (like in the example with edge labels).

Here you can see how crowded it becomes even with 4 nodes.

As a workaround I could create new ids (with numbers eventually combined with the website title) and show the url in the tooltip. But I think it might be handy to just specify the node labels as well.

Add legends for node and edge colouring

Show the legends, which map the colour of node or edges with the feature value. (Feature suggested in #11)

Export the current graph as image

We can have an export option to save the currently visible graph as an image (png or jpeg format).

Note, as of now, we can use any 3rd party screenshot application to take the image of the visible graph. "Greenshot" is one such application.

how can do weighted graphs

Is there a we can use Jaal for making weighted graphs.

Images as Nodes

It would be useful for me to be able to use images (stored as .png files) as nodes - is it possible to do this by adding an attribute to the node properties? Ideally, the node shape and size would be retained but instead of a solid color fill the node would contain the image.

Add the data loader options. It should be able to read at least from the CSV files.

UI Feature Request

Descriptions

There's a few suggestions I think would be awesome.

Double Clicking a node removes all other un-connected nodes

I'd like to be able to do this to identify the relationships within one single group. Having a lot of groups (and nodes) can be very noisy on the screen and not appealing. Alternatively might be to just dim/reduce opacity every other resource not that nodes tree.

Toggle removes a node cluster of 1

I find this would be super neat if we could remove single nodes as I'm more focused on the relationships than seeing a huge sea of nodes. This is the same approach as filtering, except more of a "show me nodes with at least 1 connection".

Legend for the color and slow down the movement of the node

Hi,

Thank you for this cool visualization.
I am trying to use this for my network. It works really well.
I have some suggestions as follows.

1. When coloring the nodes/edges, it's hard to tell which color is what. So, it would be great to see the legend
2. In my network, some nodes have quite many neighbors which make the sub-graph moving quite fast. It may be better to keep the network more still.

Thank you!

Live updating of graph

Is it currently possible to set a refresh interval so that the network updates whenever the underlying data changes?

Graph taking a lot of time to plot

Using medium-sized datasets (approx 1500 edges), the time to plot the graph is exceeding 30 mins. Need to test the app on some medium-sized datasets and fix.

JAAL not plotting graphs when a pair of nodes have more than one edge

Fantastic library you have made available Mayank..

One little problem.

If you need to plot more than one edge for a pair of nodes, the graph simply is not rendered ...

CASE A: This works
A---->B
A---->C

CASE B: This does not work. And it does not show any error.
A<----B
A---->B

I have added the directed=True option...
It does work for case A.

Case B does not work at all.

Any option that I may be missing ?

Thanks in advance

Server runs but the graph won't load.

I have a pandas dataframe of edges:

     from   to
0       1   39
1       3   41
2       5   43
3       7   45
4       9   47
...   ...  ...
1507  805  807
1508  807  809
1509  809  811
1510  811  813
1511  813  779

[1512 rows x 2 columns]

from    object
to      object
dtype: object

The server runs and I see the dashboard but the graph does not load.

I am getting this error:

Dash is running on http://127.0.0.1:8050/

 * Serving Flask app "test" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:8050/ (Press CTRL+C to quit)
127.0.0.1 - - [02/Feb/2021 13:36:54] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [02/Feb/2021 13:36:55] "GET /_dash-dependencies HTTP/1.1" 200 -
127.0.0.1 - - [02/Feb/2021 13:36:55] "GET /_dash-layout HTTP/1.1" 200 -
No trigger
[2021-02-02 13:36:56,269] ERROR in app: Exception on /_dash-update-component [POST]
Traceback (most recent call last):
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\dash.py", line 1041, in add_context
    response, cls=plotly.utils.PlotlyJSONEncoder
  File "C:\Program Files\Python_3_7_4\lib\json\__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\_plotly_utils\utils.py", line 59, in encode
    encoded_o = super(PlotlyJSONEncoder, self).encode(o)
  File "C:\Program Files\Python_3_7_4\lib\json\encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "C:\Program Files\Python_3_7_4\lib\json\encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\_plotly_utils\utils.py", line 134, in default
    return _json.JSONEncoder.default(self, obj)
  File "C:\Program Files\Python_3_7_4\lib\json\encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type type is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\_compat.py", line 39, in reraise
    raise value
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Program Files\Python_3_7_4\lib\site-packages\flask\app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\dash.py", line 1078, in dispatch
    response.set_data(func(*args, outputs_list=outputs_list))
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\dash.py", line 1044, in add_context
    _validate.fail_callback_output(output_value, output)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\_validate.py", line 261, in fail_callback_output
    _validate_value(val, index=i)
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\_validate.py", line 256, in _validate_value
    toplevel=True,
  File "C:\Program Files\Python_3_7_4\lib\site-packages\dash\_validate.py", line 210, in _raise_invalid
    bad_val=bad_val,
dash.exceptions.InvalidCallbackReturnValue: The callback for `<Output `graph.data`>`
returned a value having type `type`
which is not JSON serializable.


The value in question is either the only value returned,
or is in the top level of the returned list,

and has string representation
`<class 'dash.exceptions.PreventUpdate'>`

In general, Dash properties can only be
dash components, strings, dictionaries, numbers, None,
or lists of those.
127.0.0.1 - - [02/Feb/2021 13:36:56] "POST /_dash-update-component HTTP/1.1" 500 -

Deploy In Heroku

Hi
I can run the app in local easily
But after following the heroku guide in official dash site at Link
It says that server address is wrong which seems related to plot() server and port parameters
How can i setup Jaal to run in heroku ?
Thanks for Great repo

Use features to determine node size and edge thickness

Great work so far, thanks very much.

How might you use a node feature to determine its size and similarly an edge feature for its thickness?

"shape" property of nodes in vis can not be customized

a series of shape of nodes , like triangle, dot, circle, etc. however, it can not be customized in vis_opts.

IncorrectTypeException

Facing issue "IncorrectTypeException" while running the code. Screenshot attached. Please check.

Update information for node

Hi there,
I want to update information for a node. Do you support this?
thank you.

The state keyword argument may not be provided without the input keyword argument

Trying to run this on a vanilla python 3.9 project through PyCharm.

(pip) installed jall through PyCharm interface, it loaded correctly with all listed dependencies.

Copied sample code in main.py:
# import
from jaal import Jaal
from jaal.datasets import load_got
# load the data
edge_df, node_df = load_got()
# init Jaal and run server
Jaal(edge_df, node_df).plot()

When running, got a bunch of warning about deprecated imports from dash, plus the following error output/error:
\venv\Scripts\python.exe /main.py

Parsing the data...Done
Traceback (most recent call last):
  File "C:\Users\fdion\PycharmProjects\pythonProject1\main.py", line 24, in <module>
    Jaal(edge_df, node_df).plot()
  File "C:\Users\fdion\PycharmProjects\pythonProject1\venv\lib\site-packages\jaal\jaal.py", line 343, in plot
    app = self.create(directed=directed, vis_opts=vis_opts)
  File "C:\Users\fdion\PycharmProjects\pythonProject1\venv\lib\site-packages\jaal\jaal.py", line 272, in create
    @app.callback(
  File "C:\Users\fdion\PycharmProjects\pythonProject1\venv\lib\site-packages\dash\dash.py", line 1011, in callback
    return _callback.register_callback(
  File "C:\Users\fdion\PycharmProjects\pythonProject1\venv\lib\site-packages\dash\_callback.py", line 116, in register_callback
    ) = handle_grouped_callback_args(_args, _kwargs)
  File "C:\Users\fdion\PycharmProjects\pythonProject1\venv\lib\site-packages\dash\dependencies.py", line 308, in handle_grouped_callback_args
    inputs_state = extract_grouped_input_state_callback_args(flat_args, kwargs)
  File "C:\Users\fdion\PycharmProjects\pythonProject1\venv\lib\site-packages\dash\dependencies.py", line 250, in extract_grouped_input_state_callback_args
    raise ValueError(
ValueError: The state keyword argument may not be provided without the input keyword argument

Process finished with exit code 1

I fixed the deprecated imports following the instructions in the warnings, but the "state keyword argument" issue is still there.
Not sure how to proceed from here. Thanks.

Pipe/stream compatibility

Hi, I'm looking for interactive visualization of (potentially infinite) "streaming data" (imagine a unix pipe) which can "come & go" any time. Currently it seems this is not easily possible with Jaal.

My question is whether you'd consider supporting this.

Thanks!

Add basic filtering and searching options.

Limit node locations to visible area

Thank you for creating Jaal! It's very fun :D

Once request I have is this:
When I filter edges, nodes the end up disconnected (no incoming/outgoing edegs) tend to "fly out" of the visible area. Right now I need to zoom out to see them again, in the hope I get them all. Is it possible to limit the nodes so that they don't fly out of the visible part of the canvas? Thanks!

To reproduce (I'm using Chrome on OSX):

If you run the Jaal demo code and in the browser filter edges using weight > 90,
as the filter is set you will see that disconnected nodes get "pushed away", and some of them end up disappearing.
As a user, if I don't notice some have disappeared and zoom out, I will simply not see them (and miss out the notion of how some nodes get disconnected).

An alternative would be some button to set the zoom value so that all nodes are visible.

Thanks!

Unable to have nodes with >25 edges?

I am attempting to make a large network with nodes that extend to many others. I have found that when a node has over 25 edges, the entire network disappears when I attempt to run Jaal. I am wondering if this is a known issue?

Max width for edges and nodes

Hi,
I have problems setting the max width for edges. I tried many of the visjs-settings like value, width, widthConstrain resulting in a common width for all edges or without any effect. I find the default max width to big.

I also tried Jaal(edge_df, node_df).plot(directed=True, vis_opts={'physics':{'stabilization':{'iterations': 100}, 'solver' : 'repulsion'}, 'edges': {'scaling': {'min': 1, 'max': 2}}}) with no result (taken from layout.py.

Same would be good for node size as well. Especially if you have only few different values (e. g. 1 and 2) it becomes really obscure:

As a work around for the edges I found the posibility to add an edge that connects two non existent nodes and has higher value in the size determing column. E.g. a 10. It won't show up.

For nodes this is not so easy, because the node will show up anywaay. But you can filter it out. If you add a node that you filter out anyway you can use its id as from and to in the edge_df as well:

then becomes

Sincerely Simon

Support directed graphs

Currently, Jaal doesn't support Directed graph. This functionality should be fairly easy to add, as for starters we just need to have directed edges.

Blank plot using own data

Hi Mohit, thanks for this library. The game of thrones test data works fine but when I use my own data set the plot is blank. The settings panel appears but no nodes or connections. I've attached my node and edge CSVs - though I have tried with just the edge_df and I get the same result. Am I missing something, is there a problem with my data?

status_mapping_edge.csv
status_mapping_node.csv

Node Filtering with Uppercase Letters

Hey awesome work! I ran into an issue with node filtering for uppercase letters. If I have a node label 'AB12345', noticed that trying to filter for 'AB' excluded 'AB12345'.

looking at _callback_search_graph(), the condition is to find search_text in node['label'].lower(), so that searches for upper case letters fail.

Then again this does not seem to occur in your demos, so maybe i just set up the input dataframe wrong. anyway dont do this a lot but wanted to share an issue I experienced because my experience with jaal has been great so far

jaal/jaal/jaal.py

Line 45 in 74a7d2c

if search_text not in node['label'].lower():

Filter by "from" it doesn't work

All other columns of the filter on edges work, except from:

from == "Jon-Snow"

error while running this code

from jaal import Jaal
from dash import dcc
from jaal.datasets import load_got
edge_df, node_df = load_got()
Jaal(edge_df, node_df).plot()

ValueError Traceback (most recent call last)
in
1 # init Jaal and run server
----> 2 Jaal(edge_df, node_df).plot()

~/anaconda3/envs/python3/lib/python3.6/site-packages/jaal/jaal.py in plot(self, debug, host, port, directed)
148 Input('color_nodes', 'value'),
149 Input('color_edges', 'value')],
--> 150 state=State('graph', 'data')
151 )
152 def setting_pane_callback(search_text, filter_nodes_text, filter_edges_text, color_nodes_value, color_edges_value, graph_data):

~/anaconda3/envs/python3/lib/python3.6/site-packages/dash/dash.py in callback(self, *_args, **_kwargs)
1138 self.config.prevent_initial_callbacks,
1139 *_args,
-> 1140 **_kwargs,
1141 )
1142

~/anaconda3/envs/python3/lib/python3.6/site-packages/dash/_callback.py in register_callback(callback_list, callback_map, config_prevent_initial_callbacks, *_args, **_kwargs)
114 inputs_state_indices,
115 prevent_initial_call,
--> 116 ) = handle_grouped_callback_args(_args, _kwargs)
117 if isinstance(output, Output):
118 # Insert callback with scalar (non-multi) Output

~/anaconda3/envs/python3/lib/python3.6/site-packages/dash/dependencies.py in handle_grouped_callback_args(args, kwargs)
312 outputs = outputs[0]
313
--> 314 inputs_state = extract_grouped_input_state_callback_args(flat_args, kwargs)
315 flat_inputs, flat_state, input_state_indices = compute_input_state_grouping_indices(
316 inputs_state

~/anaconda3/envs/python3/lib/python3.6/site-packages/dash/dependencies.py in extract_grouped_input_state_callback_args(args, kwargs)
255 # Not valid to provide state as kwarg without input as kwarg
256 raise ValueError(
--> 257 "The state keyword argument may not be provided without "
258 "the input keyword argument"
259 )

ValueError: The state keyword argument may not be provided without the input keyword argument

[Feature Request] Passing arguments to Dash framework

Hi there,
for me it would be handy being able to pass arguments to the Dash framework during development. Thus I could see if my network will look as I intend to after filtering and color coding nodes and edges as I would do in a live presentation step by step.

Also it would be handy to access variables defined in the outer environment or being able to passing it in the function call as well.

For example I color code the edges by edge title and filter for specific titles:

Instead of title in ['side-nav', 'header', 'content'] it would be simple to do title in @single_place_links.

Maybe with a call like this: Jaal(edge_df, node_df).plot(directed=True, dash_opts={'color_edges' : 'title', 'filter_edges' : 'title in @single_place_links'}, args = single_place_links)

Sincerely Simon

Filter by the Content of a Column

The most important thing first: Great work thank you very much!

Is it possible to define a filter with the content of a column?

Description of the problem:
I have an edge CSV file with three columns: from, to, group. I have no CSV for the nodes.

Different groups of edges are defined in the Group column. The idea is to use these groups as a filter selection list. Then the user could simply filter via this selection list. Just as he can select the group under "Color edges by". So actually exactly what comes as "Edge legends" list.

Improvement for "Filter on edges properties":
If I enter the following Filter group=='Network internal' unfortunately all nodes not belonging to the group continue to be displayed and only the connections are removed. Here it would be desirable if these stand-alone nodes were also removed.

Concern about the tooltip

Hello Mohit,

The project is amazing! Good work! Just wondering if there is any way to implement the tooltip functionality; for example, move the mouse to the "Jon Snow" node, tooltip information such as name, gender will be shown. Move the mouse to an edge, two nodes' information such as weight will be shown? Thank you!