erdogant / d3graph

Creation of interactive networks using d3 Javascript

Interactive force-directed network creator (d3graph)

d3graph is a python package that simplifies the task of creating interactive and stand-alone networks in d3 javascript using python. For this package I was inspired by d3 javascript examples but there was no python package that could create such interactive networks. Here it is; a library that automatically creates D3 javascript and HTML code based on an input adjacency matrix in python! This library does not require you any additional installation, downloads or setting paths to your systems environments. You just need python and this library. All other is taken care off. Huray!

This library will create an interactive and stand-alone network that is build on d3 javascript. d3graph only requirs an adjacency matrix in the form of an pandas dataframe. Each column and index name represents a node whereas values >0 in the matrix represents an edge. Node links are build from rows to columns. Building the edges from row to columns only matters in directed cases. The network nodes and edges can be adjusted in weight, color etc, based on user defined paramters.

⭐️ Star this repo if you like it ⭐️

Blogs

Read the blog Creating beautiful stand-alone interactive D3 charts with Python to get a structured overview and usage of d3graph.

Documentation pages

On the documentation pages you can find detailed information about the working of the d3graph with many examples.

Installation

Install from PyPI

pip install d3graph

Import package

from d3graph import d3graph

Examples

Click on the following image to load the interactive Titanic network that is created with d3graph. Note that the relations are determined using HNet. Click here to go to the page with code to make the network.

Example: Changing node properties

Example: Convert source-target list to adjacency matrix

Example: Breaking of networks using slider

---

Contribute

• All kinds of contributions are welcome!

Citation

Please cite d3graph in your publications if this is useful for your research. See column right for citation information.

Maintainer

• Erdogan Taskesen, github: erdogant
• Contributions are welcome.
• If you wish to buy me a Coffee for this work, it is very appreciated :)

d3graph's Issues

adjmat and property-order

Issue

When using d3graph.graph() or d3graph.set_node_properties(), the order of properties provided needs to match the order of the instance attribute adjmat. I don't hink this behaviour is documented and it can lead to some issues.

1. The adjmat attribute does not maintain the same order as the user-provided adjacency matrix. This discrepancy arises because d3graph.graph() calls d3graph.set_edge_properties(), which reorders the matrix into alphabetical order.
2. Attempting to provide properties in alphabetical order may fail. d3graph.graph() calls data_checks, which calls remove_special_chars. This changes the node names, which can also change their alphabetical order (for example if the name start with a special character, like "µm_wave"). Thus d3graph.set_edge_properties() might produce a different order for adjmat than one would obtain ordering the user-provided adjacency matrix.

Suggested fix

• Document that properties need to given in alphabetic order.
• Make the removal of optional chars optional and/or issue a warning if special chars are removed from the node names

_do_checks attempts to convert version strings to float

In the _do_checks function from d3graph.py, there is an assert float(nx.__version__)>2. This is fine if the version only has two levels, but my current version of networkx is '2.8.8', so it throws:

ValueError: could not convert string to float: '2.8.8'

Consider changing to something like the following:

assert float('.'.join((nx.__version__).split('.')[:2]))>=2

Trying to get example to work in google collab

This basic code is from the example:

`!pip install d3graph
from d3graph import d3graph, vec2adjmat

Create example network

source = ['node A','node F','node B','node B','node B','node A','node C','node Z']
target = ['node F','node B','node J','node F','node F','node M','node M','node A']
weight = [5.56, 0.5, 0.64, 0.23, 0.9, 3.28, 0.5, 0.45]

Convert to adjacency matrix

adjmat = vec2adjmat(source, target, weight=weight)

Initialize

d3 = d3graph()

Proces adjmat

d3.graph(adjmat)

Plot

d3.show()`

But this does not appear to work.
It does seem to be generating the html, but not showing it in google collab.

Would it be possible to have some example code that shows how to get the project to work on that platform?

-ft

Custom Slider Range

d3graph takes slider parameter, but show() calls setup_slider(), which defaults the range to minimum and maximum edge weight. Or am I doing something wrong?

d3graph NetworkXError: Adjacency matrix not square:

I don't what's causing this error , I have 2 lists with the same length but I keep getting this . help pls !

ValueError from Jinja PackageLoader

When installing d3graph from pip I got the following error when initializing like this d3 = d3graph.d3graph().

ValueError: The 'd3graph.d3graph' package was not installed in a way that PackageLoader understands.

Error seems to be in line 538 of d3graph.js. I switched that line to the following statement, which solved the problem for me...

jinja_env = Environment(loader=PackageLoader('d3graph', package_path='d3js'))

Not sure if that is just me, or a general issue

Quickstart not work

Hello
I try
https://erdogant.github.io/d3graph/pages/html/Installation.html#quickstart
get error

---

TypeError Traceback (most recent call last)
in
10
11 # Initialize
---> 12 d3 = d3graph()

TypeError: d3graph() missing 1 required positional argument: 'adjmat'

I will change to

import d3graph
d3graph.d3graph(adjmat=adjmat)

Support for 3-dimensional layout

Networkx supports 3D network layouts, e.g. cords = nx.spring_layout(G, dim = 3, ...)

Any plans of adding support for this? I'm not too familiar with D3 but seems like it should be possible.

Is it possible to change the color of the nodes when they get clicked on ?

I mean, when I click on a node it's color changes from blue to red, and when I click on another node this new node changes from blue to red while the previous selected node goes back to blue

Color of the name of the nodes

Hello everyone.

Thanks for the amazing library. I am using it to display the Human Connectome in an interactive manner and I really like the result.

However, I was wondering if I could change the color of the names of the nodes? It's the same as the colors of the edges and when you have many nodes - like in my case - it makes it difficult to find the nodes of interest.

Best,
Davide

Is there a way to start the graph at a specific threshold of edges?

Hi,

I'm using your package to render some networks, and it works great.
I was wondering if there is a way to set the d3.show() to start at a specific value for the threshold of edges?

Thanks a lot,

Mario

Feature Request: Select Marker Shape for Nodes/Edges

Via Node/Edge, it would be nice to be able to select a few more marker shapes. I am not strong enough in JS to understand this, but I believe it is possible with SVG markers based on this link:

http://bl.ocks.org/dustinlarimer/5888271

I think this is where directed edge arrows come from, and it would be great if we could select from a few basic shapes like circle (Default as it currently is), Square, Triangle, Diamond, Star. This would give us the ability to have a great selection of nodes and edge shapes to work with, without having to do custom images. (While interesting, SVG shapes makes everything play well I think).

A stretch goal for this request would be the ability to select a Marker and Color and create a definition that can be displayed as a Key. It wouldn't need to be complicated, it could work something like this:

d3_toc = {
                  ('circle', 'black'): "Actors",
                  ('triangle', 'blue'): "Movies",
                  ('square', ''yellow'): "TV Series"
}

and then that could be passed to an optional ToC that gets displayed on the HTML. In addition to the markers shape/color the ability to put just lines of text would be good. You could have the ToC and then you could say "Node Sizes indicate popularity" or "Edge Widths determine number of roles". It would be a manual thing, but the ability to have a few extra shapes, as well as some notes on what we are seeing would look fantastic.

Remove edges label when they are filtered out by the threshold

I have a big graph and when I want to remove small edges I keep having their label floating around. It makes it unreadable.

Here is a screenshot from the doc:

Is there a way to remove them dynamically?

Handle/Control Warnings about Dataframe if highly fragmented

When working with a graph with lots of edges/nodes (3846 items) I got a full screen of warnings related to the d3graph.py:903

PerformanceWarning: DataFrame is highly fragmented. This is usually the result of calling frame.insert many time, which has poor performance. Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame use newframe=frame.copy()
adjmat[node]=0

It would be nice to do something that gets the same result here without pushing the warning at us (it makes a Jupyter Notebook real ugly).

Note, it still worked fine, it's just a nice to have.

d3graph error

code:
from d3graph import d3graph, vec2adjmat
source = ['node A','node F','node B','node B','node B','node A','node C','node Z']
target = ['node F','node B','node J','node F','node F','node M','node M','node A']
weight = [5.56, 0.5, 0.64, 0.23, 0.9,3.28,0.5,0.45]

adjmat = vec2adjmat(source, target, weight=weight)
out = d3graph(adjmat)

i got this error:
'charmap' codec can't encode character '\u03b5' in position 44258: character maps to

Fixing the position of nodes

Hi, is there a way to specify custom XY coordinates in node properties? I want to arrange the nodes in various layouts.

Adding click event handler

The show method has a click parameter that accepts a dictionary with some style information:

show(click = {'size': 1.25, 'stroke-width': 3}, ...

This is useful but it's missing the opportunity for interaction. Adding a callback could be used to take some custom action (in addition to styling the node) such as populating a separate HTML element with node details based on the node ID.

`d3graph.set_node_properties` error when provided with a list of colors

Description

Using d3graph.set_node_properties() with a color-parameter like ['A','A','B',...] results in a ValueError raised by _check_hex_color():
'[color] contains incorrect length of hex-color! Hex must be of length 7: ["#000000", "#000000", etc]'

Expected behaviour

The color-parameter is used together with cmap to generate a list of hex-colors

Sugested fix

Apply _get_hexcolor() to the color parameter if the user passed a list of str.

Generalize output for existing HTML template

The stand-alone HTML page is great but an option to include the output in an existing page would be very useful. Instead of a stand-alone HTML page, it could generate content that you can paste directly into an existing HTML document. Right now the script attaches the SVG tag directly on BODY, for example.

Change text size of the node labels

Hi,

Is there a way to check the size and color of the fonts of the node label?

Suggestion : Edge style

Thanks for sharing such a great tool to draw graph!

It would be great if we can adjust different line-styles with edge properties such as dotted('.'), dashed('-')

Allow Jinja2 Version to move beyond 3

Many of the latest Jupyter Notebook Envs require a newer version of Jinja2, but requirements have it locked. I am going to test without locking, but I thought if it's locked for a reason that we talk it through.

Question: Will d3graph be maintained independantly from D3Blocks?

I saw an article on D3 blocks (although I can't find the git repor)

• Will d3graph (which appears part of d3blocks be maintained separately
• Will d3blocks version of d3graph have all the functionality of d3graph?
• When will the d3blocks repo be up and running on git hub?

Thanks!

Directed edges have inconsistent direction

It can be seen from your big bang example
Big Bang Directed Arrows
source = ['Penny', 'Penny', 'Amy', 'Bernadette', 'Bernadette', 'Sheldon', 'Sheldon', 'Sheldon', 'Rajesh']
target = ['Leonard', 'Amy', 'Bernadette', 'Rajesh', 'Howard', 'Howard', 'Leonard', 'Amy', 'Penny']
Some arrows show direction from target to source but they should be from source to target.
In this instance it affects all arrows with Sheldon as source.
Sheldon who only appears in the source list but in the rendered network all the arrows on the edges point towards his node.

The most likely culprit is the js code as I cannot see it in the python portions.
JS Code

Type object is not subscriptable.

Hi,

When trying to use the d3graph library I get the following error:

TypeError: 'type' object is not subscriptable

878 minmax: list[float] = [0.5, 15],
879 minmax_distance: list[float] = [50, 100],

I believe this should be changed to:

878 minmax: list = [0.5, 15],
879 minmax_distance: list = [50, 100],

core examples missing

Hello,

just starting with d3graph so any guidance is greatly appreciated.

Unfortunatetly the example files mentioned here https://erdogant.github.io/d3graph/pages/html/Core_Functionalities.html can not be found in the gitrepo, can you kindly add them ?

Hide slider

Hello,

Thanks for the amazing work on this library. I was wondering if it is possible to disable or to hide the "Link Threshold" slider at the top of the graph ? I did not find any parameter to do it in the code.

Thanks for your help.

setting the path with d3graph.set_path does not seem to take the variable

I am starting to use d3graph in a notebook part of a Jupyter hub.
I need to change the default path where d3graph saves the html files

but d3gaph does not seem to take the folder given:

Is this an issue because I am working in a jupyter notebook?

Markupsafe version

Hey thanks for the nice project.

Can the markupsafe version be updated to be less restrictive?

d3graph/requirements.txt

Line 4 in a14b8e6

markupsafe==2.0.1

Changing Edge Color and Hover on Edge

Hello,

I tried d3.edge_properties[edge]['color']='#FF0000' but that does not seem to work.

Also, is hover on edge supported?

Return value length

Suggestion: Deprecate d3graph and have it be part of d3blocks

I think while it may cause a bit of pain at first, the updates/maintenance of both projects would gain value from being moved together.

Support sparse data types / make this the internal data structure for the graph

Summary

When loading a graph into d3graph users must provide a dense adjacency matrix. This is inconvenient because most real-world graphs are sparse, and the dense representation is memory-ineffiecient.

Furthermore, the d3graph class saves a copy of the adjacency matrix for its internal representation of the data when the .graph() method is called - but everytime this self.adjmat is used, it seems to have been more natural to have stored it in a different structure.
E.g set_edge_properties first converts the matrix into a dict (which is a sparse representation) and get_cluste_color effectively converts to an edgelist as far as I can tell (also sparse). Both of these operations are fairly inefficient and could have been avoided had a different structure been used to begin with. Adjacency matrices of graphs tend to be most useful when the algorithm we want to apply can be expressed as linear algebra, e.g. shortest paths, pagerank or traversals. Even in most of these situations, sparse matrices are usually the go-to for scalable solutions.

Impact

• d3graph is limiting the sizes of graphs it supports by enforcing an inefficient input type.
• d3graph is increasing its runtime due applying inefficient operations on this dense data structure when generating a figure

Possible Solutions

My preferred suggestion: since we already have networkx as a dependency, it would be wise to delegate the responsibility of storing the graph structure to that package, since:

• it's a trusted, respected and mature project - no doubt they have applied more thought on efficiency than we could here. It's also unlikey to find bugs
• its API is very stable
• users will be familiar with the API
• NetworkX already has great conversion functions to/from other data structures, meaning less inefficient conversions and boilerplate here.
• would make the python portion of the code much shorter and more readable (adjmat2* methods could be removed)