wiheto / teneto

Temporal Network Tools

Teneto

Temporal network tools.

What is the package

Package includes various tools for analyzing temporal network data. Temporal network measures, temporal network generation, derivation of time-varying/dynamic connectivities, plotting functions.

Some extra focus is placed on neuroimaging data (e.g. compatible with BIDS - NB: currently not compliant with latest release candidate of BIDS Derivatives).

Installation

With pip installed:

pip install teneto

to upgrade teneto:

pip install teneto -U

Requires: Python 3.6+

Installing teneto via pip installs all python package requirements as well.

Documentation

More detailed documentation can be found at teneto.readthedocs.io and includes tutorials.

Outlook

This package is under active development. And a lot of changes will still be made.

Contributers

For a list of contributors to teneto, see: teneto.readthedocs.io

Cite

If using this, please cite us. At present we do not have a dedicated article about teneto, but you can cite the software using the Zenodo DOI and/or the article where teneto is introduced, along with a considerable discussion about many of the measures in teneto:

Thompson et al (2017) "From static to temporal network theory applications to functional brain connectivity." Network Neuroscience, 2: 1. p.69-99 Link

teneto's Issues

How to make a community slice plot

Assume we have a temporal network as below.

import numpy as np
import teneto
import matplotlib.pyplot as plt
A1 = np.array ( [[0., 0., 0., 0., 0,0,0],[5., 0., 0., 0., 0,0,0],[1., 0., 0., 0., 0,0,0],
                 [0., 1., 2., 0., 0,0,0],[0., 0., 0., 1., 0,0,0], [0., 0., 0., 1., 0,0,0],
                 [0., 0., 0., 0., 1,1,0]] )


A2 = np.array ( [[0., 0., 0., 0., 0,0,0],[5., 0., 0., 0., 0,0,0], [1., 0., 0., 0., 0,0,0],
                 [0., 1., 2., 0., 0,0,0], [0., 0., 0., 1., 0,0,0],[0., 0., 1., 1., 0,0,0],
                 [0., 0., 0., 0., 1,1,0]] )

A3 = np.array ( [[0., 0., 0., 0., 0,0,0], [0., 0., 0., 0., 0,0,0],[0., 0., 0., 0., 0,0,0],
                 [0., 1., 2., 0., 0,0,0],[0., 0., 0., 1., 0,0,0],[0., 0., 0., 1., 0,0,0],
                 [0., 0., 0., 0., 1,1,0]] )
g_arr=np.array((A1, A2, A3, )).T

tnet = teneto.TemporalNetwork(from_array=g_arr)
ax = tnet.plot('slice_plot')
plt.show()

and, assume, we obtain a communities which represented as below

communities = np.array([[0, 0, 0], 
                            [0, 0, 1], 
                            [0, 1, 0], 
                            [0, 0, 0],
                            [1, 1, 0],
                            [0, 0, 2],
                            [0, 0, 2]])

May I know, how to plot these communities to get something as shown in the figure below?

In the figure above, let membership 0,1,2 labeled as blue, red and yellow, respectively

one work around I can think of is, by manually adjust the node as

ax.scatter([0,0,0], [0,1,2], s=100, color='red', zorder=100) # along this line of idea

tnet.T incorrect when tnet is sparse and starttime!=0

Running

import teneto as tnt
import pandas as pd

netin = {'i': [0,0,1,1], 'j': [1,2,2,2], 't': [100,101,102,103]}
df = pd.DataFrame(data=netin)
tnet = tnt.TemporalNetwork(from_df=df)

print(tnet.T)

displays 104, even when specifying starttime=100 at the creation of the network. From the documentation, T is

number of time-points in network

so that it should be 4.

error in dataframe undirected network

Mentioned in #52:

File "", line 1, in
shortest_paths = tnt.networkmeasures.shortest_temporal_path(t)

File "C:\ProgramData\Anaconda3\lib\site-packages\teneto\networkmeasures\shortest_temporal_path.py", line 201, in shortest_temporal_path
network = tnet.get_network_when(ij=list(ij), t=t)

File "C:\ProgramData\Anaconda3\lib\site-packages\teneto\classes\network.py", line 483, in get_network_when
return teneto.utils.get_network_when(self, **kwargs)

File "C:\ProgramData\Anaconda3\lib\site-packages\teneto\utils\utils.py", line 993, in get_network_when
network['j'].isin(ij))), (network['t'].isin(t)))]

TypeError: and_ expected 2 arguments, got 1

I can not update teneto

@wiheto , I can not update teneto, some bugs need fix. :)

Add communitymeasures

Different metrics for quantifying communities (e.g. measures from Network Community Toolbox. http://commdetect.weebly.com/)

• flexibility (already merged)
• recruitment (#21 )
• persistence
• promiscuity
• Integration coefficient (#30 )

These are some examples taken from http://commdetect.weebly.com/

Also something I've been working on: MISI once I've cleaned up the code.

How can I use `graphlet_stack_plot` from a `TemporalNetwork` object?

I have a weighted undirected network with 3 timepoints. How can I use this with your plotting functions? It seems the TemporalNetwork is the main object for this package so I'm confused on how I can use this with the plotting functions.

Speed up the shortest path calculation speed

Hi, I ran into an issue in using the shortest path calculation in large networks with lots of nodes and I was wondering if there is any way we can speed up the calculation speed? and if this is going to be fixed in the next versions how long do you think does it take?

An error: The object 'TemporalNetwork' can't import dataframe data of weighted network

Hello!

I get a problems when I run the following:

In theory, I import a weighted network by the dataframe data. However, I get an unweighted network with TemporalNetwork object.

There is no error when I import the dataframe data, but when I continue to use the tnet with other methods like networkmeasures.temporal_degree_centrality, an error appears.

I can solve my problem with other importing method like list of edge, but I wonder why this situation happens.

[ENH] TCTC implementation for 0.4.4

Mandetory:

• TCTC functions.
• tests

Desired (but could be delayed for future release):

• HDF5 TCTC - implementation for large datasets (problem - too slow at the moment)
• TVCTC
• Optimization functions and possible cost functions (Need to think how to implement this in a - good way).
• TenetoBIDS integration

Also to-do asap

• Tutorials and/or notebook using TCTC

wrong number of nodes from TN.N and in netshape

Running

import teneto as tnt

TN = tnt.TemporalNetwork()

tedges = [[1, 2, 100],
         [1, 2, 101],
         [1, 2, 102], 
         [1, 2, 103]]

TN.add_edge(tedges)

yields a TN.netshape of (3,4) and TN.N of 3 where in both cases the number of nodes should be 2 and not 3.

This is because the current line (322) in _calc_netshape in the TemporalNetwork class assumes integer node names and counts from 0:

n_nodes = self.network[['i', 'j']].max(axis=1).max()+1

which we could replace it with this for example

n_nodes = len(np.unique(TN.network[['i', 'j']].values))

AttributeError: module 'sqlalchemy.util' has no attribute 'dependencies' after importing TemporalNetworks()

Hi,

I am getting the below error when running the following:

!pip install teneto (on Google Cloud Platform JupyterLab in a Python 3 environment)
from teneto import TemporalNetwork
tnet = TemporalNetwork()

Any idea how to resolve this? Many thanks!

[BUG] TenetoBIDS - space-XXX and suffix should be removed from bids_filter if calling make_parcellation

after calling make_parcellation with teneto bids space is removed from filenames (because files are no longer in that space). But at present, bids_filter will keep the space.

Similarly: Suffix should be removed.

MemoryError while loading graph

If I try to load a relatively small network (~86980 edges) composed of three snapshots I get the following error:

MemoryError: Unable to allocate array with shape (110011, 110011, 3) and data type float64

I have the same behavior loading an edgelist as well as a pandas DataFrame.

Is it a known scalability issue or is there something that I am missing?
If so, what's the network size that teneto is able to handle?

Error in tutorial : inconsistent TemporalNetwork.network type?

Copy pasting the first few lines of the tutorial
https://teneto.readthedocs.io/en/latest/tutorial/networkrepresentation.html#temporalnetwork-object
yields an error at tnet.network.head() because tnet created with
tnet.generatenetwork('rand_binomial',size=(5,3), prob=0.5)
is a Numpy ndarray and not a Pandas DataFrame.

I haven't tried other ways of creating tnet, so I'm not sure if tnet.network is consistently a numpy array or not. If so, the tutorial should be updated, otherwise it should be made so that tnet.network is always the same type: either numpy array or pandas DataFrame.

change color for nodes in slice-plot

Using nodekwargs to change colors for ax.scatter with temporal community leads to double assignment of color.

for example:
ax = tnet.plot(plottype='slice_plot',ax=ax,cmap=plt.get_cmap("jet"),nodekwargs={'c':temp_community}) returns "scatter() got multiple values for keyword argument 'c'"
Here temp_community is node,time

[ENH] Prewhiten prior to derive_temporalnetwork

Considering preprocessing options like: this, prewhitening pror to deriving network connectivity for some methods may have advantages.

Instanteous phase into generate_temporalnetwork

I have seen this measure being used a fair bit in the literature, quite easy to add.

Label node on temporalnetwork ploting

import numpy as np
import teneto
import matplotlib.pyplot as plt
A1 = np.array ( [[0., 0., 0., 0., 0,0,0],[5., 0., 0., 0., 0,0,0],[1., 0., 0., 0., 0,0,0],
                 [0., 1., 2., 0., 0,0,0],[0., 0., 0., 1., 0,0,0], [0., 0., 0., 1., 0,0,0],
                 [0., 0., 0., 0., 1,1,0]] )


A2 = np.array ( [[0., 0., 0., 0., 0,0,0],[5., 0., 0., 0., 0,0,0], [1., 0., 0., 0., 0,0,0],
                 [0., 1., 2., 0., 0,0,0], [0., 0., 0., 1., 0,0,0],[0., 0., 1., 1., 0,0,0],
                 [0., 0., 0., 0., 1,1,0]] )

A3 = np.array ( [[0., 0., 0., 0., 0,0,0], [0., 0., 0., 0., 0,0,0],[0., 0., 0., 0., 0,0,0],
                 [0., 1., 2., 0., 0,0,0],[0., 0., 0., 1., 0,0,0],[0., 0., 0., 1., 0,0,0],
                 [0., 0., 0., 0., 1,1,0]] )
g_arr=np.array((A1, A2, A3, )).T

tnet = teneto.TemporalNetwork(from_array=g_arr)
ax = tnet.plot('slice_plot')
ax.scatter([0,0,0], [0,1,2], s=100, color='red', zorder=100)
plt.show()

May I know how to provide label on the plot. Currently, the node is assign as int. What if I have a label say

label=['a','b','c','d','e','f','g']

temporal_betweenness_centrality out of bounds error

When I run

import teneto as tnt
N = 4
n_t = 7

TN = tnt.TemporalNetwork()
TN.generatenetwork("rand_binomial", size=(N, n_t), prob=0.3)

tnt.networkmeasures.temporal_betweenness_centrality(TN, calc='pertime')

sometimes the code runs fine, but sometimes I get the following error (also when calc='overtime')

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-24-b3a8528aeb32> in <module>
      6 
      7 
----> 8 tnt.networkmeasures.temporal_betweenness_centrality(TN, calc='pertime')

~/.local/lib/python3.6/site-packages/teneto/networkmeasures/temporal_betweenness_centrality.py in temporal_betweenness_centrality(tnet, paths, calc)
     84     # if shortest paths are not calculated, calculate them
     85     if tnet is not None:
---> 86         paths = shortest_temporal_path(tnet)
     87 
     88     bet = np.zeros([paths[['from', 'to']].max().max() +

~/.local/lib/python3.6/site-packages/teneto/networkmeasures/shortest_temporal_path.py in shortest_temporal_path(tnet, steps_per_t, i, j, it, minimise)
    199                         # Only select i if directed, ij if undirected.
    200                         if tnet.nettype[1] == 'u':
--> 201                             network = tnet.get_network_when(ij=list(ij), t=t)
    202                         elif tnet.nettype[1] == 'd':
    203                             network = tnet.get_network_when(i=list(ij), t=t)

~/.local/lib/python3.6/site-packages/teneto/classes/network.py in get_network_when(self, **kwargs)
    477         """
    478         """
--> 479         return teneto.utils.get_network_when(self, **kwargs)
    480 
    481     def df_to_array(self):

~/.local/lib/python3.6/site-packages/teneto/utils/utils.py in get_network_when(tnet, i, j, t, ij, logic, copy, asarray, netshape, nettype)
    987             ind = np.hstack([ind, ind2])
    988 
--> 989         edges = network[ind[0], ind[1], ind[2]]
    990         ind = ind[:, edges != 0]
    991         edges = edges[edges != 0]

IndexError: index 7 is out of bounds for axis 2 with size 7

[ENH] add templateflow dependency and drop packaged parcellation data

Also include loading of tsv meta information that includes communities

[BUG] TemporalNetwork object slow

Been reported in emails that TemporalNetwork can be slow.

It was a little unclear in correspondence if this was a dense or sparse matrix. But the problem is probably when there is no need to make very dense matrices have a sparse representation.

Probably fix: add flag for TemporalNetwork to remain dense.

I can not install or update from v0.2.0 to v.0.23

Dear @wiheto :
I can not install or update from v0.2.0 to v.0.23, when I do it by "pip install teneto -U", it mentioned that some wrong in my computer. Thank you :)

BUG Test data not included with pip

Not confirmed, but reported to me that the dummybids data is not included in 0.4.6.

Complete HDF5 compatibility

Some networkmeasures will not run if TemporalNetwork is in the HDF5 format. These should be fixed.

Measures are:

temporal_degree_centrality, volatility, fluctuability, sid, topological overlap, and participation coefficient with negative edges)

Edit: After this is done, some benchmarking for speed should be done. Some of the upcoming trajectory stuff is snailspeed in HDF5

Improve shortest temporal paths (speed issues)

Collecting issues relating to this issue. To be fixed for Version 0.6.

Memory error with large network - centralities

Hi,

I am just starting with Teneto. Installed with pip on anaconda - windows.
I am trying to load the temporal network from here

I put a line "i,j,t" at the beginning of the file, then loaded it with pandas as dataframe and used:
teneto.TemporalNetwork(from_df=dataframe) but I receive a memory error:

emoryError Traceback (most recent call last)
in
----> 1 tnet2 = tnet.TemporalNetwork(from_edgelist=[list(d) for d in D.values])

C:\ProgramData\Anaconda3\lib\site-packages\teneto\classes\network.py in init(self, N, T, nettype, from_df, from_array, from_dict, from_edgelist, timetype, diagonal, timeunit, desc, starttime, nodelabels, timelabels, hdf5, hdf5path, forcesparse)
131 self.network_from_df(from_df)
132 if from_edgelist is not None:
--> 133 self.network_from_edgelist(from_edgelist)
134 elif from_array is not None:
135 self.network_from_array(from_array, forcesparse=forcesparse)

C:\ProgramData\Anaconda3\lib\site-packages\teneto\classes\network.py in network_from_edgelist(self, edgelist)
257 colnames = ['i', 'j', 't']
258 self.network = pd.DataFrame(edgelist, columns=colnames)
--> 259 self._update_network()
260
261 def network_from_dict(self, contact):

C:\ProgramData\Anaconda3\lib\site-packages\teneto\classes\network.py in _update_network(self)
220 """
221 self._calc_netshape()
--> 222 self._set_nettype()
223 if self.nettype:
224 if self.nettype[1] == 'u':

C:\ProgramData\Anaconda3\lib\site-packages\teneto\classes\network.py in _set_nettype(self)
172 self.nettype = 'xu'
173 G1 = teneto.utils.df_to_array(
--> 174 self.network, self.netshape, self.nettype)
175 self.nettype = 'xd'
176 G2 = teneto.utils.df_to_array(

C:\ProgramData\Anaconda3\lib\site-packages\teneto\utils\utils.py in df_to_array(df, netshape, nettype)
749 if len(df) > 0:
750 idx = np.array(list(map(list, df.values)))
--> 751 G = np.zeros([netshape[0], netshape[0], netshape[1]])
752 if idx.shape[1] == 3:
753 if nettype[-1] == 'u':

MemoryError:

Am I doing something wrong or maybe this representation cannot handle large networks?

Error TenetoBIDS numpy input does not save to json sidecar

Query about compatible data preprocessing pipeline

I am using data from ABIDE Preprocessed Initiative (C-PAC pipeline) as input to the derive function and using Fisher and Box-Cox transformation to normalize the derived graphlet.
However I am getting a warning saying that Box-Cox is returning edges with uniform values through time. Could this be due to the preprocessing strategy or the number of time points (~120 timepoints, 2 TR)?

Closeness centrality running without interruption

Hi,
I am using Teneto to calculate the closeness centrality. My code is running more than two days without stopping. what is the problem ?
Resources

• Processor(s): 32
• Threads par core: 2
• Core per socket : 8
• Model : Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
• Memory : 130 giga

Code:

import pandas as pd
import teneto as tn
import numpy as np
from teneto import TemporalNetwork

filename = 'data/Sociopatterns/primaryschool.csv'
df_PS = pd.read_csv(filename, names = ['t', 'i', 'j', 'Ci', 'Cj'], header=None, delimiter='\t')
df_PS_Day = df_PS[ df_PS['t'] < 63000] 
df_PS_final = df_PS_Day[[ 't', 'i', 'j']]

df_PS_final

	t	i	j
0	31220	1558	1567
1	31220	1560	1570
2	31220	1567	1574
3	31220	1632	1818
4	31220	1632	1866
...	...	...	...
60618	62300	1719	1859
60619	62300	1720	1843
60620	62300	1739	1872
60621	62300	1767	1787
60622	62300	1800	1820

60623 rows × 3 columns

# The time resolution between two signal is 20 seconds.
# This function normalize it. 
def contact_graph(data, cfg=None, resolution=0): 
    
    nodes = pd.concat([data['i'], data['j']]) 
    nodes = nodes.sort_values()
    nodes = nodes.unique()
    nb_nodes = len(nodes)
    
    ## changes the label of each node to be the index of np.array
    keys = nodes
    values = list(range(nb_nodes))
    nodeslabels = dict(zip(keys, values))
   
    if resolution == 0 :
        times = data['t']
        times = times.sort_values()
        times = times.unique()
        n_times = len(times)
    else:
        times = range(min(data['t']), max(data['t'])+resolution, resolution)
        n_times = len(times)
    
    ## changes the time counter 
    keys_times = times
    items_times = list(range(n_times))
    timesCounter  = dict(zip(keys_times,items_times ))          
    
    df = data.copy()
    frame =pd.DataFrame()
    frame['i'] = df['i'].map(nodeslabels)
    frame['j'] = df['j'].map(nodeslabels)
    frame['t'] = df['t'].map(timesCounter)
                  
    contacts = [tuple([frame.iloc[i,0], frame.iloc[i,1], frame.iloc[i,2]]) for i in range(len(frame))]
    contacts = np.array(contacts)
    contacts = contacts[contacts[:, 2].argsort()]
    
    return contacts

contacts_PS = contact_graph(df_PS_final, resolution=20)
contacts_PS[:50]

array([[ 58,  63,   0],
       [ 59,  64,   0],
       [ 63,  66,   0],
       [ 85, 185,   0],
       [ 85, 209,   0],
       [101, 114,   0],
       [186, 194,   0],
       [186, 209,   0],
       [186, 194,   1],
       [183, 189,   1],
       [141, 187,   1],
       [101, 114,   1],
       [ 85, 185,   1],
       [ 63,  66,   1],
       [ 58,  63,   1],
       [ 85, 209,   1],
       [101, 114,   2],
       [179, 191,   2],
       [179, 181,   2],
       [159, 168,   2],
       [141, 187,   2],
       [ 85, 209,   2],
       [ 58,  62,   2],
       [ 62,  66,   2],
       [ 62,  63,   2],
       [ 59,  64,   2],
       [ 58,  63,   2],
       [ 63,  66,   2],
       [ 85, 186,   3],
       [179, 181,   3],
       [150, 152,   3],
       [101, 114,   3],
       [ 85, 185,   3],
       [ 85, 209,   3],
       [ 62,  66,   3],
       [ 62,  63,   3],
       [ 58,  63,   3],
       [ 58,  62,   3],
       [ 36,  51,   3],
       [ 63,  66,   3],
       [183, 189,   4],
       [150, 153,   4],
       [101, 114,   4],
       [ 85, 209,   4],
       [ 63,  66,   4],
       [ 58,  63,   4],
       [ 17,  39,   4],
       [ 80,  87,   4],
       [162, 164,   5],
       [158, 169,   5]])

tnetHS = TemporalNetwork(from_edgelist=list(contacts_PS))

tnetHS.network.head()

	i	j	t
0	58	63	0
1	59	64	0
2	63	66	0
3	85	185	0
4	85	209	0

 closeness = tn.networkmeasures.shortest_temporal_path(tnetHS)

Failed installation via pip on Mac OS Sierra

I keep getting this error when trying to install the package on Python3.6 and Python2.7:

/usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -stdlib=libc++ -Iinclude -Iigraphcore/include -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c src/CPMVertexPartition.cpp -o build/temp.macosx-10.6-intel-3.6/src/CPMVertexPartition.o clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) error: command '/usr/bin/clang' failed with exit status 1

I think it's due to mistake in installation of louvain package. Any help is much appreciated.

Control colormap of temporal graph nodes

Is your feature request related to a problem? Please describe.
Would it be possible to control the color of the temporal graph nodes when using tnet.plot()? For example give an array [0, 10] then the nodes color will change from light to dark from left to light. Thanks.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Better pybids integration

There is some duplicate code in TenetoBIDS with pybids. Pybids will be better maintained and Teneto should migrate towards using it more.

[ENH] Permutations of temporal network

General

1. create teneto.stats module

NHSTing if network is not more "x" than random

1. allow for multiple ways to permute network (taken from Holme 2012).
2. allow for a networkmeasure(s) to be calculated on permuted network.
3. return permutation distribution.

NHSTing of network measure if groups of network-A are different to groups of network-B

1. shuffle groups regardless of measure.

[ENH] TenetoBIDS should only select relevant files

Network Properties calculation speed problem

Hello there, I am trying to use teneto to calculate several network features for my temporal network, such as closeness centrality, shortest path and etc. Other than degree centrality that only took a few seconds to finish, they all takes very very long. I am inputing a 3 temporal layered, 50 nodes network, but it has taken more than 24 hours now and the closeness centrality calculation has not finished yet. I am wondering if there is a way to improve the speed of calculation?

error in params to make_parcellation

When I run make_parcellation I get the following error which seems to be removed when I comment 'sidecar'.

~/anaconda3/lib/python3.6/site-packages/teneto/classes/bids.py in make_parcellation(self, parcellation, parc_type, parc_params, network, update_pipeline, removeconfounds, tag, njobs, clean_params)
859
860 for i,f in enumerate(files):
--> 861 self._run_make_parcellation(f,i,tag,parcellation,parc_name,parc_type,parc_params)
862
863 if update_pipeline == True:

~/anaconda3/lib/python3.6/site-packages/teneto/classes/bids.py in _run_make_parcellation(self, f, i, tag, parcellation, parc_name, parc_type, parc_params)
878 fsave, tag, 'parcellation', 'roi')
879 roi = teneto.utils.make_parcellation(
--> 880 f, parcellation, parc_type, parc_params)
881 # Make data node,time
882 roi = roi.transpose()

~/anaconda3/lib/python3.6/site-packages/teneto/utils/utils.py in make_parcellation(data_path, parcellation, parc_type, parc_params)
863 if parc_type == 'sphere':
864 parcellation = load_parcellation_coords(parcellation)
--> 865 seed = NiftiSpheresMasker(np.array(parcellation), **parc_params)
866 data = seed.fit_transform(data_path)
867 elif parc_type == 'region':

TypeError: init() got an unexpected keyword argument 'description'

can't run make_parcellation

tnet.make_parcellation(parcellation,removeconfounds=True,parc_type='region',parc_params=nilearn_params)

No (static) network community file found.
Process Process-12:
Traceback (most recent call last):
File "/disk2/home/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/disk2/home/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/disk2/home/anaconda3/lib/python3.6/concurrent/futures/process.py", line 169, in _process_worker
call_item = call_queue.get(block=True)
File "/disk2/home/anaconda3/lib/python3.6/multiprocessing/queues.py", line 113, in get
return _ForkingPickler.loads(res)
File "/disk2/home/anaconda3/lib/python3.6/site-packages/bids/layout/core.py", line 278, in getattr
if attr in self.entities:
File "/disk2/home/anaconda3/lib/python3.6/site-packages/bids/layout/core.py", line 278, in getattr
if attr in self.entities:
File "/disk2/home/anaconda3/lib/python3.6/site-packages/bids/layout/core.py", line 278, in getattr
if attr in self.entities:
[Previous line repeated 986 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object

BrokenProcessPool Traceback (most recent call last)
in ()
----> 1 tnet.make_parcellation(parcellation,removeconfounds=True,parc_type='region',parc_params=nilearn_params)
2

~/anaconda3/lib/python3.6/site-packages/teneto/classes/bids.py in make_parcellation(self, parcellation, parc_type, parc_params, network, update_pipeline, removeconfounds, tag, njobs, clean_params)
863 parc_name, parc_type, parc_params) for i, f in enumerate(files)}
864 for j in as_completed(job):
--> 865 j.result()
866
867 if update_pipeline == True:

~/anaconda3/lib/python3.6/concurrent/futures/_base.py in result(self, timeout)
423 raise CancelledError()
424 elif self._state == FINISHED:
--> 425 return self.__get_result()
426
427 self._condition.wait(timeout)

~/anaconda3/lib/python3.6/concurrent/futures/_base.py in __get_result(self)
382 def __get_result(self):
383 if self._exception:
--> 384 raise self._exception
385 else:
386 return self._result

BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

TenetoBIDS history export

History is recorded in TenetoBIDS.history. Two/Three features:

1. export history to json
2. run TenetoBIDS on json for reproduceibility
3. plot history (not all steps will necessarily be linked, so there may not be an easy solution for this in the way teneto is currently built.)

communityplot

Plotting tool to show communities through time.

Hi, some thing need update

dear @wiheto , I run the new teneto(0.2.4), I think some little bugs need update, the information of the version and the example of the teneto. :)

Error during derive TVC possible related to confounds report

Hey,
I got the following error:
I had selected confounds_report=True. The tvc files where generated.

.../lib/python3.6/site-packages/teneto/classes/bids.py in derive_temporalnetwork(self, params, update_pipeline, tag, njobs, confound_corr_report)
246 dfc_df_z = (dfc_df - dfc_df.mean())
247 df_z = (df - df.mean())
--> 248 R_df = dfc_df_z.T.dot(df_z).div(len(dfc_df)).div(
249 df_z.std(ddof=0)).div(dfc_df_z.std(ddof=0), axis=0)
250 description = R_df.describe().transpose().to_html()

.../lib/python3.6/site-packages/pandas/core/frame.py in dot(self, other)
1076 common = self.columns.union(other.index)
1077 if len(common) > len(self.columns) or len(common) > len(other.index):
-> 1078 raise ValueError("matrices are not aligned")
1079
1080 left = self.reindex(columns=common, copy=False)

[Question] shortest_temporal_path

Must the temporal network be fully connected for all times to get a value returned ?
I'm attempting to evaluate a random spatial temporal network of my own design ( which is not guaranteed to be a fully connected network, since edges are based on a radial distance and nodes are randomly placed) and the shortest_temproal_path method just hangs with no result nor an exception/error.

Possible Typo in test_communitydetection.py

IIUC, this should be 4 time points, instead of 3.

teneto/test/communitydetection/test_communitydetection.py

Lines 12 to 13 in e7bfc47

	# Make into 3 time points
	G = np.stack([a, a, b, b]).transpose([1, 2, 0])

G has the shape (6,6,4)

does df_to_array return array with wrong size?

Trying to extract array from tvc dataframe.
tnet.load_data('tsv')
df = tnet.tvc_data_
teneto.utils.df_to_array(df,[N,N,T],'wu')

This returns array with size N,N,N. What could be wrong?

Calculating consensus partition in teneto

In the paper by Braun 2015, the measure the consensus partition from the modular allegiance matrix, and finally quantify the integration (See Fig 2 of the paper).

Correct me if I'm wrong, but is it correct, using teneto, this can be achieved as below

np.random.seed(10)
# Two graphlets templates with definite community structure
a = np.array([[1, 1, 1, 0, 0, 0], [1, 1, 1, 0, 0, 0], [1, 1, 1, 0, 0, 0], [
                 0, 0, 0, 1, 1, 1], [0, 0, 0, 1, 1, 1], [0, 0, 0, 1, 1, 1]])
b = np.array([[1, 1, 1, 1, 0, 0], [1, 1, 1, 1, 0, 0], [1, 1, 1, 1, 0, 0], [
                 1, 1, 1, 1, 0, 0], [0, 0, 0, 0, 1, 1], [0, 0, 0, 0, 1, 1]])
# Make into 4 time points
G = np.stack([a, a, b, b]).transpose([1, 2, 0])

# returns node,time array of community assignments
community_temporal = teneto.communitydetection.temporal_louvain(G, intersliceweight=0.1, n_iter=1)

# Calculate the allegiance
alleg = allegiance(community_temporal)


tnet_static = teneto.TemporalNetwork(from_array=np.tril(alleg),nodelabels=ch_names)

# Get the community/consensus partition
communities_static= teneto.communitydetection.temporal_louvain(tnet_static, intersliceweight=0.1, n_iter=1)

# Get the integration
icoeff = integration( communities_static)

Inconsistency when plotting slice plot a temporal network

When ploting a temporal network using .plot, I notice neighboring interaction were not properly connected.

Below are the plot produced by teneto plot

The red line indicate where connection should exist. Specifically, at slice 2,3,4

This issue can be replicate with the code below.

For reproducibility, the connection array can be download via:
link to array

import teneto
import matplotlib.pyplot as plt
g_arr=np.load('g_arr.npy')
plt.style.use('mpl20')
tnet = teneto.TemporalNetwork(from_array=g_arr)
ax = tnet.plot('slice_plot')
ax.scatter([0,0,0], [0,1,2], s=150, color='red', zorder=50)
plt.show()

Plotting slice_plot will fail if dataframe columns in wrong order

If a TemporalNetwork object is generated from a DataFrame that complies to the format (i, j, t columns)

TemporalNetwork(from_df=t_df, nettype='bd')

there will be an IndexError when plotting a slice-plot:

~/.virtualenvs/ethereum-network-iTOQKzl-/lib/python3.8/site-packages/teneto/classes/network.py in plot(self, plottype, ij, t, ax, **plotparams)
    460             _, ax = plt.subplots(1)
    461         data_plot = teneto.utils.get_network_when(self, ij=ij, t=t)
--> 462         data_plot = teneto.utils.df_to_array(
    463             data_plot, self.netshape, self.nettype)
    464         ax = funs[plottype](data_plot, ax=ax, **plotparams)

~/.virtualenvs/ethereum-network-iTOQKzl-/lib/python3.8/site-packages/teneto/utils/utils.py in df_to_array(df, netshape, nettype)
    769                 idx = np.vstack([idx, idx[:, [1, 0, 2]]])
    770             idx = idx.astype(int)
--> 771             tnet[idx[:, 0], idx[:, 1], idx[:, 2]] = 1
    772         elif idx.shape[1] == 4:
    773             if nettype[-1] == 'u':

IndexError: index 9 is out of bounds for axis 2 with size 9

This happens specifically if the column order is t, i, j.
A workaround is changing the column order in the DataFrame:

df= df[['i', 'j', 't']]

[ENH] temporal consensus clustering by persitence maximization

Outlined in https://arxiv.org/pdf/1501.00040.pdf

Remove dependencies on neural-specific modules but issue a warning if a function is trying to be used that requires it

Is your feature request related to a problem? Please describe.
No, but it would make this tool easier for broad scale usage
Describe the solution you'd like
Currently, there are several neural-specific package dependencies that may not be necessary for all researchers using this tool for temporal networks:

          'nilearn>=0.6.0',
          'pybids>=0.9',
          'statsmodels>=0.8.0',
          'networkx>=2.0',
          'python-louvain>=0.13',
          'pandas>=0.21',
          'scipy>=1.4.1',
          'numpy>=1.16.1',
          'templateflow>=0.4.1'],

Would it be possible to not require templateflow, nilearn, and pybids for installation? For example, there could be a warning shown during installation that says for full functionality, please additionally install x, y, and z?
Describe alternatives you've considered
This could also be implemented as a decorator that you could use for neural-dependent functions?

import importlib
def assert_installed_packages(func, packages=["templateflow", "nilearn", "pybids"]):
    packages = np.asarray(sorted(set(packages)))
    def wrapper():
        installed = list()
        for package in packages:
            try: 
                globals()[package] = importlib.import_module(package)
                installed.append(True)
            except ImportError:
                installed.append(False)
        installed = np.asarray(installed)
        assert np.all(installed), "Please install the following packages to use this functionality:\n{}".format(", ".join(packages[~installed]))
    return wrapper

def function_not_dependent_on_neuralpackages(x):
    return x

@assert_installed_packages(packages=["hello"])
def function_dependent_on_neuralpackage(x):
    return x

 

If I tried to use function_dependent_on_neuralpackage but didn't have the required packages it would tell me to install templateflow, nilearn, and pybids.

The code above is fully functional but I could help out with that if you were interested.

Additional context
Add any other context or screenshots about the feature request here.

Better confounds in reports

At the moment there plots are reports are generated in relation to all confounds in the confound file with the time varying connectivity. These are however dull and unreadable.

This should be a forestplot of all the confounds. Or stacked histograms.