scorelab / bellydynamic Goto Github PK
View Code? Open in Web Editor NEWBellyDynamic: A scalable data structure to handle online and offline dynamic graph objects
License: Apache License 2.0
BellyDynamic: A scalable data structure to handle online and offline dynamic graph objects
License: Apache License 2.0
Currently it only supports Linux, while SNAP library also supports 64-bit Windows so I think there is no problem to build simple cmd scripts to make Windows setup as easy as Linux one
When open the GUI file format of python, there is a error related to print function making it unable to work/run.
So there should be some kind of environment like JAVA_ENVIRONMENT for running bash files of Cassandra that could fix the bugs and run the files without the error of print function.
To be compatible with people who use Python 3 forward, print statements across many .py files (such as in Centrality.py and MultiGraph.py) need to have parentheses around what needs to be printed, since it is now a function. Otherwise, many files do not work.
getting started wiki
is very outdated-
sudo sh setup.sh
in the terminal but that command is not provided in the wiki
.
#!/bin/bash
wget https://snap.stanford.edu/snappy/release/snap-4.0.0-4.0-centos6.5-x64-py2.6.tar.gz;
tar zxvf snap-4.0.0-4.0-centos6.5-x64-py2.6.tar.gz;
cd snap-4.0.0-4.0-centos6.5-x64-py2.6;
sudo python setup.py install & cd ..;
cp snap-4.0.0-4.0-centos6.5-x64-py2.6/snap.py ./bellydynamic-adv/;
cp snap-4.0.0-4.0-centos6.5-x64-py2.6/_snap.so ./bellydynamic-adv/
chmod +x ./bellydynamic-adv/snap.py;
chmod +x ./bellydynamic-adv/_snap.so;
* New File-
#!/bin/bash
wget https://snap.stanford.edu/snappy/release/snap-4.1.0-4.1-centos6.5-x64-py2.6.tar.gz;
tar zxvf snap-4.1.0-4.1-centos6.5-x64-py2.6.tar.gz;
cd snap-4.1.0-4.1-centos6.5-x64-py2.6;
sudo python setup.py install & cd ..;
cp snap-4.1.0-4.1-centos6.5-x64-py2.6/snap.py ./bellydynamic-adv/;
cp snap-4.1.0-4.1-centos6.5-x64-py2.6/_snap.so ./bellydynamic-adv/
chmod +x ./bellydynamic-adv/snap.py;
chmod +x ./bellydynamic-adv/_snap.so;
python ./bellydynamic-adv/SchemaGraph.py
command results in a module error about MultiGraph
.
sudo sh setup.sh
should be given on the Getting Started page.In the current repo, we append the module libs using e.g. sys.path.append("../bellydynamic-util/"),
sys.path.append("../bellydynamic-metrics/"). This task is to improve this basic way of importing project libs other than appending project folders.
As End of Life of Python 2.7 is near and Python 2.6 is even dead it would be better to make it Python 3 compatible , in that case , snap library should be updated to py3.6 version.
Include an ISSUE SUBMITTING TEMPLATE
and Contributing guidelines.
Add shellcheck fixes to the shell files ( You can refer to -> https://www.shellcheck.net )
Also please update the wget
command as the current one fails.
Both of the scripts do the same thing in a different way, and both are broken.
Some of the Important Files that must be included in an Open Source Project are missing
Contributor Documentation
Issue Template
Pull Request Template
No Proper Readme File
No Build Testings Used
No Config Files
FIX: Update the bash command to wget https://snap.stanford.edu/snappy/release/snap-4.1.0-4.1-macosx10.13.2-x64-py2.7.tar.gz;
Users have an option of building a docker image from the dockerfile provided inside the project. But the build fails due to incorrect bash files [setup.sh, install-snap-python.sh]
Steps to reproduce:
STEP 1: Clone BellyDynamic
STEP 2: Navigate to project core
STEP 3: Build using Docker ('docker build ./')
Find this task on my website: http://itsrohan.com/gci/gcitasks
On the RandomGraph algorithm, GetRndNId does not return an id of a random node in graph each subsequent program executions.
The solution is : The random generator must be seeded first.
Reference :
GetRndNId()
Returns an ID of a random node in the graph. For this method to return different values on subsequent program executions, the random generator must be seeded first, for example with TRnd.Randomize().
MICRO PROBLEMS.....
1. Some grammar problem and not highlight the main point.
2. Not showing the requirement list to run the project.
3. Not recommending the best website to download the requirements.
4. Not showing the screen shoot of the working project.
MISSING FILES.........
1. No config files
2. No contributor documents
3. Pull Request Template
4. No Proper Readme File
5. No Build Testings Used
Traceback (most recent call last):
File "snap.py", line 21, in
_snap = swig_import_helper()
File "snap.py", line 20, in swig_import_helper
return importlib.import_module('_snap')
File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module
import(name)
It's basically impossible to use this framework in real life cases, because it's not documented well.
Since BellyDynamic is designed as a tool for developesr, it should have documentation for functions and classes that the developer should be using, what they do, what parameters to pass, what they return etc.
It is also best practice to include doc-strings, with short explanation of function/class and what inputs they take.
When I open the GUI file format of python, there is a error related to print function making it unable to work/run.
So there should be some kind of environment like JAVA_ENVIRONMENT for running bash files of Cassandra that could fix the bugs and run the files without the error of print function.But is it there or anything else that can help .
Firstly, When I run sudo bash install-snap-python.sh
for installing snap-python
--2018-11-27 22:49:10-- https://snap.stanford.edu/snappy/release/snap-4.0.0-4.0-centos6.5-x64-py2.6.tar.gz
Resolving snap.stanford.edu (snap.stanford.edu)... 171.64.75.80
Connecting to snap.stanford.edu (snap.stanford.edu)|171.64.75.80|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-11-27 22:49:11 ERROR 404: Not Found.
tar (child): snap-4.0.0-4.0-centos6.5-x64-py2.6.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
install-snap-python.sh: line 4: cd: snap-4.0.0-4.0-centos6.5-x64-py2.6: No such file or directory
python: can't open file 'setup.py': [Errno 2] No such file or directory
cp: cannot stat 'snap.py': No such file or directory
cp: cannot stat '_snap.so': No such file or directory
chmod: cannot access '../bellydynamic-travis/snap.py': No such file or directory
chmod: cannot access '../bellydynamic-travis/_snap.so': No such file or directory
Secondly, when I run sudo bash setup.sh
--2018-11-27 22:53:38-- https://snap.stanford.edu/snappy/release/snap-4.0.0-4.0-centos6.5-x64-py2.6.tar.gz
Resolving snap.stanford.edu (snap.stanford.edu)... 171.64.75.80
Connecting to snap.stanford.edu (snap.stanford.edu)|171.64.75.80|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-11-27 22:53:39 ERROR 404: Not Found.
tar (child): snap-4.0.0-4.0-centos6.5-x64-py2.6.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
setup.sh: line 5: cd: snap-4.0.0-4.0-centos6.5-x64-py2.6: No such file or directory
cp: cannot stat 'snap-4.0.0-4.0-centos6.5-x64-py2.6/snap.py': No such file or directory
cp: cannot stat 'snap-4.0.0-4.0-centos6.5-x64-py2.6/_snap.so': No such file or directory
chmod: cannot access './bellydynamic-adv/snap.py': No such file or directory
chmod: cannot access './bellydynamic-adv/_snap.so': No such file or directory
So, I view 2 file setup.sh
& install-snap-python.sh
and compare with version on http://snap.stanford.edu and I changed snap version 4.1.0 in 2 file.
And it worked successfully.
Files such as..
No config files
No contributor documents
And the README file does not contain sufficient information.
And there is error while opening the Bash python GUI file.
only compatible in python 2.7, cannot install snap in python 3.7
failed when running the 'python setup.py install'
SchemaGraph.py has errors, wgnuplot.exe isn't recognized
'wgnuplot.exe' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the path specified.
[S:\glib-core\gnuplot.cpp:892] Cannot find GnuPlot (wgnuplot.exe) for plot outDeg.sweden_labeled_5000.plt. Set the $$PATH variable or TGnuPlot::GnuPlotPath. (C:\gnuplot)
Degree distribution of sweden_labeled_5000 is in: outDeg.sweden_labeled_5000.png
'wgnuplot.exe' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the path specified.
[S:\glib-core\gnuplot.cpp:892] Cannot find GnuPlot (wgnuplot.exe) for plot diam.sweden_labeled_5000.plt. Set the $$PATH variable or TGnuPlot::GnuPlotPath. (C:\gnuplot)
Shortest path distribution of sweden_labeled_5000 is in: diam.sweden_labeled_5000.png
Fraction of nodes in largest connected component in sweden_labeled_5000: 0.023256
'wgnuplot.exe' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the path specified.
[S:\glib-core\gnuplot.cpp:892] Cannot find GnuPlot (wgnuplot.exe) for plot scc.sweden_labeled_5000.plt. Set the $$PATH variable or TGnuPlot::GnuPlotPath. (C:\gnuplot)
Component size distribution of sweden_labeled_5000 is in: scc.sweden_labeled_5000.png
The .gitignore file is not generated using gitignore.io and should therefore be remade.
I identified snap.py as a relevant technology that can be used to generate the file.
=>The library should be specifically explained what does it and what is it for...
=>The snap folder which we download should be a bit more organised
=>Should definitely improve the ui/ux of the website
=>Everything is too messed up in snap folder after we download it
=>Some components are missing in the algorithms like headerfiles are wrong,etc
The result on SwedenGraph and SchemaGraph both have the same result on the between the id and label.
This source code should represent the top 10 node with the highest number of Betweenness Centrality, but on the source code given, there are no representing value of Betweeness Centrality.
It happen because the function in namesDict.setdefault(param1,param2), set the value default to param2 if the dic[param1] is none. Therefor on the code, the namesDict is never been set before (always have none value). So self.namesDict.setdefault(str(key),str[key]) will always return the str[key] (which is the value of the id)
These are the solution i found :
Change :
for key in map: output += "nodeID: %d centrality: %f\n" % (key, map[key]) index += 1 if (index <= 10): label = self.namesDict.setdefault(str(key),str[key]) topNodes += "\tnodeID: %d label: %s\n" % (key, label) return output, topNodes
To :
for key in map: output += "nodeID: %d centrality: %f\n" % (key, map[key]) index += 1 if (index <= 10): label = self.namesDict.setdefault(str(key), map[key]) topNodes += "\tnodeID: %d label: %s\n" % (key, label) return output, topNodes
import MultiGraph as MG
ImportError: No module named MultiGraph
Error :solved i made the line as from networkx.classes.multigraph import MultiGraph as MG
setup.sh and install-snap-python.sh serve the same purpose, that is to set up snap python.
Both have the same and deprecated version of snap python which should be updated to version 4.1.0-4.1.
Following the wiki getting started will also lead to the same wrong version.
Check the latest version of snap.py required in the file system:
http://snap.stanford.edu/snappy/release/
wget https://snap.stanford.edu/snappy/release/snap-4.1.0-4.1-centos6.5-x64-py2.6.tar.gz
tar zxvf snap-4.1.0-4.1-centos6.5-x64-py2.6.tar.gz
cd snap-4.1.0-4.1-centos6.5-x64-py2.6
sudo python setup.py install & cd ..
cp snap-4.1.0-4.1-centos6.5-x64-py2.6/snap.py ./bellydynamic-adv/
cp snap-4.1.0-4.1-centos6.5-x64-py2.6/_snap.so ./bellydynamic-adv/
chmod +x ./bellydynamic-adv/snap.py
chmod +x ./bellydynamic-adv/_snap.so
Function snap.GetBetweennessCentr(self.G, nodesKeyCentrVals, edgesKeyCentrVals, 1.0, boolean) should have been filled with boolean value (True), by default its False. On the documentation, its stated that True means direct
ed graph, and False means undirected graph. In the source code, stated that the node has source and dest node (which means that its a directed graph).
So my solution for this case is:
On Centrality.py (function genGraphInfoBetweeness())
Change:
snap.GetBetweennessCentr(self.G, nodesKeyCentrVals, edgesKeyCentrVals, 1.0)
To:
snap.GetBetweennessCentr(self.G, nodesKeyCentrVals, edgesKeyCentrVals, 1.0, True)
When I want to run this command:
python ./bellydynamic-adv/SchemaGraph.py
I got this kind of error:
Traceback (most recent call last):
File "./bellydynamic-adv/SchemaGraph.py", line 8, in
import MultiGraph as MG
ImportError: No module named MultiGraph
When i run main.py i got this
Traceback (most recent call last):
File "main.py", line 2, in
context = snap.TTableContext()
NameError: name 'snap' is not defined
unnecessary variable:
Got this error when running
cd BellyDynamic/bellydynamic-adv
python SchemaGraph.py
'wgnuplot.exe' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the path specified.
[S:\glib-core\gnuplot.cpp:892] Cannot find GnuPlot (wgnuplot.exe) for plot outDeg.CollegeMsg.plt. Set the $$PATH variable or TGnuPlot::GnuPlotPath. (C:\gnuplot)Degree distribution of CollegeMsg is in: outDeg.CollegeMsg.png
'wgnuplot.exe' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the path specified.
[S:\glib-core\gnuplot.cpp:892] Cannot find GnuPlot (wgnuplot.exe) for plot diam.CollegeMsg.plt. Set the $$PATH variable or TGnuPlot::GnuPlotPath. (C:\gnuplot)
Shortest path distribution of CollegeMsg is in: diam.CollegeMsg.png
Fraction of nodes in largest connected component in CollegeMsg: 0.681411
'wgnuplot.exe' is not recognized as an internal or external command,
operable program or batch file.
The system cannot find the path specified.
[S:\glib-core\gnuplot.cpp:892] Cannot find GnuPlot (wgnuplot.exe) for plot scc.CollegeMsg.plt. Set the $$PATH variable or TGnuPlot::GnuPlotPath. (C:\gnuplot)
Possible Solution
Here
In Kronecker, at line 127 and 129 there should be typecasting of int to double as it divided the number by double and remainder should not be an integer.
Code :
int TKronMtx::GetKronIter(const int& Nodes) const {
return (int) ceil(log(double(Nodes)) / log(double(GetDim()))); // upper bound
//return (int) TMath::Round(log(double(Nodes)) / log(double(GetDim()))); // round to nearest power
}
At last store the whole answer in a variable and type cast to get a pure integer.
In the place of (int) there should be double as the ceil(double(Nodes)) is divided by log(double(GetDim())) and we don't want the remainder to be zero as it can cause problem to Node Id.
The first command 'wget https://snap.stanford.edu/snappy/release/snap-3.0.2-3.0-centos6.5-x64-py2.6.tar.gz' is not able to load the required snap files. Without being able to load this it is not possible to run the program.
This repos should follow Gitflow workflow and follow are the items you have to check.
- [ ] There is a `master` branch and a `develop` branch.
- [ ] `master` branch is locked for direct commits and,
- [ ] Require pull request reviews before merging is active
- [ ] Require status checks to pass before merging is active
- [ ] Require branches to be up to date before merging
- [ ] Appropriate Status checks are required
- [ ] Enforce all configured restrictions for administrators is active
- [ ] Restrict only maintainers group can push to the master branch
- [ ] `develop` branch is locked for direct commits and,
- [ ] Require pull request reviews before merging
- [ ] Dismiss stale pull request approvals when new commits are pushed is active
- [ ] Require pull request reviews before merging is active
- [ ] Require status checks to pass before merging is active
- [ ] Require branches to be up to date before merging
- [ ] Appropriate Status checks are required
- [ ] Enforce all configured restrictions for administrators is active
- [ ] Restrict only maintainers group can push to the master branch
snap library requires the user to use Python 2.7.x and 32-bit windows to work. So it's not supported anymore for Python 3 and higher user.
Possible fix:
On the Wiki 'Getting Started' page, the versions that are mentioned in for wget(snap) are unresponsive.
These are the versions on the Wiki page:
It returns this error when executed on a terminal:
It is because the versions are not available on the main source anymore. They are updated, therefore the older snap versions return an unresponsive error. When I went to https://snap.stanford.eu/snappy/release
, which is specified on the first image I found updated versions for all operating systems
These are the updated versions:
Therefore, if the version on the Wiki is changed from snap-3.0.2-3.0
to snap-4.1.0-4.1
it should perfectly!
Meanwhile you can manually input the command changing the versions and it should give a positive output. I did try it and it worked fine
I hope the Wiki page updates the versions soon.
This command used to download snap
wget https://snap.stanford.edu/snappy/release/snap-3.0.2-3.0-centos6.5-x64-py2.6.tar.gz
should be changed to
wget https://snap.stanford.edu/snappy/release/snap-4.0.0-4.0-centos6.5-x64-py2.6.tar.gz
because the current link doesn't exist anymore.
We need to make setup.sh
and Dockerfile improved with the comments from the reviewers.
here should be a CONTRIBUTORS.md page in the repository to guide the developers.So please add it.
=> The graph structural properties that have been implemented so far would capture information related to static graphs.
=>You need to implement a distributed data structure to handle fast dynamic graph updates.
=>Also you should implement an efficient index structure to query annotated graph information.
=>Need to design a framework that supports cross-libraries (e.g. snap, networkx, igraph, Boost)
As End of Life of Python 2.7 is near and Python 2.6 is even dead it would be better to make it Python 3 compatible , in that case , snap library should be updated to py3.6 version.
wget https://snap.stanford.edu/snappy/release/snap-4.0.0-4.0-centos6.5-x64-py2.6.tar.gz
to wget http://snap.stanford.edu/snappy/release/beta/snap-5.0.9-64-3.0-centos6.5-x64-py3.6.tar.gz
in install-snap-python.sh
and setup.sh
scripts filesNot entirely sure if this is a problem on my end or one of the program, but when trying to run "python ./bellydynamic-adv/SchemaGraph.py" as specified in the wiki the following error is returned: ImportError: DLL load failed: The specified module could not be found. It concerned the importlib module, of which I already have. I have a Windows OS, and therefore was only able to follow the Wiki instructions of "Getting Started" until the chmod commands - before that, I transposed the Linux commands to those of Windows.
As a consequence of this, perhaps in the future there should be set-up and use information for Windows users in the wiki, as well as specifying in the README that such information for any OS user can be accessed in the wiki, for the README so far is potentially confusing.
Coda: The BellyDynamic logo should also appear in the README :)
The Project should have Unit Tests to validate that each unit of the software performs as designed.
There is no Unit Tests Available for bellyDynamic-adv/metrics/util codes.
All the commands/Documentation that are required To Write Unit Tests for Python are present Here
install-snap-python.sh
and setup.sh
scripts fails, because snap version 4.0.0-4.0
is no longer available , use version 4.1.0-4.1
or 5.1.0-64-3.0
instead
Change wget https://snap.stanford.edu/snappy/release/snap-4.0.0-4.0-centos6.5-x64-py2.6.tar.gz
to wget http://snap.stanford.edu/snappy/release/snap-4.1.0-4.1-centos6.5-x64-py2.6.tar.gz
or wget http://snap.stanford.edu/snappy/release/beta/snap-5.1.0-64-3.0-centos6.5-x64-py2.6.tar.gz
in install-snap-python.sh
and setup.sh
scripts files
Note : Updating to Python 3 is necesarry because Life of Python 2.6 has already ended
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.