vigna / webgraph

WebGraph is a framework for graph compression.

Welcome to WebGraph!

WebGraph is a framework for graph compression aimed at studying web graphs. It provides simple ways to manage very large graphs, exploiting modern compression techniques.

This version of WebGraph is limited to graphs with at most 2^31 nodes. For larger graphs, have a look at the big version.

Building

You need Ant and Ivy.

Then, run ant ivy-setupjars jar and set your CLASSPATH by source setcp.sh

seba (mailto:[email protected])

webgraph's Issues

Resolving dependencies

Hi,

Can you please let me know the installation instructions? Simply running ant ivy-setupjars jar does not suffice as ultimately running a command such as java it.unimi.dsi.webgraph.BVGraph subsequently requires dsiutils and fastutil. Would I need to install these dependencies one by one by hand? Thanks!

Disclaimer: I am a newbie in the java world.

Can't not find or load main class

Hi,

I have been able to build the project with the ant command given ant ivy-setupjars jar which gives me a BUILD SUCCESSFUL. However when I run the command java it.unimi.dsi.webgraph.BVGraph.java -o -O -L it tells me

Error: Could not find or load main class it.unimi.dsi.webgraph.BVGraph.java
Caused by: java.lang.ClassNotFoundException: it.unimi.dsi.webgraph.BVGraph.java

I am in Ubuntu 18.04.
Is there something more I should be doing then the ant setup to get this to work?

what's the content(format) in .properties file?

Hi @vigna ,
thanks for your contribution to this open-source project

I try to load a .txt graph dataset in ASC2 format by the function ImmutableGraph.load(basename, pl); from local disk. When I run it, there is an error of The property file for {basename} does not contain a graphclass property. when I refer to the source code of "load()" function, the extension of "basename" file is automatically added with ".properties" as a java configuration file.

what should I write in the ".properties" file and how to contain graphclass properties? thanks

IncrementalImmutableSequentialGraph got blocked

In https://github.com/vigna/webgraph/blob/master/src/it/unimi/dsi/webgraph/IncrementalImmutableSequentialGraph.java, the successorQueue has a default size 100.

So if we call add for more than 100 times, this queue will be blocked and nothing can proceed.

`IllegalArgumentException: Too large value` on changing the window size

I would like to ask whether I have to exchange some parameter when modifying the window size for a recompression of a WebGraph.
When using an invocation like

java -Xmx100G it.unimi.dsi.webgraph.BVGraph -w 64 /scratch/data/webgraph/uk-2014-hc-t /scratch/data/webgraph/uk-2014-hc-t

I receive the following error message:

09:58:45.405 [main] INFO it.unimi.dsi.webgraph.BVGraph - Loading offsets...
Exception in thread "main" java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
    at it.unimi.dsi.webgraph.ImmutableGraph.load(ImmutableGraph.java:675)
    at it.unimi.dsi.webgraph.ImmutableGraph.loadMapped(ImmutableGraph.java:550)
    at it.unimi.dsi.webgraph.BVGraph.main(BVGraph.java:2390)
Caused by: java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at it.unimi.dsi.webgraph.ImmutableGraph.load(ImmutableGraph.java:671)
    ... 2 more
Caused by: java.lang.IllegalArgumentException: Too large value: 38965632897 > 38965632889
    at it.unimi.dsi.sux4j.util.EliasFanoMonotoneLongBigList.<init>(EliasFanoMonotoneLongBigList.java:306)
    at it.unimi.dsi.sux4j.util.EliasFanoMonotoneLongBigList.<init>(EliasFanoMonotoneLongBigList.java:274)
    at it.unimi.dsi.webgraph.BVGraph.loadInternal(BVGraph.java:1506)
    at it.unimi.dsi.webgraph.BVGraph.load(BVGraph.java:1302)
    at it.unimi.dsi.webgraph.BVGraph.loadMapped(BVGraph.java:1350)
    ... 7 more

This invocation works for smaller graphs, but neither for uk-2014-hc-t nor gsh-2015-hc-t.

I am using webgraph version 3.6.10 with OpenJDK 11.0.9.1.

Could you please write more information in ReadMe.md about how to compile&build this project and use it to generate dataset?