flashxio / flashx Goto Github PK

fg <- fg.load.graph("wiki-Vote.adj-v4", index="wiki-Vote.index-v4", graph.name="wiki")
wiki-Vote.adj-v4 doesn't exist
test.directed(fg, ig)
[1] "test directed degree"
Wrong state! Can't get a graph

*** caught segfault ***
address (nil), cause 'memory not mapped'

we should use some lightweight (linear or near linear) graph clustering algorithm to increase data locality when converting a graph to the FlashGraph format.

Instead of materializing the subgraph, we can use the original graph and some extra information to define the subgraph, so we don't need to physically copy the data of the subgraph from the original graph and create a new graph to store the data.

also kmeans++ and kmeans||

vertex_index_reader.h:95:4: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]

MAX_MULTI_BUFS shouldn't be hard coded.

maybe change the function coreness to kcore.

fg <- fg.load.graph("wiki-Vote.adj-v4", index="wiki-Vote.index-v4", graph.name="wiki")
stat: No such file or directory
R: /home/zhengda/parallel-cache/user-space-cache/flash-graph/in_mem_storage.cpp:292: static in_mem_graph::ptr in_mem_graph::load_graph(const string&): Assertion `size > 0' failed.
Aborted (core dumped)

When converting a graph from edge list format to FlashGraph format, users should be able to specify whether or not to use disks to store intermediate results.

flash-graph/test-algs/test_algs flash-graph/conf/run_graph.txt data/wiki.adj-v4 data/wiki.index-v4 wcc

test_algs: ../graph_index.h:174: void fg::graph_local_partition<vertex_type, part_vertex_type>::init() [with vertex_type = {anonymous}::wcc_vertex, part_vertex_type = fg::empty_part_compute_vertex]: Assertion vertex_arr' failed. test_algs: ../graph_index.h:174: void fg::graph_local_partition<vertex_type, part_vertex_type>::init() [with vertex_type = {anonymous}::wcc_vertex, part_vertex_type = fg::empty_part_compute_vertex]: Assertionvertex_arr' failed.

some of the libraries required by FlashGraph aren't critical. Users need to give the option to disable them when compiling FlashGraph

SAFS

• implement pthread lock
• implement pthread barrier.
• avoid using -fopenmp
• avoid unnecessary boost functions.
• don't get thread id and bind threads to CPU cores. They are Linux-specific methods.
• don't bind I/O threads to CPU cores.
• disable direct I/O this might be problematic in the future
• fix type mismatch (size_t and off_t).
• avoid using Linux specific headers and functions: malloc.h, memalign, allocating huge pages, posix_fadvise, etc

Multi-Threaded Modularity Based Graph Clustering using the Multilevel Paradigm

SAFS should be elastic, so that when we increase the number of SSDs, we don't need to reformat SAFS and reload data to it.