kdbanman / visualcrawl

Visualizing the behaviour of semantic web crawl algorithms.

Semantic Web Crawling Visualization

Kirby Banman, [email protected]

Important Bits

Examples and visualizations of semantic web crawls using:

rdfGraph version:

• Graph Visualizer: Gephi 0.8.1 beta 201202141941 using plugin SemanticWebImport
• Semantic Crawler: LDSpider 1.1e revision 304 checked out for extension to .gexf output using the gexf4j 0.4.0 library
• Triplestore Instance: 4Store 1.1.3

trafficGraph version:

• Graph Visualizer: Gephi 0.8.1 beta 201202141941 using plugin HttpGraph 1.0.6
• Semantic Cawler: LDSpider 1.1e

Overview

There are many unanswered questions regarding the nature of semantic web crawling (see SemCrawl.md in docs directory), and this is my attempt to sharpen those questions with some visual depictions of different crawling strategies.

*The initial motivation for this project was to produce this video and this other video. There are now visualizations of the RDF graphs themselves as .gexf files in each of the folders in the rdfGraph/example_crawls directory, see the readme in version 0.2 to learn how to use them.

License

All software used in this project that is not my own is licensed as open source.

All content authored by me licensed under the terms of the GNU General Public License as published by the Free Software Foundation, either 3 of the License, or (at your option) any later gc.

This content is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License in the docs directory. If not, see http://www.gnu.org/licenses.