drbenway / siteresearch Goto Github PK
View Code? Open in Web Editor NEWphp scripts to analyse webdata. Currently it includes a configurable crawler with various export options
Home Page: http://www.westworld.be/siteResearch/API/
License: Other
php scripts to analyse webdata. Currently it includes a configurable crawler with various export options
Home Page: http://www.westworld.be/siteResearch/API/
License: Other
export the crawl result to sitemap.xml file
provide the option to import an xml sitemap file into the crawler tables
Allow the export of the crawler table for later use.
create a class to support httpwatch based on http://www.phpied.com/automating-httpwatch-with-php/
windows only
Allow for a random amount of time between two page fetches
update wiki or project description to point to the api docs and designs
Option to define the maximum crawle depth
plugin to export crawler data to xgmml support (https://gephi.org/)
Add support for Gephi export
option to export the crawler table to csv file
add phpdoc comments to setup script
do tests with different php and mysql versions and settings do define a set of minimum requirements
Allow the export of the crawler table for later use.
phpdoc on crawler.php contains lots of errors.
do implementation tests on mamp, wamp & xampp
add the option to run multiple scripts at the same time.
option to export the crawler table to an excel or open office document.
write unit tests for crawler.php
regard settings in the robot.txt file when crawling urls
When setting up the crawler, provide an option to populate the url table with all known urls from google.
(google search "site:www.yourdomain.com")
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.