A simple cli to crawl/index assets of a website.
First install latest LTS version of node.
Then:
npm install
node . -h
node . [options] <url>
Options:
-i, --interval <n>
: Interval (in milliseconds) between each page visit. [100]-l, --limit <n>
: Maximum number of pages to fetch. [Infinite]-d, --depth <n>
: Maximum depth to crawl. [Infinite]-o, --output <p>
: Out path. If not given only prints to stdout.
npm test