deonheyns / domain_crawl

Crawl an entire domain with Zillabyte

Easy Domain Crawling

This Zillabyte Component will crawl an entire web domain and return the results as a new dataset.

You can execute this component from the command line:

$ zillabyte execute "domain_crawl" "docs.zillabyte.com"

Or, you can stitch this component into your own Data App.

app = Zillabyte.app("your_app")
stream = app.source(...)
stream = stream.call_component("domain_crawl")
stream.sink(...)