工具说明:百度URL采集工具,可自行修改线程;未加入去重复;使用后在同目录下生成home_url.txt
和original_url.txt
两个文件,前一个文件是域名列表,后一个是采集到的url列表.
使用例子:
python BaiDuUrlSpider.py inurl:index.action
Tool Description: Baidu URL collection tool, you can modify the thread; did not join to repeat; after use in the same directory to generate home_url.txt
and original_url.txt
two files, the former is a domain name list, the latter is The collected url list.
Use example:
python BaiDuUrlSpider.py inurl:index.action