CaiJiEngine是一款基于Python3的URL采集工具,支持使用百度,必应,谷歌搜索引擎进行URL采集。
可打印格式:URL、域名、协议+域名。
结合Google Hacking语法食用更佳。:)
usage: CaiJi.py [-h] [-E ENGINE] -K KEYWORDS [-P {url,domain,domainpro}] [-C COUNTS] [-d] [-O OUTPUT] [-V]
optional arguments:
-h, --help show this help message and exit
-E ENGINE, --engine ENGINE
Use the following search engines to collect: [Baidu], [Bing], [Google] (default:Baidu)
-K KEYWORDS, --keywords KEYWORDS
Enter the keywords you want to query
-P {url,domain,domainpro}, --print {url,domain,domainpro}
Print as follows: [url], [domain], [domainpro] (default:domainpro)
-C COUNTS, --counts COUNTS
Number of pages found (default:5)
-d, --detail View crawling details
-O OUTPUT, --output OUTPUT
Output file
-V, --version show program's version number and exit
需要安装Python3.6以上环境
git clone git://github.com/UlyssesTakusen/CaiJiEngine
cd CaiJiEngine
pip3 install -r requirements.txt
python3 CaiJi.py -h
python3 CaiJi.py -K intitle:后台登录 -P url -d
python3 CaiJi.py -K site:baidu.com -E Bing,Baidu -O baidu.csv