I am β¨Mr0Widoβ¨, and my primary goal will be to share information on every subject I am interested in and want to improve myself. These topics are many, but my primary topics are Penetration Tests, Web Security, Ethical Hacking, and their associated CTFs. I'm an enthusiastic CTF player. I will learn forever. π¬ Ask me about anything, and you can follow my social mediaπ.
mr0wido / commoncrawl.py Goto Github PK
View Code? Open in Web Editor NEWThis Python script is a multi-threaded tool for retrieving data from the CommonCrawl index. It allows you to specify a domain or a list of domains, and it will retrieve all URLs associated with those domains that are indexed by CommonCrawl.