eranzim / pt-tools Goto Github PK

Generate a file containing a (unique) IP:Location list, for all IPs found (country is enough, based on IP ranges).

Currently, DllMain is called (with DLL_PROCESS_ATTACH), but that's it, and DllMain is limited in the logic it can do. SDI should be able to run an EP of choice after the DLL loads.

In addition to all the other data that is collected - also bring robots.txt (if exists). Maybe also CrossDomain.xml.

Possibly need to create a C version, or maybe we can take some of the implementation of socket.gethostbyname instead of calling it (e.g. we don't need to get the IP from the reply, just know that there is one).

find_comments.py finds html comments with beautifulsoup, it should be extended to look for other comments, such as JS (and possibly others such as CSS) comments.

Since robots.txt often includes interesting pages, we should make sure the extractor script gets them recursively (they could just be downloaded with another wget -r, and that would assure they'd be there if they exist). At this point, it might be worth moving the whole "get entire website" logic to a separate script file, or at least another function.