Hitchhiker's Guide to URL Analysis

Intro

URL Analysis is a collection of techniques to determine whether a given URL is malicious or not. Generally we encounter two types of URLs i.e, Short URLs & Long URLs. In this repo we will see some tools & techniques for analyzing these URLs and figure out whether they are safe or not.

Short URLs

Short URLs are often used in malicious campaigns. The issue with them is that we can't simply determine where they are pointing unless we open them in some manner. Here are some tricks for converting them to original form without direct interaction.

ZSCALER ZULU
UNSHORTEN.IT
URLSCAN.IO
EXPANDURL.NET
WHEREGOES.COM
CHECKSHORTURL.COM
Some services like bit.ly, goo.gl, tiny.url can be expanded by appending "+". For e.g, bit.ly/abcdefgh+
You can also use UrlXpand, a CLI tool I cooked after seeing Rob VandenBrink's Unshorten script.

Long URLs

Now that we have a long URL, we can perform some analysis on it as follows:-

Check the WHOIS records on Databases like WHOIS, WHO.IS, or ICANN LOOKUP.
Check the DNS records using tools like DNSDUMPSTER, or MXTOOLBOX.
Check the site reputation & related data using Threat Intelligence platforms like VIRUSTOTAL, CISCO TALOS, PULSEDIVE, OPSWAT METADEFENDER, or CLOUDFLARE RADAR.
Open the URL in browser sandboxes like BROWSERLING and WANNABROWSER, or SQRX with custom user agents.
Some URLs maybe in format https://example.com/@not-malicious.com. Here first part of URL is just for show, once a user clicks this link not-malicious.com will load instead of example.com.

wand3rlust / hitchhikers-guide-to-url-analysis Goto Github PK