arielcostas / domain-scanner

Basic domain scanner in .NET, and using Azure CosmosDB

Domain scanner

A small tool that scans a domain's main A/AAAA records, TXT records and nameservers where the domain is hosted.

This tool was built to apply my knowledge of Microsoft Azure, using CosmosDB for the database.

Configuration

The configuration is done through the usual appsettings.<environment>.json files, or through environment variables or user secrets.

{
  "Cosmos": {
    "Endpoint": "https://<cosmosdb-account>.documents.azure.com:443/",
    "Key": "<cosmosdb-key>",
    "Database": "<cosmosdb-database>"
  }
}

Detailed documentation

The detailed documentation can be found in the GitHub Wiki.

License

This project is licensed under the MIT Licence - see the LICENCE file for details.

domain-scanner's Issues

Obtain and process MX headers

Implement parsing of TXT records to obtain providers

Parse the TXT records from the base domain, since they could contain interesting data:

• v=spf1 contains the allowed email senders, including those that may not receive email back (such as SendGrid, Mailgun or Postmark) as well as some other IP addresses

Domain verifications

• google-site-verification for Google
• 1password-site-verification for 1Password
• adobe-idp-site-verification for Adobe
• facebook-domain-verification for Facebook
• cisco-ci-domain-verification for Cisco's something (probably Webex)
• atlassian-domain-verification for Atlassian
• canva-site-verification or _canva-domain-verify for Canva
• autodesk-domain-verification for Autodesk
• box-domain-verification for Box.com

Implement deployment with IaC

Implement associating nameserver domains to providers

For example, zones containing X are hosted on Y:

• *.ns.cloudflare.com. is hosted on Cloudflare
• ns*.dondominio.com is hosted on DonDominio
• *.azure-dns.(com|net|org|info|...). is hosted on Azure DNS
• ns*.awsdns-*.(org|co.uk|com|net) are on AWS Route 53 -- This one is trickier since not every single awsdns-[something] is registered by Amazon.
• ns-*.1and1-dns.(com|org|biz|es|...) is hosted on IONOS (1&1)
• ns-cloud-*.googledomains.com is Google Cloud

Consider replacing primary/partition key generation

Consider using a SHA-256 hash (or some other) of the domain, generation date and some other factor for the primary key generation. And generate+store a partition key that is a substring of that hash (first N characters, for example).

The current domain partition key approach is problematic, since reports could be generated for certain domains WAY more than for others.

Continue building wiki