denji / hibp-578m

Hash cracking wordlist, rules, masks, and tokens taken from cracking over half a billion passwords.

HIBPv7 Hash Cracking Resources

I needed a large sample of actual password hashes for analysis to improve password security. The HaveIBeenPwned repository was a great collection of password hashes from breaches and covered a diversity of environments. The hashes were made public and over time I cracked them for analysis:

• HIBPv7: 578,006,177/613,584,246 (94.20%) passwords

Resources in this repository use this sample as a base of real-world passwords and are utilized in a few different applications.

While looking at the HIBP dataset, I realized that many passwords were low quality and would not meet AD password complexity requirements. I wanted to clean up the input data because the better the input set, the better the output. I created PwdStat to filter down password lists and provide meaningful analysis that could be used to develop other resources.

Getting Started:

• Wordlists
• Masks and Tokens
• Rules
• Summary

Wordlists

HIBPv7_7M.txt: Contains the top 7 million most popular passwords from HIBPv7 in no particular order. From the top 7 million most popular passwords, I cracked 99.65% of them, and this is the resulting list.

HIBPv7_100M-min-reqs.txt: From the top 100 million most popular passwords in HIBPv7, I cracked 99.18% of them, and this is the result after filtering for minimum complexity requirements. The wordlist contains around 8.5 million passwords.

HIBPv7_Top15_Masks-min-reqs.txt: Wordlist from the filtered down HIBP data (~70m passwords) and the set's top 15 most popular password masks. The wordlist contains around 30 million passwords and would meet the minimum complexity requirements of an AD domain.

Masks and Tokens

Password Masks: These password masks are in Hashcat format taken from the filtered down HIBP dataset and sorted by most popular.

Common tokens: Using PwdStat, a list of password tokens is generated using the NTLK library. The passwords were passed to a parser that attempted to filter down the passwords to their base tokens and then sorted by most common. This list results from parsing tokens from the filtered down HIBP set.

Rules

The repository contains rules generated from the unfiltered and filtered HIBP set using unique methods. They are separated into smaller lists to account for different hashing algorithms and sorted by most effective (most cracks) to least effective.

Squid Rules: This ruleset was created using rounds of randomized hashes, wordlists, and rule order to sort Hashcat rules by effectiveness. The set was "trained" on the entire collection of HIBP passwords and only included rules found in public hash cracking rule sets. The set is sorted by most effective to least effective.

• A methodology writeup can be found here for Squid Rules.

A comparison of squid rules to other rules can be found in COMPARE.md. Credit and thank you to penguinkeeper and others for creating the document used.

---

Summary:

• Wordlists:

  • HIBPv7_7m.txt
    • Top 7M passwords (99.65%) from HIBPv7
  • HIBPv7_100M_min-reqs.txt
    • 8.5m passwords from HIBPv7 that would meet min AD complexity requirements
  • HIBPv7_Top15_Masks-min-reqs.txt
    • 30m passwords from HIBPv7 based on the top 15 masks that would meet min AD complexity requirements

• Masks and Tokens

  • password_masks_min-reqs
    • Most popular password masks from all HIBP data filtered for minimum complexity requirements.
  • password_masks_no-reqs
    • Most popular password masks from all HIBP data filtered for passwords greater than or equal to 8 characters.
  • common_tokens_min-reqs
    • Most popular password words/tokens from all HIBP data filtered for minimum complexity requirements.
  • Text files are just the words and csv files contain additional metadata

• Rules

  • Squid Rule
    • Hashcat rules sorted from most effective to least effective from public hashcracking sets
    • Same rules broken into multiple sizes for specific applications