607011 / pwned Goto Github PK

Consider files containing user IDs or mail addresses separated from the password by different symbols (':', ';', '\t', ' '). Also consider files where the password is contained in hex encoding ($HEX[...]).

6: Test command: ~/Workspace/pwned/build/Release/pwned-converter-cli/test/test_userpasswordreader_executable
6: Test timeout computed to be: 10000000
6: Running 1 test case...
6: ~/Workspace/pwned/pwned-converter-cli/test/test_userpasswordreader.cpp:74: error: in "test_userpasswordreader/test_userpasswordreader_password_only": check correctHash == gotHash has failed [3ffca08260070f894dd47244c034c216 != faeac4e1eef307c2ab7b0a3821e6c667]
6: ~/Workspace/pwned/pwned-converter-cli/test/test_userpasswordreader.cpp:74: error: in "test_userpasswordreader/test_userpasswordreader_password_only": check correctHash == gotHash has failed [99974f541e529d1425304e27772be176 != 751d31dd6b56b26b29dac2c0e1839e34]
6: ~/Workspace/pwned/pwned-converter-cli/test/test_userpasswordreader.cpp:74: error: in "test_userpasswordreader/test_userpasswordreader_password_only": check correctHash == gotHash has failed [f0eb63005912fce8c66288a633229d9f != f041e892017d152e7467b6cfde6bf6f1]
6: ~/Workspace/pwned/pwned-converter-cli/test/test_userpasswordreader.cpp:74: error: in "test_userpasswordreader/test_userpasswordreader_password_only": check correctHash == gotHash has failed [78fa5fe2f5f6682e415e1a1ec607ba4c != e45ee7ce7e88149af8dd32b27f9512ce]
6: ~/Workspace/pwned/pwned-converter-cli/test/test_userpasswordreader.cpp:74: error: in "test_userpasswordreader/test_userpasswordreader_password_only": check correctHash == gotHash has failed [712f79adead18f54eac40a84887d82f5 != 418c3ce12ee0101abecd427e313f6b82]
6: 
6: *** 5 failures are detected in the test module "test userpasswordreader"

This strangely happens on macOS Mojave (10.14.6), but not on Linux (Ubuntu 19.04, CentOS 7).

the project sounds like abusing the cryptographic hash function md5 for some kind of hash table (https://en.wikipedia.org/wiki/Hash_table). shouldn't it be faster to use a real hash table with O(1) opposed to binary search with O(log n)?

and if you would stick with md5, it should be possible to to skip the first steps of the binary search and guess that e.g. a hash starting with ffff... is near the end. since md5 hashes should by design be distributed evenly in the number space, that should be possible.

If the destination file already exists, it will be overwritten without notice. Not good!

Is there a chance to add static linking to have a "binary release"?
Maybe I could run the "armhf"-binary on router or NAS via pen drive without installing gcc and libs.

Is there a chance to run it with Windows Subsystem for Linux (WSL)?

Background:
I wanted to make a RPi2 server to test my passwords... (Raspbian Buster, GCC7.1) It isn't important to me, if it needs some some ms oder a second to check.

Problems & Solutions:
I had to install some additional boost librarys...
prepend NO_POPCNT=1 did not work, I guess it must be -DNO_POPCNT=1 - I fixed it by removing the condition in the CMAKE lists file.
Afterwards make didn't know the "-mno-popcnt" command -> I deleted it and it compiled via make (without j 5) in ~10 minutes. The j 5 option needed that much RAM and CPU that the whole system didn't respond. Without it needed ~700MB RAM and compiled.

Better for unit testing …

Write a test to cover smart searches in small files, e.g. in pwned-lib/test/testset-10000-existent-collection1+2+3+4+5.md5.

Proposal to extend pwned-lib/test/test_inspector.cpp:

BOOST_AUTO_TEST_CASE(test_existent_smart)
{
  const std::string inputFilename = "../../../../pwned-lib/test/testset-10000-existent-collection1+2+3+4+5.md5";
  const uint64_t size = boost::filesystem::file_size(inputFilename);
  const uint64_t hashCount = size / pwned::PHC::size;
  std::vector<pwned::PHC> phcs;
  phcs.reserve(hashCount);
  std::ifstream testset(inputFilename, std::ios::binary);
  pwned::PasswordInspector inspector(inputFilename);
  pwned::PHC phc;
  uint64_t nFound = 0;
  while (phc.read(testset))
  {
    if (inspector.smart_binsearch(phc.hash).count > 0)
    {
      ++nFound;
    }
  }
  BOOST_TEST(nFound == hashCount);
}

BOOST_AUTO_TEST_CASE(test_nonexistent_smart)
{
  const std::string inputFilename = "../../../../pwned-lib/test/testset-10000-existent-collection1+2+3+4+5.md5";
  const std::string nonExistentInputFilename = "../../../../pwned-lib/test/testset-10000-nonexistent-collection1+2+3+4+5.md5";
  const uint64_t size = boost::filesystem::file_size(nonExistentInputFilename);
  const uint64_t hashCount = size / pwned::PHC::size;
  std::vector<pwned::PHC> phcs;
  phcs.reserve(hashCount);
  std::ifstream testset(nonExistentInputFilename, std::ios::binary);
  pwned::PasswordInspector inspector(inputFilename);
  pwned::PHC phc;
  uint64_t nNotFound = 0;
  while (phc.read(testset))
  {
    if (inspector.smart_binsearch(phc.hash).count == 0)
    {
      ++nNotFound;
    }
  }
  BOOST_TEST(nNotFound == hashCount);
}