Trying to replicate the save file format serialization/deserialization. about monke

[mriswithe]

https://github.com/mriswithe/pybtd6/blob/main/btd6_save_explorer/save_file.py

is my main Save class definition. Not much else exists in the repo atm

[codeshaunted]

You seem to be on the right track. Are you utilizing PBKDF2 to generate the decryuption key and IV? The real killer here would be differences in encryption implementation, I'm lucky that it worked so well with crypto++.

[mriswithe]

I thought I was but it seems like I am doing something wrong. The main issue I am having is even with debuggers running simultaneously in CLion and PyCharm the raw data and the like look entirely different and I am unsure why.

This one is from CLion showing the dummy_header

This one is from PyCharm showing the same dummy_header

I usually would try and compare directly, but I am confused why they show the same data so differently.

[mriswithe]

I am using the std library hashlib in Python: https://docs.python.org/3/library/hashlib.html#hashlib.pbkdf2_hmac

Function signature:

 hashlib.pbkdf2_hmac(hash_name, password, salt, iterations, dklen=None)

I am passing in:

{'dklen': 32, # Derived Key Length
 'hash_name': 'SHA1',
 'iterations': 10,
 'password': b'11',
 'salt': b'\xde;1\x87\xac\x0f\xa3\xda\xea\x9a\x9d\xe0R\xc0;+@k\xcd\xcc'
         b'\x11\xb8=\xf2'}

Going to try using the key derivation stuff that comes with Cryptodome (python library I am using) instead of making the Key with stdlib and trying to use it with Cryptodome.

[codeshaunted]

Yeah, I know nothing about hashlib, so you're on your own there. I at first implemented my pack/unpack methods in C# (for consistency with Bloons TD6, made in Unity), and then switched to C++ after. I remember there being some amount of friction when I was attempting to port it as certain configs either didn't exist or were different. If you have discord, my discord is averysumner#3787, I can discuss this with you in more depth there.

[mriswithe]

Sounds like a better place for the discussion! thanks!

[mriswithe]

To the future folk like me if you exist. This is what I was doing wrong. hashlib.pbkdf2_hmac returns a bytes object that is 32 bytes long (in this case). The first 16 bytes are the "Initialization Vector", the second 16 bytes are the "Key"

     @property
    def derived_key(self):
        return hashlib.pbkdf2_hmac(
            "SHA1",
            self._DEFAULT_PASSWORD,
            self.salt,
            self._HASH_ITERATIONS,
            self.dklen,
        )

    @property
    def dklen(self):
        return self._IV_LENGTH + self._KEY_LENGTH

    @property
    def key(self) -> bytes:
        return self.derived_key[16:]

    @property
    def init_vector(self) -> bytes:
        return self.derived_key[:16]

    @cached_property
    def cipher(self):
        return AES.new(self.key, AES.MODE_CBC, iv=self.init_vector)

    @cached_property
    def decrypted_save(self) -> bytes:
        return self.cipher.decrypt(self.encrypted_save)

The reason I didn't catch this was my misreading of the C++. I misread and didn't catch that there was pointer math going on here. when calling decryptor.SetKeyWithIV

  // derive key and iv from salt and password
  CryptoPP::byte derived_key[MONKE_KEY_LENGTH + MONKE_IV_LENGTH];
  CryptoPP::PKCS5_PBKDF2_HMAC<CryptoPP::SHA1> pbkdf2;
  pbkdf2.DeriveKey(derived_key, MONKE_KEY_LENGTH + MONKE_IV_LENGTH, 0,
                   (const CryptoPP::byte*)__password.data(), __password.size(),
                   (const CryptoPP::byte*)salt, sizeof(salt), MONKE_DERIVE_ITERATIONS, 0.0f);

  // set key and iv
  CryptoPP::CBC_Mode<CryptoPP::AES>::Decryption decryptor;
  decryptor.SetKeyWithIV(derived_key + MONKE_IV_LENGTH, MONKE_KEY_LENGTH, derived_key, MONKE_IV_LENGTH);

derived_key is a ptr to a char[32], and it passes in saying to go to the memory address 16 ahead of where the start of the char[32] is and says to read 16 bytes. Essentially the same thing that I am doing with return self.derived_key[16:].
Since I don't know the actual idea of what to do with the crypto stuff already, I didn't already know what to feed where.