iamgroot42 / mimir

Python package for measuring memorization in LLMs.

MIMIR

MIMIR - Python package for measuring memorization in LLMs.

Documentation is available here.

Instructions

First install the python dependencies

pip install -r requirements.txt

Then, install our package

pip install -e .

To use, run the scripts in scripts/bash

Note: Intermediate results are saved in tmp_results/ and tmp_results_cross/ for bash scripts. If your experiment completes successfully, the results will be moved into the results/ and results_cross/ directory.

Setting environment variables

You can either provide the following environment variables, or pass them via your config/CLI:

MIMIR_CACHE_PATH: Path to cache directory
MIMIR_DATA_SOURCE: Path to data directory

Using cached data

The data we used for our experiments is available on Hugging Face Datasets. You can either choose to either load the data directly from Hugging Face with the load_from_hf flag in the config (preferred), or download the cache_100_200_.... folders into your MIMIR_CACHE_PATH directory.

MIA experiments how to run

python run.py --config configs/mi.json

Attacks

We include and implement the following attacks, as described in our paper.

• Likelihood (loss). Works by simply using the likelihood of the target datapoint as score.
• Reference-based (ref). Normalizes likelihood score with score obtained from a reference model.
• Zlib Entropy (zlib). Uses the zlib compression size of a sample to approximate local difficulty of sample.
• Neighborhood (ne). Generates neighbors using auxiliary model and measures change in likelihood.
• Min-K% Prob (min_k). Uses k% of tokens with minimum likelihood for score computation.
• Min-K%++ (min_k++). Uses k% of tokens with minimum normalized likelihood for score computation.
• Gradient Norm (gradnorm). Uses gradient norm of the target datapoint as score.

Adding your own dataset

To extend the package for your own dataset, you can directly load your data inside load_cached() in data_utils.py, or add an additional if-else within load() in data_utils.py if it cannot be loaded from memory (or some source) easily. We will probably add a more general way to do this in the future.

Adding your own attack

To add an attack, create a file for your attack (e.g. attacks/my_attack.py) and implement the interface described in attacks/all_attacks.py. Then, add a name for your attack to the dictionary in attacks/utils.py.

If you would like to submit your attack to the repository, please open a pull request describing your attack and the paper it is based on.

Citation

If you use MIMIR in your research, please cite our paper:

@article{duan2024membership,
      title={Do Membership Inference Attacks Work on Large Language Models?}, 
      author={Michael Duan and Anshuman Suri and Niloofar Mireshghallah and Sewon Min and Weijia Shi and Luke Zettlemoyer and Yulia Tsvetkov and Yejin Choi and David Evans and Hannaneh Hajishirzi},
      year={2024},
      journal={arXiv:2402.07841},
}

mimir's Issues

Some issue in model.py

Hi,

I found a strange line here in your code - https://github.com/iamgroot42/mimir/blob/main/mimir/models.py#L183

Is this a bug or is there any other particular reason? It is currently overriding what I am specifying in the config file.

Thanks.

Some issues in configs/mi.json

Hi,
I found your project to be helpful to my current paper, but I met some problems when I was trying python run.py --config configs/mi.json.

Problems:

1. An error reports split is Nonetype, which can be fixed by adjusting the mi.json as:
   "dataset_member": "the_pile_pile_cc_ngram_13_0.2",
   "dataset_nonmember": "the_pile_pile_cc_ngram_13_0.2",

2. An error reports there is no dataset key, which can be fixed by adding this line to the mi.json:
   "dataset_key": "the_pile",

I don't whether my adjustments to mi.json are correct. If it's not, could you please give me some suggestions? If these are correct, I suggest adding some tutorial about config files in the github project.

Thanks,
Ethan

Step by step guide/documentation?

Hi,

Thanks for sharing your code. I want to run the attacks you implemented on my own datasets, but I'm not sure where to start. Is there a guide or documentation the describes this?

Best,
Shane

Types of gradients computed by GradNormAttack

Hello,
Thanks for your valuable work on mimir!

If I understand correctly, GradNormAttack is computing the average (across layers) of the gradient norm wrt. the model weights.

mimir/mimir/attacks/gradnorm.py

Line 41 in 6c61109

grad_norms.append(param.grad.detach().norm(p))

But the docstring indicates that the gradients are computed w.r.t. input tokens.

mimir/mimir/attacks/gradnorm.py

Line 18 in 6c61109

Gradient Norm Attack. Computes p-norm of gradients w.r.t. input tokens.

Since the original paper proposes both, I think there are two solutions:

• Simply fixing the docstring and keeping the current implementation
• Or implementing both gradients norms. I guess that computing gradients wrt input tokens would require modifying Model.get_probabilities()

The results in Appendix C.1 suggest that in certain settings, one gradient type outperforms the other, while in other settings, the reverse is observed.

What do you think?

dataset_nonmember_key: str missing from mimir/config.py

Hi,

Just checked out the code, dataset_nonmember_key: str is missing from the definition of class ExperimentConfig(Serializable):
in mimir/config.py.

Best,
John

Original dataset?

Hi,

Thank you for the great work! I was not able to find the original Pile subcategory dataset (arXiv, Github etc.) in the huggingface data repo. There are only the processed ones (7-gram, 13-gram). Could you share the original ones as well?

Thank you!