ryansteed / ieat

Statistical test for bias in unsupervised image representations.

Image Embedding Association Test (iEAT)

Ryan Steed, Aylin Caliskan

Read the paper here. View code and data here.

.. info:: iEAT @ FAccT 2021 - Slides - Paper

Installation

# gather the dependencies for running scripts in this repo
conda env update environment.yml
conda activate ieat
# install the weat package locally
pip install -e weat
# install the ieat package locally
pip install -e .

Usage

The ieat package does not have a CLI. Use it programmatically by accessing the API module (ieat.api).

To run a basic test on a set of images, use the test function in ieat.api. SimCLR is downloaded automatically - but you must download a pre-trained version of iGPT yourself.

For an example of how to use the API programmatically, see the documentation and tutorials.

Tutorials and Replications

This repo uses Colab scripts in the notebooks/ directory. Check out notebooks/README.md for a full description.

To open a .ipynb file in Colab, navigate to Colab's Github Interface and search for this repo.

Documentation

Documentation for the ieat API is published at rbsteed.com/ieat.

To generate the documentation, use pdoc3:

pdoc3 --html --output-dir docs --force ieat --template-dir docs/templates
git subtree push --prefix docs/ieat origin gh-pages

Contents

• data/ - images and other data used for bias tests in the paper
• embeddings/ - location for caching computed embeddings - includes pre-computed embeddings for convenience; to generate your own, use the from_cache=False option
• ieat/ - software package for generating image embeddings and testing for bias
• notebooks/ - Colab notebooks containing tutorials and data exploration
• output/ - location for storing results tables
• environment.yml - Conda environment file with dependencies for Jupyter, etc.
• docs/ieat - source for documentation

ieat's Issues

Biases in embeddings extracted by random Resnet

Hi,

I tried your code with a random resnet "torchvision.models.resnet50(pretrained=False)", extracting the embeddings from the AvgPool layer. It still found significant biases in many domains, especially intersectional ones. It found biases even in embeddings extracted by a random MLP with one hidden layer. (see attached screenshot) Do you know why that is the case? Thanks.

Seeking clarification on one-sided p-value from the permutation test

Hi Authors,

Thank you so much for the great work! After reading the paper, I have a question regarding the permutation test that tests the significance of the differential association 𝑠(𝑋, 𝑌, 𝐴, 𝐵). I noticed that in both WEAT and IEAT, the one-sided p-value is calculated as 𝑃𝑟[𝑠(𝑋𝑖, 𝑌𝑖, 𝐴, 𝐵) > 𝑠(𝑋, 𝑌, 𝐴, 𝐵)], with the null hypothesis that the differential association should be 0. However, since this is a one-tailed test, does this assume that the alternative hypothesis is "the differential association is larger than 0"? But I am wondering why the alternative hypothesis is not "the differential association is not equal to 0"? Because the differential association can be in either direction, positive or negative, (if I understand correctly), and only testing the bias in the positive direction may not be complete. May I know the reason that a two-sided test is not used? Thank you!