charmlab / mace Goto Github PK
View Code? Open in Web Editor NEWModel Agnostic Counterfactual Explanations
Model Agnostic Counterfactual Explanations
Hello Amir, thanks for the amazing work!
Installing the package went like a breeze, yet I am facing a few errors when running some of the suggested examples:
Running 'python batchTest.py -d credit -m mlp -n one_norm -a MACE_eps_1e-3 -b 0 -s 1'
In this case, I got an assertion error due to insufficient accuracy. Does not seem a big issue, perhaps some bad luck during training.
Running ''python batchTest.py -d credit -m forest -n one_norm -a MACE_eps_1e-3 -b 0 -s 1'
In this case, I get the following error: "ValueError: Number of features of the model must match the input. Model n_features is 20 and input n_features is 14". After some analysis, I have the impression that this error occurs due to two "ordinal" attributes in the data set that seems to trigger a one-hot encoding, and later some inconsistencies between different data structures.
Running ''python batchTest.py -d credit -m forest -n one_norm -a FT -b 0 -s 1'
This seems to lead to an error that is similar to the previous test.
Am-I correctly using the code, and is there a way to fix this issue?
Thanks a lot!
--Thibaut
Hi, I am trying to use this library and have followed the README but I'm getting this error when I use the command:
python batchTest.py -d credit -m mlp -n one_norm -a MACE_eps_1e-3 -b 0 -s 1
I've attached the error below.
Do you have any advice to solve this? Do I need to load the data and model in some way?
Also, does your MACE (or MINT) algorithm work well for gradient boosting models?
Thanks so much.
I tried to execute the command: python batchTest.py -d adult -m lr -n zero_norm -a AR -b 0 -s 1
However, it resulted in the following error trace:
[ENV WARNING] process_synthetic_data not available
[ENV WARNING] deactivate virtualenv to allow for testing Actionable Recourse
Experimenting with dataset_string = adult
Experimenting with model_class_string = lr
Experimenting with norm_type_string = zero_norm
Experimenting with approach_string = AR
Looking for file adult.data in the /home/t-dimaha/Mace/mace/_data_main directory..
File found in current directory..
/home/t-dimaha/Mace/mace/_data_main/adult.data
Looking for file adult.test in the /home/t-dimaha/Mace/mace/_data_main directory..
File found in current directory..
/home/t-dimaha/Mace/mace/_data_main/adult.test
Replacing column WorkClass with {WorkClass_cat_0, WorkClass_cat_1, WorkClass_cat_2, WorkClass_cat_3, WorkClass_cat_4, WorkClass_cat_5, WorkClass_cat_6}
Replacing column EducationLevel with {EducationLevel_ord_0, EducationLevel_ord_1, EducationLevel_ord_2, EducationLevel_ord_3, EducationLevel_ord_4, EducationLevel_ord_5, EducationLevel_ord_6, EducationLevel_ord_7, EducationLevel_ord_8, EducationLevel_ord_9}
Replacing column MaritalStatus with {MaritalStatus_cat_0, MaritalStatus_cat_1, MaritalStatus_cat_2, MaritalStatus_cat_3, MaritalStatus_cat_4, MaritalStatus_cat_5, MaritalStatus_cat_6}
Replacing column Occupation with {Occupation_cat_0, Occupation_cat_1, Occupation_cat_2, Occupation_cat_3, Occupation_cat_4, Occupation_cat_5, Occupation_cat_6, Occupation_cat_7, Occupation_cat_8, Occupation_cat_9, Occupation_cat_10, Occupation_cat_11, Occupation_cat_12, Occupation_cat_13}
Replacing column Relationship with {Relationship_cat_0, Relationship_cat_1, Relationship_cat_2, Relationship_cat_3, Relationship_cat_4, Relationship_cat_5}
Looking for file adult.data in the /home/t-dimaha/Mace/mace/_data_main directory..
File found in current directory..
/home/t-dimaha/Mace/mace/_data_main/adult.data
Looking for file adult.test in the /home/t-dimaha/Mace/mace/_data_main directory..
File found in current directory..
/home/t-dimaha/Mace/mace/_data_main/adult.test
Replacing column WorkClass with {WorkClass_cat_0, WorkClass_cat_1, WorkClass_cat_2, WorkClass_cat_3, WorkClass_cat_4, WorkClass_cat_5, WorkClass_cat_6}
Replacing column EducationLevel with {EducationLevel_ord_0, EducationLevel_ord_1, EducationLevel_ord_2, EducationLevel_ord_3, EducationLevel_ord_4, EducationLevel_ord_5, EducationLevel_ord_6, EducationLevel_ord_7, EducationLevel_ord_8, EducationLevel_ord_9}
Replacing column MaritalStatus with {MaritalStatus_cat_0, MaritalStatus_cat_1, MaritalStatus_cat_2, MaritalStatus_cat_3, MaritalStatus_cat_4, MaritalStatus_cat_5, MaritalStatus_cat_6}
Replacing column Occupation with {Occupation_cat_0, Occupation_cat_1, Occupation_cat_2, Occupation_cat_3, Occupation_cat_4, Occupation_cat_5, Occupation_cat_6, Occupation_cat_7, Occupation_cat_8, Occupation_cat_9, Occupation_cat_10, Occupation_cat_11, Occupation_cat_12, Occupation_cat_13}
Replacing column Relationship with {Relationship_cat_0, Relationship_cat_1, Relationship_cat_2, Relationship_cat_3, Relationship_cat_4, Relationship_cat_5}
[INFO] Training lr
on 15,399 samples (%70.00 of 22,000 samples)...
Training accuracy: %80.51
Testing accuracy: %79.76
[INFO] done.
Generating explanation for batch #0 sample #1/1 (sample index 17148): Traceback (most recent call last):
File "batchTest.py", line 346, in
args.process_id)
File "batchTest.py", line 243, in runExperiments
standard_deviations, # used solely for feature_tweaking method
File "batchTest.py", line 118, in generateExplanations
return generateARExplanations.genExp(
NameError: name 'generateARExplanations' is not defined
I tried to execute method AR on adult dataset with one norm/infinity norm using the command:
python batchTest.py -d credit -m lr -n infty_norm -a AR -b 0 -s 1
It resulted in the following error:
Generating explanation for batch #0 sample #1/1 (sample index 17148): Traceback (most recent call last):
File "batchTest.py", line 346, in
args.process_id)
File "batchTest.py", line 243, in runExperiments
standard_deviations, # used solely for feature_tweaking method
File "batchTest.py", line 122, in generateExplanations
dataset_obj
File "/home/t-dimaha/Mace/mace/generateARExplanations.py", line 75, in genExp
mip_cost_type=mip_cost_type
File "/home/t-dimaha/Mace/mace/mace_envs_2/lib/python3.7/site-packages/recourse/builder.py", line 632, in init
super().init(action_set = action_set, x = x, **kwargs)
File "/home/t-dimaha/Mace/mace/mace_envs_2/lib/python3.7/site-packages/recourse/builder.py", line 87, in init
self.x = x
File "/home/t-dimaha/Mace/mace/mace_envs_2/lib/python3.7/site-packages/recourse/builder.py", line 180, in x
assert len(x) == self.n_variables
AssertionError
The same issue occurs across different datasets, which seems to arise from the 'mip_cost_type' parameter value in the call to recourse.builder.RecourseBuilder().
Thanks for this library! I see you support LR and MLP models from sklearn currently, but can you give some pointers on how to make it work with PyTorch or TensorFlow models?
I checked generateSATExplanations.py
and found that you convert a model to a logic formula in modelConversion.py
, but do so only for four model classes from sklearn. How to replicate this for any model from say PyTorch? In particular, how to update mlp2formula(model, model_symbols)
method?
Hi @amirhk , I would like to use one of the CF explanation methodologies in the Repo. I'm using MACE, but I see that the CF generation is model-dependent. Exist in the repo a CF generation methodology (AR, MINT) that is model agnostic or is there a way to generate an agnostic "getModelFormula" in MACE?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.