IRT_XAI

Results of the paper "Explanation-by-Example based on Item Response Theory"

This repository is organized as follows:

decodIRT_OtML_mod.py: Script of the decodIRT tool that was modified to suit the interests of this research.
dataset_properties.csv: File containing the 15 properties of the 4 datasets that were used as a case study.
decodIRT_output: Folder containing all the files generated by the decodIRT tool.
results: Folder containing other results that were generated during the experiments.

In the decodIRT_output directory, there is a folder for each dataset that contains the following files:

dataset_name.csv: The file without suffix indicates that its content is the answer of each of the ML models of the real classifiers and the artificial classifiers.
dataset_name_acuracia.csv: The file with the suffix “_acuracia” means that its content is composed of a table containing the average accuracy of each real classifier, during cross-validation.
dataset_name_final.csv: The file with the suffix “_final” means that its content consists of a table containing the accuracy of the real classifiers on the separate instances for testing.
dataset_name_irt.csv: Just like the file without suffix in the name (dataset_name.csv), this file has an array of responses. However, it does contain a response vector for all real, artificial and RF classifiers.
dataset_name_mlp.csv: Contains the final accuracy that the first set RF's classifiers obtained after the classification.
irt_item_param.csv: Table containing all item parameters (Difficulty, Discrimination and Guessing) generated for the test set instances.
dataset_name_train.csv: Contains the set of dataset instances used to train the models.
dataset_name_test.csv: Contains the set of dataset instances used to test the models.
models: Contains all ML models that were generated, saved in .pkl format.

In the results directory, there is a folder for each dataset that contains the following files:

corrInst_DisNeg.csv: Correlation of features with negative discrimination, considering instances with negative discrimination.
corrInst_Erro.csv: Correlation of features with item parameters, considering the instances that the analyzed model failed.
corrInst_ItemParam.csv: Correlation of features with item parameters, considering all test instances.
heatInst_DisNeg.png: Heatmap of the correlation of features with negative discrimination.
heatInst_error.png: Heatmap of the correlation of features with item parameters, considering the instances that the analyzed model failed.
heatInst_ItemParam.png: Heatmap of the correlation of features with item parameters, considering all test instances.
ItemParam_3D.png: 3D scatter graph of the relationship between dataset instances arranged for each item parameter and divided by class.

lucasferrarocardoso / irt_xai Goto Github PK