I'm using dice with sklearn models and it works great, but when using it with Tensorflow2 I can't get the important features.

Code where it gives error:

       query = pd.DataFrame([self.xts[i]], columns=self.id_list)
        e1 = exp.generate_counterfactuals(
            query,
            total_CFs=10,
            desired_class="opposite", 
            features_to_vary=lst_features,
            verbose=True,
            posthoc_sparsity_algorithm="binary",
        )
        imp = exp.local_feature_importance([self.xts[i]], cf_examples_list=e1.cf_examples_list)

This code works with sklearn but with TF2 I get this error

  ....
  imp = exp.local_feature_importance([self.xts[i]], cf_examples_list=e1.cf_examples_list)
  AttributeError: 'CounterfactualExamples' object has no attribute 'cf_examples_list'

Can important local features be obtained with TF2?

Traceback (most recent call last):
File "G:/nus/Deep_explanation/Dice_baseline.py", line 22, in
dice_exp = exp.generate_counterfactuals(query_instance, total_CFs=4, desired_class="opposite")
File "F:\Anaconda\lib\site-packages\dice_ml\explainer_interfaces\dice_tensorflow1.py", line 139, in generate_counterfactuals
self.do_loss_initializations(yloss_type, diversity_loss_type, feature_weights)
File "F:\Anaconda\lib\site-packages\dice_ml\explainer_interfaces\dice_tensorflow1.py", line 336, in do_loss_initializations
self.yloss = self.compute_yloss(self.yloss_type)
File "F:\Anaconda\lib\site-packages\dice_ml\explainer_interfaces\dice_tensorflow1.py", line 250, in compute_yloss
temp_logits = temp_logits[:, (self.num_ouput_nodes-1):]
AttributeError: 'DiceTensorFlow1' object has no attribute 'num_ouput_nodes'

When I run the example codes in the readme, this error occurs. I dive into the source code of class "DiceTensorFlow1" and its father class "ExplainerBase". They indeed do not have this attribute.

I am attempting to use DiCE on a classification problem and form a DiCE Explanation Instance as such:

# DiCE explanation instance
exp = dice_ml.Dice(d,m)

However, after spending time on this I am running into difficulty in defining m for this purpose.

I have created a neural network configuration and the model trained successfully.

>>> from tensorflow.keras.models import Sequential
>>> from tensorflow.python.keras.layers import Dense
>>> from tensorflow.python.keras.wrappers.scikit_learn import KerasRegressor

>>> sess = tf.InteractiveSession()

>>> # Fitting a dense neural network model
>>> ann_model = Sequential()
>>> ann_model.add(Dense(20, input_shape=(x1.shape[1],), activation=tf.nn.relu))
>>> ann_model.add(Dense(1, activation=tf.nn.sigmoid))
>>> ann_model.compile(loss='binary_crossentropy', metrics=['accuracy'])
>>> ann_model.fit(x1, y, validation_split=0.20, epochs=100)

Epoch 1/100
320/320 [==============================] - 0s 226us/sample - loss: 1.3851 - acc: 0.6750 - val_loss: 1.1140 - val_acc: 0.6375
Epoch 2/100
320/320 [==============================] - 0s 74us/sample - loss: 0.9420 - acc: 0.6969 - val_loss: 0.9910 - val_acc: 0.6375
...

The model is of the type:

tensorflow.python.keras.engine.sequential.Sequential

However, when I attempt to generate the DiCE model:

# Generate the DiCE model for explanation
m = model.Model(model=ann_model)

I get the error:

NameError                                 Traceback (most recent call last)

<ipython-input-15-ec54e1758be4> in <module>()
      1 # Generate the DiCE model for explanation
----> 2 m = model.Model(model=ann_model)

NameError: name 'model' is not defined

For reference, I am using TensorFlow 1.15 and Python 3.6.9. Any help or advice greatly appreciated.

d = dice_ml.Data(dataframe=pipdata, continuous_features=['A','B','C','D','E','F','H','I','J','K'],outcome_name='G')from tensorflow.keras.models import Sequential
from tensorflow.python.keras.layers import Dense
from tensorflow.python.keras.wrappers.scikit_learn import KerasRegressor
import tensorflow.compat.v1 as tf
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
from sklearn.model_selection import train_test_split
sess = tf.InteractiveSession()

Generating train and test data

datasetX = pipdata.drop("G",axis=1)
datasetY = pipdata["G"]

X_train,X_test,Y_train,Y_test = train_test_split(datasetX,datasetY,test_size=0.2,random_state=0)

Fitting a dense neural network model

ann_model = Sequential()
ann_model.add(Dense(6, input_shape=(X_train.shape[1],), activation=tf.nn.relu))
ann_model.add(Dense(1, activation=tf.nn.sigmoid))
ann_model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
history=ann_model.fit(X_train, y_train, validation_split=0.20, epochs=30, verbose=0)
#, class_weight={0:1,1:2})
history
m = dice_ml.Model(model=ann_model,backend='sklearn')

new_d = dice_ml.Data(features={
'A':[-130, 4500],
'B':[-150,7000],
'C':[-54,6200],
'D':[-150,8900],
'E':[-26,1250],
'F':[-26,2545],
'H':[-625,8270],
'I':[-30,1581],
'J':[-135,7364],
'K':[-72,4666]},
outcome_name='G')
exp = dice_ml.Dice(new_d,m)
exp

---> 37 self.precisions = self.data_interface.get_decimal_precisions(output_type="dict")
38 if self.data_interface.outcome_name in self.precisions:
39 self.outcome_precision = [self.precisions[self.data_interface.outcome_name]]

TypeError: get_decimal_precisions() got an unexpected keyword argument 'output_type'

Hi, I notice that the model you have trained is using the one hot encoded training data for categorical features, e.g. for workclass , you have three features to represent it -- workclass_Government, workclass_Other/Unknown, workclass_Private

However, we often train models just using one single feature to represent it with 0 (Government), 1(Other/Unknown), 2(Private). Is there a simple way that I can use my model directly when I am trying to generate the counterfactural example for a certain query instance or I have to convert all of the categorical features into one hot encored format when I am training the model in order to use " exp = dice_ml.Dice". ?

Thanks!

Yan

I have the following query instance, but in the output original outcome is not same as query instance, I assume they should be same otherwise it is useless.

dice_exp = exp.generate_counterfactuals({ 'five_star_rate': 3.5, 'nights_booked': 1.0 }, total_CFs = 4, desired_class="opposite")
dice_exp.visualize_as_dataframe()

Query instance (original outcome : 1)

Diverse Counterfactual set (new outcome : 0)

import dice_ml
from dice_ml.utils import helpers # helper functions

# Dataset for training an ML model
d = dice_ml.Data(dataframe=helpers.load_adult_income_dataset(),
                 continuous_features=['age', 'hours_per_week'],
                 outcome_name='income')
# Pre-trained ML model
m = dice_ml.Model(model_path=dice_ml.utils.helpers.get_adult_income_modelpath(), backend="TF2")

# DiCE explanation instance
exp = dice_ml.Dice(d,m)

query_instance = {'age':22,
    'workclass':'Private',
    'education':'HS-grad',
    'marital_status':'Single',
    'occupation':'Service',
    'race': 'White',
    'gender':'Female',
    'hours_per_week': 45}

# Generate counterfactual examples
dice_exp = exp.generate_counterfactuals(query_instance, total_CFs=4, desired_class=0)
# Visualize counterfactual explanation
dice_exp.visualize_as_dataframe()

Produces:

https://github.com/microsoft/DiCE/blob/072f6f7ac62ba6129e6f8ac4d67fbbe1d7359863/dice_ml/data_interfaces/public_data_interface.py#L192

When the input data has continuous values with sparse input (majority 0), the calculated MAD will be zero. Then, this will cause a problem on:

https://github.com/microsoft/DiCE/blob/072f6f7ac62ba6129e6f8ac4d67fbbe1d7359863/dice_ml/dice_interfaces/dice_tensorflow1.py#L273

Data.one_hot_encode_data() being an alias for pd.get_dummies(df, columns=categorical_features) would one-hot-encode every dataframe using the categories present in df. This is problematic when you need to one-hot-encode a set of counter-factuals (post-hoc) that do not include all categories of a particular feature.

If pos-hoc pre-processing of data is not intended for the Data class, I suggest prefixing the method with "_", or otherwise include the stateful methods (in this case: a method to use the exact scheme that has been used in encoding the train/test data for arbitrary data).

When generating the counterfactual explanation, i'm getting an error related to a feature value which value is considered to be outside of the dataset. This is strange for me, since the variable is a dummy and is not allowed to vary. The variable name is not in the list of feature to vary. My target variable is continous. Perhaps i'm missing something, could someone help, please.

Sincerely yours

Hi,

I hope I can explain my current situation clear enough.

At the moment I've been trying to evaluate multiple model-agnostic methods to generate explanations on a Keras TensorFlow neural network model. Since each XAI method has different requirements and some need to have a numerical encoding to perform the perturbances on categorical data,

The approach that I had implemented to be able to use multiple XAI methods is to have a "numerical encoded" dataset with 18 features (3 numerical and 15 categorical) where categories are encoded as integers. To feed this dataset to the keras NN model I used a column transformer pipeline that performs a ohe-hot-encoding of the 15 categorical variables and a min-max scaler for the numerical variables. Thus, when I call for a prediction i usually include the column transformer

nn.predict(ct.transform(x))

I am trying to initialize the dice explainer however I haven't been able to figure out how I could connect this pipeline step to the model and provide it to dice. If I tried to load the ordinal encoded dataset and provide the NN model, I get an error during initialization stating

"Input 0 of layer dense_104 is incompatible with the layer: expected axis -1 of input shape to have value 66 but received input with shape [1, 65]"

My column transformer takes 18 features and transforms them in 66 columns. I will continue to debug, but I would appreciate if you could let me know if the current design of my pipeline could be integrated to DICE explainer.

Let me know if something is not clear enough and I will try to further explain.

Thanks a lot

Generate counterfactual examples

dice_exp = exp.generate_counterfactuals(query_instance, total_CFs=4, desired_class="opposite")#,proximity_weight=1.5, diversity_weight=1.0)

Visualize counterfactual explanation

dice_exp.visualize_as_dataframe()

--> 116 temp_preds = self.model.get_output(input_instance).numpy()
117 return np.array([preds[(self.num_output_nodes-1):] for preds in temp_preds], dtype=np.float32)
118

AttributeError: 'Tensor' object has no attribute 'numpy'

Sorry if this is a silly question, but I copied the notebook DiCE_with_advanced_options.ipynb and just changed the model to xgboost. When I try to run the line
dice_exp = exp.generate_counterfactuals(query_instance, total_CFs=4, desired_class="opposite")
I get the error in the title. Would you be able to tell me what I'm doing wrong? Thanks

Can I generate more than 4 counterfactual explanations with the Dice package?

All Dice examples have exactly 4 counterfactual explanations via the "total_CFs=4" code.

Changing "total_CFs" to different numbers will cause errors as well.

Thanks

Hi,

I am following the tutorial posted in

When I tried to split the data, I got PublicData object has no attribute 'split_data'. Is this because it has removed in the new version? if yes then how can I split the data. I appreciate your help.

Hi, I was wondering if DiCE works with continuous variables such as time-series data or not.

The MAD is mainly for the heterogeneous features(different features have different scales, ranges).
If you normalize the features with a min-max scaler. all features are mapped into [0,1].

In the adult example, the data interface normalizes the features, why does the defaulted setting is "inverse_mad"? From my understanding, the l2 distance is good. The paper "COUNTERFACTUAL EXPLANATIONS WITHOUT
OPENING THE BLACK BOX: AUTOMATED DECISIONS AND THE GDPR" P18, Equation 5, the authors also suggest the l2 distance.

Do you find any differences between these two kinds of distance ？

I have one question about the loss function in measuring the proximity between generated counterexamples c and input instance x. From your paper, formula (5) and (6) illustrates the distance loss function. However, I understand (6) is not a differentiable function. How do you cope with this problem?

I also check your code in dice_pytorch.py, and it seems like you only measure the distance by calculating the L_1 distance.

Thank you for your time. Looking forward to your reply.

I'd like to enforce constraints on the ranges that each feature can take when being perturbed - for example limiting 'hours_per_week' to anything within [40,60]. Is this currently possible, either directly through the interface or by changing some internals?

I read about it in your paper, but haven't yet found out how to implement this in practice. Relevant excerpt:

(...) to provide constraints on feature manipulation. They can be specified in two ways. First, as box constraints on feasible ranges for each feature, within which CF examples need to be searched."

Thanks in advance!
Siebe

You have based DiCE genetic on GeCo if I understand correctly. Is there a way to 'group' features as they define in their paper;
"The first statement says that education and income are correlated: GeCo will consider only counterfactual values that occur together in the data."
So for instance if we have feature A and B, make sure that the algorithm always generates counterfactuals for which the combination of A=a and B=b exists in the training data.

Hi,

Can DiCE support any regression model? E.g., just specifiy the loss function as the MSE loss?

Thanks

I tried to run DICE with a pretrained XGBoost model and ran into this error. AttributeError: module 'tensorflow' has no attribute 'get_default_session'(). I am using TensorFlow version 2.3.0. Seems to be an issue with the version. Is there a fix for this?

I have a credit scoring dataset with a binary target variable (bad borrower=1, good borrower=0). I have trained a RF, XGB and MLP and then wanted to generate counterfactuals on the exact same observation (where bad borrower=1), for each of the models.

Now, for some reason the MLP outputs "Query instance (original outcome: 0)". The RF and XGB both correctly output "Query instance (original outcome: 1) for the same instance.

I then created a subsample of the dataset with only the bad borrowers and each of them were classified as "Query instance (original outcome : 0)" with the MLP classifier. It seems as if sklearn.neural_network.MLPClassifier() does not work properly with the DiCE framework.

Hi,
The examples you provided for Adult Income work perfectly.
However, when 'capital_gain' and 'capital_loss' are used in addition to 'age' and 'hours_per_week', DiCE has some trouble to deal with those features. I suspect it may have something to do with the fact that they are highly skewed. Did you experience the same thing? Is that the reason why they are not used in the examples you provided? Thanks!

flake8 . --count --max-complexity=10 --max-line-length=127 --statistics
.\dice_ml\data_interfaces\public_data_interface.py:17:5: C901 'PublicData.init' is too complex (18)

Do you have any plans to make DiCE compatible with tensorflow 2.0 and above?
Would be great to have!

Hello! Thank you so much for the great work!

But could you please tell me how to load a trained pytorch models .pth? I always get this error:

AttributeError: '_IncompatibleKeys' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.

Thank you!

There is currently a bug that turns all float features into integers.

Problems causing this:

• An extra empty space in ' float' :
  https://github.com/microsoft/DiCE/blob/bab6166530e846456bc913ce3529ab9aa909df5c/dice_ml/data_interfaces/public_data_interface.py#L69
• Occurrences of self.continuous_feature_name in lines 70 to 74 should be replaced with self.feature
  https://github.com/microsoft/DiCE/blob/bab6166530e846456bc913ce3529ab9aa909df5c/dice_ml/data_interfaces/public_data_interface.py#L70

A reprex:

import dice_ml
from dice_ml.utils import helpers # helper functions
# Dataset for training an ML model
df = helpers.load_adult_income_dataset()
d = dice_ml.Data(dataframe=df,
                         continuous_features=['age', 'hours_per_week'],
                         outcome_name='income')
print(df.head())
print(d.get_decimal_precisions())

df2 = df.copy()
# Make it float
df2['age'] = df2['age'] + 0.1 

d2 = dice_ml.Data(dataframe=df,
                         continuous_features=['age', 'hours_per_week'],
                         outcome_name='income')

print(df2.head())

print(d.get_decimal_precisions())

Hi there

I run into an error when I try to use DiCE with a Keras model that only uses numerical features.
The error I get is IndexError: list index out of range.

From the paper, I understood that DiCE should work with only numerical features as well?
In the code, it seems that when no categorical features are present, still code is called to work with the categorical features which throws the IndexError in public_data_interface.py .

Here is a reproducible example which is adapted from one of your notebooks:

import tensorflow as tf
from tensorflow import keras
import dice_ml
from dice_ml.utils import helpers # helper functions

dataset = helpers.load_adult_income_dataset()
categorical_features = ['workclass', 'education', 'marital_status', 'occupation', 'race', 'gender']
dataset =  dataset.drop(categorical_features, axis=1)

features = dataset.columns
features = [feature for feature in features if feature != 'income']
d = dice_ml.Data(dataframe=dataset, continuous_features=features, outcome_name='income')

sess = tf.InteractiveSession()
train, _ = d.split_data(d.normalize_data(d.one_hot_encoded_data))
X_train = train.loc[:, train.columns != 'income']
y_train = train.loc[:, train.columns == 'income']

ann_model = keras.Sequential()
ann_model.add(keras.layers.Dense(20, input_shape=(X_train.shape[1],), kernel_regularizer=keras.regularizers.l1(0.001), activation=tf.nn.relu))
ann_model.add(keras.layers.Dense(1, activation=tf.nn.sigmoid))

ann_model.compile(loss='binary_crossentropy', optimizer=tf.keras.optimizers.Adam(0.01), metrics=['accuracy'])
ann_model.fit(X_train, y_train, validation_split=0.20, epochs=1, verbose=1,
class_weight={0:1,1:2})
query_instance = {'age':22,
                  'hours_per_week': 45}
m = dice_ml.Model(model=ann_model)
exp = dice_ml.Dice(d, m)
dice_exp = exp.generate_counterfactuals(query_instance, total_CFs=1, desired_class="opposite")

Hi, I have the three questions.

Firstly, I get this error when I try to use 'desired_range'.

Secondly, sometimes permitted_range cannot ensure the controlled feature is in the range. For instance, the second image shows that the age is not in the range of 23 and 24.

Lastly, it seems that I have to set the range all for continuous features when using 'permitted_range'. For instance, in the second image I have to control another continuous feature 'hours_per_week' although I only want to control the 'age' feature.
Otherwise, I will get a key error because I only control one continuous feature-'age'.
How can I only control one continuous feature by DICE?

Thank you very much

import dice_ml
May i know how to install, is there pip install?

flake8 . --count --max-complexity=10 --max-line-length=127 --statistics
.\dice_ml\data_interfaces\private_data_interface.py:16:5: C901 'PrivateData.init' is too complex (11)

Hi again,
I understand you must be extremely busy. I appreciate your time.

I was now trying to use the same ANN model as in your notebook, but with the German credit data. The model trains fine. But I get the following error message when I try to run the line:
dice_exp = exp.generate_counterfactuals(query_instance, total_CFs=4, desired_class=1)

I have copier the entire error message below. Thanks

divide by zero encountered in double_scalars

InvalidArgumentError Traceback (most recent call last)
in
1 # generate counterfactuals
----> 2 dice_exp = exp.generate_counterfactuals(query_instance, total_CFs=4, desired_class=1)

~\Anaconda3\lib\site-packages\dice_ml\dice_interfaces\dice_tensorflow2.py in generate_counterfactuals(self, query_instance, total_CFs, desired_class, proximity_weight, diversity_weight, categorical_penalty, algorithm, features_to_vary, yloss_type, diversity_loss_type, feature_weights, optimizer, learning_rate, min_iter, max_iter, project_iter, loss_diff_thres, loss_converge_maxiter, verbose, init_near_query_instance, tie_random, stopping_threshold, posthoc_sparsity_param)
92 self.update_hyperparameters(proximity_weight, diversity_weight, categorical_penalty)
93
---> 94 query_instance, test_pred = self.find_counterfactuals(query_instance, desired_class, optimizer, learning_rate, min_iter, max_iter, project_iter, loss_diff_thres, loss_converge_maxiter, verbose, init_near_query_instance, tie_random, stopping_threshold, posthoc_sparsity_param)
95
96 return exp.CounterfactualExamples(self.data_interface, query_instance,

~\Anaconda3\lib\site-packages\dice_ml\dice_interfaces\dice_tensorflow2.py in find_counterfactuals(self, query_instance, desired_class, optimizer, learning_rate, min_iter, max_iter, project_iter, loss_diff_thres, loss_converge_maxiter, verbose, init_near_query_instance, tie_random, stopping_threshold, posthoc_sparsity_param)
418
419 # get gradients
--> 420 grads = tape.gradient(loss_value, self.cfs)
421
422 # freeze features other than feat_to_vary_idxs

~\Anaconda3\lib\site-packages\tensorflow_core\python\eager\backprop.py in gradient(self, target, sources, output_gradients, unconnected_gradients)
1027 output_gradients=output_gradients,
1028 sources_raw=flat_sources_raw,
-> 1029 unconnected_gradients=unconnected_gradients)
1030
1031 if not self._persistent:

~\Anaconda3\lib\site-packages\tensorflow_core\python\eager\imperative_grad.py in imperative_grad(tape, target, sources, output_gradients, sources_raw, unconnected_gradients)
75 output_gradients,
76 sources_raw,
---> 77 compat.as_str(unconnected_gradients.value))

~\Anaconda3\lib\site-packages\tensorflow_core\python\eager\backprop.py in _gradient_function(op_name, attr_tuple, num_inputs, inputs, outputs, out_grads, skip_input_indices)
139 return [None] * num_inputs
140
--> 141 return grad_fn(mock_op, *out_grads)
142
143

~\Anaconda3\lib\site-packages\tensorflow_core\python\ops\linalg_grad.py in _MatrixDeterminantGrad(op, grad)
362 a = op.inputs[0]
363 c = op.outputs[0]
--> 364 a_adj_inv = linalg_ops.matrix_inverse(a, adjoint=True)
365 multipliers = array_ops.reshape(grad * c,
366 array_ops.concat([array_ops.shape(c), [1, 1]],

~\Anaconda3\lib\site-packages\tensorflow_core\python\ops\gen_linalg_ops.py in matrix_inverse(input, adjoint, name)
1401 raise
1402 except _core._NotOkStatusException as e:
-> 1403 _ops.raise_from_not_ok_status(e, name)
1404 # Add nodes to the TensorFlow graph.
1405 if adjoint is None:

~\Anaconda3\lib\site-packages\tensorflow_core\python\framework\ops.py in raise_from_not_ok_status(e, name)
6604 message = e.message + (" name: " + name if name is not None else "")
6605 # pylint: disable=protected-access
-> 6606 six.raise_from(core._status_to_exception(e.code, message), None)
6607 # pylint: enable=protected-access
6608

~\Anaconda3\lib\site-packages\six.py in raise_from(value, from_value)

InvalidArgumentError: Input is not invertible. [Op:MatrixInverse]

The post-hoc sparsity enhancement can enter in an infinite loop in certain conditions (I tested with all numerical data, with data ranging, mostly, from -1 to 1). If you need, I can attach here the data. The cause is, for small numbers, sometimes the rate of change is very small or even zero, then, it stucks in this step.

I think one possible solution is to check the difference between the new diff and old difference old_diff, something like:

        if (old_diff == diff) or ((old_diff-diff) < 1e-5):
            n_tolerance += 1
        else:
            n_tolerance = 0

and adding another stop condition on the while loop: n_tolerance <= max_tolerance

Hello everyone,

I am looking for a way to export the generated examples to something like a pandas-dataframe.
Maybe, I am missing something but I tried to do so for sometime now but could not find a solution.
I think and hope that this is somehow possible. The output generated with the ".visualize_as_dataframe()" looks already like a dataframe but I do not see a way to save the displayed examples.
I also tried the "to_json()"-method. However, I could no figure out yet, how to get from there to a dataframe.

I am grateful for any suggestion.
Best regards, Arnim

Hi, I'm trying to write the dice_ml object to a file and I'm getting the follow error:

backend = 'TF'+tf.__version__[0]
dice_model = dice_ml.Model(model=model, backend=backend)

exp = dice_ml.Dice(d, dice_model)

from joblib import load, dump

with open("explainer_dice.joblib", "wb") as f:
    dump(exp, f, compress="lz4")

TypeError: cannot pickle '_thread.RLock' object

exp = Dice(d, m, method="random")
query_instance = x_train[1:2]
e1 = exp.generate_counterfactuals(query_instance, total_CFs=0, desired_range=None,desired_class="opposite",
permitted_range=None, features_to_vary="all")
e1.visualize_as_dataframe(show_only_changes=True)

ValueError Traceback (most recent call last)
in
2 query_instance = x_train[1:2]
3 e1 = exp.generate_counterfactuals(query_instance, total_CFs=0, desired_range=None,desired_class="opposite",
----> 4 permitted_range=None, features_to_vary="all")
5 e1.visualize_as_dataframe(show_only_changes=True)

c:\users\gaugup\documents\github\dice\dice_ml\explainer_interfaces\explainer_base.py in generate_counterfactuals(self, query_instances, total_CFs, desired_class, desired_range, permitted_range, features_to_vary, stopping_threshold, posthoc_sparsity_param, posthoc_sparsity_algorithm, verbose, **kwargs)
87 posthoc_sparsity_algorithm=posthoc_sparsity_algorithm,
88 verbose=verbose,
---> 89 **kwargs)
90 cf_examples_arr.append(res)
91 return CounterfactualExplanations(cf_examples_list=cf_examples_arr)

c:\users\gaugup\documents\github\dice\dice_ml\explainer_interfaces\dice_random.py in _generate_counterfactuals(self, query_instance, total_CFs, desired_range, desired_class, permitted_range, features_to_vary, stopping_threshold, posthoc_sparsity_param, posthoc_sparsity_algorithm, sample_size, random_seed, verbose)
136 cfs_df = cfs_df.sample(total_CFs)
137 cfs_df.reset_index(inplace=True, drop=True)
--> 138 self.cfs_pred_scores = self.predict_fn(cfs_df)
139 cfs_df[self.data_interface.outcome_name] = self.get_model_output_from_scores(self.cfs_pred_scores)
140

c:\users\gaugup\documents\github\dice\dice_ml\explainer_interfaces\dice_random.py in predict_fn(self, input_instance)
225 def predict_fn(self, input_instance):
226 """prediction function"""
--> 227 return self.model.get_output(input_instance)

c:\users\gaugup\documents\github\dice\dice_ml\model_interfaces\base_model.py in get_output(self, input_instance)
38 input_instance = self.transformer.transform(input_instance)
39 if self.model_type == "classifier":
---> 40 return self.model.predict_proba(input_instance)
41 else:
42 return self.model.predict(input_instance)

~\Anaconda3\envs\env1\lib\site-packages\sklearn\utils\metaestimators.py in (*args, **kwargs)
118
119 # lambda, but not partial, allows help() to work with update_wrapper
--> 120 out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
121 # update the docstring of the returned function
122 update_wrapper(out, self.fn)

~\Anaconda3\envs\env1\lib\site-packages\sklearn\pipeline.py in predict_proba(self, X)
472 Xt = X
473 for _, name, transform in self._iter(with_final=False):
--> 474 Xt = transform.transform(Xt)
475 return self.steps[-1][-1].predict_proba(Xt)
476

~\Anaconda3\envs\env1\lib\site-packages\sklearn\compose_column_transformer.py in transform(self, X)
563 "data given during fit."
564 )
--> 565 Xs = self._fit_transform(X, None, _transform_one, fitted=True)
566 self._validate_output(Xs)
567

~\Anaconda3\envs\env1\lib\site-packages\sklearn\compose_column_transformer.py in _fit_transform(self, X, y, func, fitted)
442 message=self._log_message(name, idx, len(transformers)))
443 for idx, (name, trans, column, weight) in enumerate(
--> 444 self._iter(fitted=fitted, replace_strings=True), 1))
445 except ValueError as e:
446 if "Expected 2D array, got 1D array instead" in str(e):

~\Anaconda3\envs\env1\lib\site-packages\joblib\parallel.py in call(self, iterable)
1027 # remaining jobs.
1028 self._iterating = False
-> 1029 if self.dispatch_one_batch(iterator):
1030 self._iterating = self._original_iterator is not None
1031

~\Anaconda3\envs\env1\lib\site-packages\joblib\parallel.py in dispatch_one_batch(self, iterator)
845 return False
846 else:
--> 847 self._dispatch(tasks)
848 return True
849

~\Anaconda3\envs\env1\lib\site-packages\joblib\parallel.py in _dispatch(self, batch)
763 with self._lock:
764 job_idx = len(self._jobs)
--> 765 job = self._backend.apply_async(batch, callback=cb)
766 # A job can complete so quickly than its callback is
767 # called before we get here, causing self._jobs to

~\Anaconda3\envs\env1\lib\site-packages\joblib_parallel_backends.py in apply_async(self, func, callback)
204 def apply_async(self, func, callback=None):
205 """Schedule a func to be run"""
--> 206 result = ImmediateResult(func)
207 if callback:
208 callback(result)

~\Anaconda3\envs\env1\lib\site-packages\joblib_parallel_backends.py in init(self, batch)
568 # Don't delay the application, to avoid keeping the input
569 # arguments in memory
--> 570 self.results = batch()
571
572 def get(self):

~\Anaconda3\envs\env1\lib\site-packages\joblib\parallel.py in call(self)
251 with parallel_backend(self._backend, n_jobs=self._n_jobs):
252 return [func(*args, **kwargs)
--> 253 for func, args, kwargs in self.items]
254
255 def reduce(self):

~\Anaconda3\envs\env1\lib\site-packages\joblib\parallel.py in (.0)
251 with parallel_backend(self._backend, n_jobs=self._n_jobs):
252 return [func(*args, **kwargs)
--> 253 for func, args, kwargs in self.items]
254
255 def reduce(self):

~\Anaconda3\envs\env1\lib\site-packages\sklearn\utils\fixes.py in call(self, *args, **kwargs)
220 def call(self, *args, **kwargs):
221 with config_context(**self.config):
--> 222 return self.function(*args, **kwargs)

~\Anaconda3\envs\env1\lib\site-packages\sklearn\pipeline.py in _transform_one(transformer, X, y, weight, **fit_params)
731
732 def _transform_one(transformer, X, y, weight, **fit_params):
--> 733 res = transformer.transform(X)
734 # if we have a weight for this transformer, multiply output
735 if weight is None:

~\Anaconda3\envs\env1\lib\site-packages\sklearn\pipeline.py in _transform(self, X)
558 Xt = X
559 for _, _, transform in self._iter():
--> 560 Xt = transform.transform(Xt)
561 return Xt
562

~\Anaconda3\envs\env1\lib\site-packages\sklearn\preprocessing_data.py in transform(self, X, copy)
884 accept_sparse='csr', copy=copy,
885 estimator=self, dtype=FLOAT_DTYPES,
--> 886 force_all_finite='allow-nan')
887
888 if sparse.issparse(X):

~\Anaconda3\envs\env1\lib\site-packages\sklearn\base.py in _validate_data(self, X, y, reset, validate_separately, **check_params)
419 out = X
420 elif isinstance(y, str) and y == 'no_validation':
--> 421 X = check_array(X, **check_params)
422 out = X
423 else:

~\Anaconda3\envs\env1\lib\site-packages\sklearn\utils\validation.py in inner_f(*args, **kwargs)
61 extra_args = len(args) - len(all_args)
62 if extra_args <= 0:
---> 63 return f(*args, **kwargs)
64
65 # extra_args > 0

~\Anaconda3\envs\env1\lib\site-packages\sklearn\utils\validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator)
670 " minimum of %d is required%s."
671 % (n_samples, array.shape, ensure_min_samples,
--> 672 context))
673
674 if ensure_min_features > 0 and array.ndim == 2:

ValueError: Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required by StandardScaler.

Testing with other Dice explainers, kdtree/genetic and it seems they handle this scenario much more gracefully.

I'm having an issue with the version of DiCE installed from pip.
Here's the exact traceback:

File "CounterfactualDriver.py", line 71, in <module>
    dice_exp = exp.generate_counterfactuals(query_instance, total_CFs=1, desired_class="opposite")
  File "~/miniconda3/envs/python_env/lib/python3.6/site-packages/dice_ml/dice_interfaces/dice_pytorch.py", line 97, in generate_counterfactuals
    query_instance, test_pred = self.find_counterfactuals(query_instance, desired_class, optimizer, learning_rate, min_iter, max_iter, project_iter, loss_diff_thres, loss_converge_maxiter, verbose, init_near_query_instance, tie_random, stopping_threshold, posthoc_sparsity_param)
  File "~/miniconda3/envs/python_env/lib/python3.6/site-packages/dice_ml/dice_interfaces/dice_pytorch.py", line 367, in find_counterfactuals
    desired_class = 1.0 - round(test_pred)
TypeError: type numpy.ndarray doesn't define __round__ method

Here is my environment setup:

dice-ml=0.2
numpy = 1.19.1
pandas = 1.1.1
scikit-learn = 0.23.2
tensorflow=2.3.0
torch=1.6.0
torchvision=0.7.0
h5py=2.10.0

Do you have any advice on how to move past this error? Are my package versions of the requirements for dice-ml correct?

Thanks so much!

I am trying to find CF 5 examples on twodatasets: iris, wine. However, sometimes it takes more time for iris than wine (25sec VS 3sec). Moreover, the time spent on iris also fluctuates. The iris dataset owns lessfeatures, which I expect to be fast. So, what is the relationship of explanation time and feature number (and query number and CF examples number). There are 5 queries and each query is required to find 5 CF examples.

While running
d = dice_ml.Data(dataframe=dataset, continuous_features=['age', 'hours_per_week'], outcome_name='income')
I get an error

---> 25 from scipy.misc import comb
     26 from ..utils import indexable, check_random_state, safe_indexing
     27 from ..utils.validation import _num_samples, column_or_1d

ImportError: cannot import name 'comb'

This has been depreciated
from scipy.misc import comb

update needed

from scipy.special import comb
https://docs.scipy.org/doc/scipy-1.2.1/reference/generated/scipy.misc.comb.html

Hi,
Thanks a lot for the wonderful library. As you guys have mentioned in the roadmap, can you tell us by when we can expect the integration of scikit-learn like libraries. I am facing issues when using trying to integrate the same.
Thanks in advance!

If the category in some column in query_instance is not present in the training instance, then dice_genetic.py and dice_KD.py validate and throw an exception. But dice_random.py doesn't throw this exception. Specifically, the function

Installed dice-ml from pypi.

import dice_ml
from dice_ml.utils import helpers # helper functions

d = dice_ml.Data(dataframe=helpers.load_adult_income_dataset(),
                 continuous_features=['age', 'hours_per_week'],
                 outcome_name='income')

m = dice_ml.Model(model_path=dice_ml.utils.helpers.get_adult_income_modelpath())

exp = dice_ml.Dice(d,m)

Getting the following error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-8-d3dad7eed1c9> in <module>
      8 m = dice_ml.Model(model_path=dice_ml.utils.helpers.get_adult_income_modelpath())
      9 # DiCE explanation instance
---> 10 exp = dice_ml.Dice(d,m)

~/anaconda3/envs/diceml/lib/python3.6/site-packages/dice_ml/dice.py in __init__(self, data_interface, model_interface, **kwargs)
     15         """
     16 
---> 17         self.decide_implementation_type(data_interface, model_interface, **kwargs)
     18 
     19     def decide_implementation_type(self, data_interface, model_interface, **kwargs):

~/anaconda3/envs/diceml/lib/python3.6/site-packages/dice_ml/dice.py in decide_implementation_type(self, data_interface, model_interface, **kwargs)
     21 
     22         self.__class__  = decide(data_interface, model_interface)
---> 23         self.__init__(data_interface, model_interface, **kwargs)
     24 
     25 # To add new implementations of DiCE, add the class in explainer_interfaces subpackage and import-and-return the class in an elif loop as shown in the below method.

~/anaconda3/envs/diceml/lib/python3.6/site-packages/dice_ml/explainer_interfaces/dice_tensorflow1.py in __init__(self, data_interface, model_interface)
     26 
     27         # create TensorFLow session if one is not already created
---> 28         if tf.get_default_session() is not None:
     29             self.dice_sess = tf.get_default_session()
     30         else:

AttributeError: module 'tensorflow' has no attribute 'get_default_session'

My pip freeze result

absl-py==0.11.0
appnope==0.1.0
argon2-cffi==20.1.0
astunparse==1.6.3
async-generator==1.10
attrs==20.2.0
backcall==0.2.0
bleach==3.2.1
cachetools==4.1.1
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
dataclasses==0.7
decorator==4.4.2
defusedxml==0.6.0
dice-ml==0.4
entrypoints==0.3
future==0.18.2
gast==0.3.3
google-auth==1.22.1
google-auth-oauthlib==0.4.2
google-pasta==0.2.0
grpcio==1.33.2
h5py==2.10.0
idna==2.10
importlib-metadata==2.0.0
ipykernel==5.3.4
ipython==7.16.1
ipython-genutils==0.2.0
ipywidgets==7.5.1
jedi==0.17.2
Jinja2==2.11.2
joblib==0.17.0
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==6.1.7
jupyter-console==6.2.0
jupyter-core==4.6.3
jupyterlab-pygments==0.1.2
Keras-Preprocessing==1.1.2
lazy-import==0.2.2
Markdown==3.3.3
MarkupSafe==1.1.1
mistune==0.8.4
mkl-fft==1.2.0
mkl-random==1.1.1
mkl-service==2.3.0
nbclient==0.5.1
nbconvert==6.0.7
nbformat==5.0.8
nest-asyncio==1.4.2
notebook==6.1.4
numpy @ file:///opt/concourse/worker/volumes/live/e6a5a904-ea21-4841-4eec-d53e1ac1014c/volume/numpy_and_numpy_base_1603479626870/work
oauthlib==3.1.0
olefile==0.46
opt-einsum==3.3.0
packaging==20.4
pandas==0.25.3
pandocfilters==1.4.3
parso==0.7.1
pexpect==4.8.0
pickleshare==0.7.5
Pillow @ file:///opt/concourse/worker/volumes/live/06069510-e277-4aed-54f4-6dfdcb84a461/volume/pillow_1603822272490/work
prometheus-client==0.8.0
prompt-toolkit==3.0.3
protobuf==3.13.0
ptyprocess==0.6.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
Pygments==2.7.2
pyparsing==2.4.7
pyrsistent==0.17.3
python-dateutil==2.8.1
pytz==2020.1
pyzmq==19.0.2
qtconsole==4.7.7
QtPy==1.9.0
requests==2.24.0
requests-oauthlib==1.3.0
rsa==4.6
scikit-learn==0.23.2
scipy==1.5.3
Send2Trash==1.5.0
six==1.15.0
tensorboard==2.3.0
tensorboard-plugin-wit==1.7.0
tensorflow==2.3.1
tensorflow-cpu==2.3.1
tensorflow-estimator==2.3.0
tensorflow-gpu==1.1.0
termcolor==1.1.0
terminado==0.9.1
testpath==0.4.4
threadpoolctl==2.1.0
torch==1.7.0
torchaudio==0.7.0a0+ac17b64
torchvision==0.8.1
tornado==6.0.4
traitlets==4.3.3
typing-extensions @ file:///tmp/build/80754af9/typing_extensions_1598376058250/work
urllib3==1.25.11
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wrapt==1.12.1
zipp==3.4.0

If I change torch or tensorflow version other issues crop up. .

If fresh dice-ml install done in a new env where dice only install torch and tensorflow, then the error is
AttributeError: module 'torch.jit' has no attribute '_script_if_tracing'

Hello,

I would appreciate some help when handling unknown categories values in new data predicted with the DICE explainer predict_fn method.

Well, we have implemented the DICE technique to be used in our ANN model trained with our historical data, which contains categorical features. Once we tested our model and tried to predict new data containing unknown categories values in categorical features using explainer predict_fn we had some issues.

You may find following a code example demonstrating our problem.

Have you had these kind of problems before?

import dice_ml
from dice_ml.utils import helpers # helper functions

# Tensorflow libraries
import pandas as pd
import numpy as np
import tensorflow as tf
from tensorflow import keras

dataset = helpers.load_adult_income_dataset()


d = dice_ml.Data(dataframe=dataset, continuous_features=['age', 'hours_per_week'], outcome_name='income')

train, _ = d.split_data(d.normalize_data(d.one_hot_encoded_data))

X_train = train.loc[:, train.columns != 'income']
y_train = train.loc[:, train.columns == 'income']

ann_model = keras.Sequential()
ann_model.add(keras.layers.Dense(20, input_shape=(X_train.shape[1],), kernel_regularizer=keras.regularizers.l1(0.001), activation=tf.nn.relu))
ann_model.add(keras.layers.Dense(1, activation=tf.nn.sigmoid))

ann_model.compile(loss='binary_crossentropy', optimizer=tf.keras.optimizers.Adam(0.01), metrics=['accuracy'])
ann_model.fit(X_train, y_train, validation_split=0.20, epochs=2, verbose=0, class_weight={0:1,1:2})
# the training will take some time for 100 epochs.

backend = 'TF'+tf.__version__[0] # TF2
model = dice_ml.Model(model=ann_model, backend=backend)

exp = dice_ml.Dice(d, model)

# Real world new data
new_data = pd.DataFrame({'age': 40, 'workclass': 'Private', 'education': 'Bachelors','marital_status': 'Married',
                        'occupation': 'Business', 'race': 'other', 'gender': 'Male',
                         'hours_per_week': 30}, index=[32562])

query_instance  = d.prepare_query_instance(query_instance=dict(new_data.iloc[0]), encode=True)
print(query_instance.shape)

query_instance  = np.array([query_instance.iloc[0].values])

exp.predict_fn(tf.constant(query_instance, dtype=tf.float32))[0][0]

InvalidArgumentError: Matrix size-incompatible: In[0]: [1,31], In[1]: [29,20] [Op:MatMul]

We know you use pd.get_dummies in your implementation for categorical features, we also know that OneHotEncoder has the handle_unknown attribute that when set to ignore would solve this problem.

from sklearn.preprocessing import OneHotEncoder

category = ['workclass', 'education', 'marital_status', 'occupation', 'race', 'gender']
aux = []
for c in category:
    aux.append(list(dataset[c].unique()))

enc = OneHotEncoder(categories=aux, handle_unknown="ignore", sparse=False)
enc.fit(dataset[category])

enc.transform(new_data[category])

Do you have any other suggestions to help us?

Hi all, I was going through the paper and find it really interesting. I was exploring the DICE code and I was not able to figure out which configurations were used in the FAT20 paper. What would be the appropriate steps for me to reproduce the dpp based method described in the paper (with the corresponding numbers). Any help in this regard would be appreciated.

Thanks,
Prateek

Hello,

I'm not sure if this is something I'm doing wrong or you've encountered this before. Consider the following actions:

In the adult dataset I put aside a validation set, which neither my model nor DiCE has seen. From those, I select a number of samples whose age = 31, and use those to generate counterfactuals by varying everything other than age. Other than this I am not using any weights.
After that, I use my model to get a prediction on these generated counterfactuals. In a lot of the cases my model's prediction is not what DiCE thinks it would give.

Is there any way for me to increase DiCE's fidelity to my model?

Thanks

Hi,

I have solved this problem by myself.

Thanks very much.

flake8 . --count --max-complexity=10 --max-line-length=127 --statistics
.\dice_ml\explainer_interfaces\explainer_base.py:521:5: C901 'ExplainerBase.is_cf_valid' is too complex (11)

Hey there,
I'm trying to create an exp like this example
exp = dice_ml.Dice(data,m)
I'm using # XGBClassifier and # LogisticRegression as prediction model
but all the time I encounter in this error :
AttributeError: 'LogisticRegression' object has no attribute 'backend'
AttributeError: 'XGBClassifier ' object has no attribute 'backend'

Anybody can help me please?I will appreciate a lot!
Thanks
Hen