So I am trying to implement SHAP to a CNN model I trained on an image dataset on Kaggle. But I keep getting the following Attribute Error:
<code>---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[9], line 1
----> 1 explainer = shap.DeepExplainer(s_model, val_data)
2 sp_values = explainer.shap_values(val_data)
4 shap.summary_plot(sp_values, val_data)
File /opt/conda/lib/python3.10/site-packages/shap/explainers/_deep/__init__.py:84, in DeepExplainer.__init__(self, model, data, session, learning_phase_flags)
81 framework = 'tensorflow'
83 if framework == 'tensorflow':
---> 84 self.explainer = TFDeep(model, data, session, learning_phase_flags)
85 elif framework == 'pytorch':
86 self.explainer = PyTorchDeep(model, data)
File /opt/conda/lib/python3.10/site-packages/shap/explainers/_deep/deep_tf.py:155, in TFDeep.__init__(self, model, data, session, learning_phase_flags)
153 self.expected_value = None
154 else:
--> 155 if self.data[0].shape[0] > 5000:
156 warnings.warn("You have provided over 5k background samples! For better performance consider using smaller random sample.")
157 if not tf.executing_eagerly():
AttributeError: '_PrefetchDataset' object has no attribute 'shape'
</code>
<code>---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[9], line 1
----> 1 explainer = shap.DeepExplainer(s_model, val_data)
2 sp_values = explainer.shap_values(val_data)
4 shap.summary_plot(sp_values, val_data)
File /opt/conda/lib/python3.10/site-packages/shap/explainers/_deep/__init__.py:84, in DeepExplainer.__init__(self, model, data, session, learning_phase_flags)
81 framework = 'tensorflow'
83 if framework == 'tensorflow':
---> 84 self.explainer = TFDeep(model, data, session, learning_phase_flags)
85 elif framework == 'pytorch':
86 self.explainer = PyTorchDeep(model, data)
File /opt/conda/lib/python3.10/site-packages/shap/explainers/_deep/deep_tf.py:155, in TFDeep.__init__(self, model, data, session, learning_phase_flags)
153 self.expected_value = None
154 else:
--> 155 if self.data[0].shape[0] > 5000:
156 warnings.warn("You have provided over 5k background samples! For better performance consider using smaller random sample.")
157 if not tf.executing_eagerly():
AttributeError: '_PrefetchDataset' object has no attribute 'shape'
</code>
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[9], line 1
----> 1 explainer = shap.DeepExplainer(s_model, val_data)
2 sp_values = explainer.shap_values(val_data)
4 shap.summary_plot(sp_values, val_data)
File /opt/conda/lib/python3.10/site-packages/shap/explainers/_deep/__init__.py:84, in DeepExplainer.__init__(self, model, data, session, learning_phase_flags)
81 framework = 'tensorflow'
83 if framework == 'tensorflow':
---> 84 self.explainer = TFDeep(model, data, session, learning_phase_flags)
85 elif framework == 'pytorch':
86 self.explainer = PyTorchDeep(model, data)
File /opt/conda/lib/python3.10/site-packages/shap/explainers/_deep/deep_tf.py:155, in TFDeep.__init__(self, model, data, session, learning_phase_flags)
153 self.expected_value = None
154 else:
--> 155 if self.data[0].shape[0] > 5000:
156 warnings.warn("You have provided over 5k background samples! For better performance consider using smaller random sample.")
157 if not tf.executing_eagerly():
AttributeError: '_PrefetchDataset' object has no attribute 'shape'
I am using the following code to split the dataset:
<code>from tensorflow.keras.utils import image_dataset_from_directory
batch_size = 32
img_height = 224
img_width = 224
train_data = image_dataset_from_directory(
data_dir,
validation_split=0.3,
subset="training",
seed=123,
image_size=(img_height, img_width),
batch_size=batch_size)
val_data = image_dataset_from_directory(data_dir,
validation_split=0.3,
subset="validation",
seed=123,
image_size=(img_height,img_width),
batch_size=batch_size)
</code>
<code>from tensorflow.keras.utils import image_dataset_from_directory
batch_size = 32
img_height = 224
img_width = 224
train_data = image_dataset_from_directory(
data_dir,
validation_split=0.3,
subset="training",
seed=123,
image_size=(img_height, img_width),
batch_size=batch_size)
val_data = image_dataset_from_directory(data_dir,
validation_split=0.3,
subset="validation",
seed=123,
image_size=(img_height,img_width),
batch_size=batch_size)
</code>
from tensorflow.keras.utils import image_dataset_from_directory
batch_size = 32
img_height = 224
img_width = 224
train_data = image_dataset_from_directory(
data_dir,
validation_split=0.3,
subset="training",
seed=123,
image_size=(img_height, img_width),
batch_size=batch_size)
val_data = image_dataset_from_directory(data_dir,
validation_split=0.3,
subset="validation",
seed=123,
image_size=(img_height,img_width),
batch_size=batch_size)
After loading a previously trained model, I use the following code:
<code>explainer = shap.KernelExplainer(s_model, val_data)
sp_values = explainer.shap_values(val_data)
shap.summary_plot(sp_values, val_data)
</code>
<code>explainer = shap.KernelExplainer(s_model, val_data)
sp_values = explainer.shap_values(val_data)
shap.summary_plot(sp_values, val_data)
</code>
explainer = shap.KernelExplainer(s_model, val_data)
sp_values = explainer.shap_values(val_data)
shap.summary_plot(sp_values, val_data)
This is where I get the error.
I am using Tensorflow and Keras for the code. Can anyone help me solve this? I want to implement SHAP, LIME and any other interpretability method on some image models.
New contributor
B.RATH is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.