Cannot generate heatmaps using Gradcam library

[jeev0306]

import numpy as np
import tensorflow as tf
from tf_explain.core.grad_cam import GradCAM
import matplotlib.pyplot as plt

Initialize Grad-CAM explainer

explainer = GradCAM()

Specify the layer for Grad-CAM in the VGG16 model (vgg16_tr)

conv_layer_name = 'block5_conv2' # Adjusted to use 'block5_conv2'

Get a batch from the validation generator

X_val, y_val = val_generator.getitem(0)

Process each image in the batch

for i in range(len(X_val)):
# Add batch dimension to match the expected input shape for Grad-CAM
image_to_explain = np.expand_dims(X_val[i], axis=0) # Shape: (1, 128, 128, 3)
label_to_explain = np.expand_dims(y_val[i], axis=0) # Shape: (1, num_classes)
true_class = np.argmax(y_val[i]) # Get the true class index from the label

# Ensure the image is passed as a tf.Tensor (not a NumPy array)
image_to_explain = tf.convert_to_tensor(image_to_explain, dtype=tf.float32)

# Generate the Grad-CAM heatmap
heatmap = explainer.explain(
    validation_data=(image_to_explain, label_to_explain),  # Pass the single image and its label
    model=vgg16_tr,
    class_index=true_class,
    layer_name=conv_layer_name
)

# Plot the original image and Grad-CAM heatmap
fig, ax = plt.subplots(1, 2, figsize=(10, 5))

# Original image
ax[0].imshow(X_val[i])
ax[0].set_title(f'Original Image {i+1}')
ax[0].axis('off')

# Grad-CAM heatmap overlay
ax[1].imshow(X_val[i])  # Show the original image first
ax[1].imshow(heatmap, cmap='jet', alpha=0.3)  # Overlay heatmap with transparency
ax[1].set_title(f'Grad-CAM Heatmap {i+1}')
ax[1].axis('off')

plt.tight_layout()
plt.show()

I'm trying to generate heatmaps using inbuilt GradCam function for the celebA dataset. But I get the below error. I have reviewed the size to the input going to the GradCam it is (1, 128,128,3).

error: When providing inputs as a list/tuple, all values in the list/tuple must be KerasTensors. Received: inputs=[[<KerasTensor shape=(None, 128, 128, 3), dtype=float32, sparse=None, name=keras_tensor>]] including invalid value [<KerasTensor shape=(None, 128, 128, 3), dtype=float32, sparse=None, name=keras_tensor>] of type <class 'list'>

If anyone could help me debug this, it would be much appriciated.

[fchollet]

This looks like you're using an old version of Keras 3, from before the time when we supported nested model inputs.

[dhantule]

Hi @jeev0306,
Thanks for reporting the issue. Could you please share the version of Keras you are using and also dummy dataset to reproduce this issue?