I am having issues with transfer learning the Mobilenetv2 for image classification of food items. I had initially used resnet but was facing the same overfitting issue but mobilenet runs faster so i decided to stick with that. I’ve posted code snippits below:
RELEVENT CODE:

# Create an ImageDataGenerator with data augmentation for training
data_generator = keras.preprocessing.image.ImageDataGenerator(
    preprocessing_function=keras.applications.mobilenet.preprocess_input,
    # rescale=1./255,
    validation_split=0.3,
    rotation_range=30,  # Random rotations
    width_shift_range=0.2,  # Horizontal shifts
    height_shift_range=0.2,  # Vertical shifts
    shear_range=0.2,  # Shear transformations
    zoom_range=0.2,  # Zoom
    horizontal_flip=True,  # Horizontal flips
)

# Load and preprocess training data
train_data_flow = data_generator.flow_from_directory(
    dataset_path,
    target_size=(128, 128),  # Resize images to 224x224
    batch_size=32,
    class_mode='categorical',
    subset='training'  # Use training subset
)



# Load and preprocess validation data
val_data_flow = data_generator.flow_from_directory(
    dataset_path,
    target_size=(128, 128),  # Resize images to 224x224
    batch_size=32,
    class_mode='categorical',
    subset='validation'  # Use validation subset
)

# Load the model from TensorFlow Hub
model_url = "https://www.kaggle.com/models/google/mobilenet-v2/TensorFlow2/035-128-classification/2"
hub_layer = hub.KerasLayer(model_url, input_shape=(128, 128, 3) ,  trainable=True, arguments=dict(batch_norm_momentum=0.997))

Creating the model:

# Create a Sequential model with dropout and batch normalization
model = keras.Sequential([
    hub_layer,
    layers.Dropout(0.2),  # Lower dropout rate
    layers.Dense(256, activation='relu'),
    layers.BatchNormalization(),  # Batch normalization
    layers.Dropout(0.2),  # Dropout
    layers.Dense(9, activation='softmax')
])

# Build the Sequential model
model.build((None, 128, 128, 3))

# Summary of the model
model.summary()

Fitting the model:

# Define the EarlyStopping callback
early_stopping_callback = keras.callbacks.EarlyStopping(
    monitor='val_loss',  # Monitor validation loss
    patience=20,          # Number of epochs with no improvement after which training will be stopped
    restore_best_weights=True  # Restore model weights from the epoch with the best value of the monitored quantity
)

# Fit the model with early stopping
history = model.fit(
    train_data_flow,
    validation_data=val_data_flow,
    epochs=100,
    callbacks=[early_stopping_callback],  # Pass the EarlyStopping callback as a list
    shuffle= True
)

Typical case of overfitting. I’ve tried data augmentation, different splits for training and validation , shuffling etc. Any Ideas?

Here are some of the epochs:

Epoch 1/100
113/113 [==============================] - 73s 641ms/step - loss: 0.5302 - accuracy: 0.8623 - val_loss: 3.2597 - val_accuracy: 0.4723
Epoch 2/100
113/113 [==============================] - 70s 618ms/step - loss: 0.4798 - accuracy: 0.8751 - val_loss: 2.4527 - val_accuracy: 0.6070
Epoch 3/100
113/113 [==============================] - 70s 621ms/step - loss: 0.4891 - accuracy: 0.8695 - val_loss: 2.0448 - val_accuracy: 0.6018
Epoch 4/100
113/113 [==============================] - 70s 624ms/step - loss: 0.4618 - accuracy: 0.8851 - val_loss: 2.0285 - val_accuracy: 0.5764
Epoch 5/100
113/113 [==============================] - 70s 618ms/step - loss: 0.4320 - accuracy: 0.8948 - val_loss: 1.7363 - val_accuracy: 0.6344
Epoch 6/100
113/113 [==============================] - 70s 619ms/step - loss: 0.3853 - accuracy: 0.9051 - val_loss: 1.9907 - val_accuracy: 0.5901
Epoch 7/100
113/113 [==============================] - 70s 618ms/step - loss: 0.4188 - accuracy: 0.8909 - val_loss: 2.0851 - val_accuracy: 0.5777
Epoch 8/100
113/113 [==============================] - 71s 625ms/step - loss: 0.3877 - accuracy: 0.9062 - val_loss: 1.4624 - val_accuracy: 0.6630
Epoch 9/100
113/113 [==============================] - 70s 621ms/step - loss: 0.3589 - accuracy: 0.9115 - val_loss: 1.6237 - val_accuracy: 0.6376
Epoch 10/100
113/113 [==============================] - 70s 618ms/step - loss: 0.3782 - accuracy: 0.8998 - val_loss: 1.6615 - val_accuracy: 0.6207
Epoch 11/100
113/113 [==============================] - 70s 621ms/step - loss: 0.3714 - accuracy: 0.9040 - val_loss: 2.5165 - val_accuracy: 0.4782
Epoch 12/100
113/113 [==============================] - 71s 629ms/step - loss: 0.3514 - accuracy: 0.9149 - val_loss: 1.9595 - val_accuracy: 0.5901
Epoch 13/100
...
113/113 [==============================] - 76s 669ms/step - loss: 0.2445 - accuracy: 0.9466 - val_loss: 1.3978 - val_accuracy: 0.6630
Epoch 44/100
113/113 [==============================] - 71s 632ms/step - loss: 0.2272 - accuracy: 0.9527 - val_loss: 1.3801 - val_accuracy: 0.6760

FOR PREDICITONS:

from sklearn.metrics import classification_report


predictions = model.predict(val_data_flow)

# Convert predictions to class labels
predicted_classes = np.argmax(predictions, axis=1)

# Step 4: Evaluate the Model

# Get the true labels from the test data
true_labels = val_data_flow.classes  

# Calculate accuracy
accuracy = np.mean(predicted_classes == true_labels)  

# Accuracy as the score
print("Model Accuracy on Test Set:", accuracy)

# sklearn's classification_report for more detailed metrics
print("Classification Report:")
print(classification_report(true_labels, predicted_classes, target_names=list(val_data_flow.class_indices.keys())))

Results:

Model Accuracy on Test Set: 0.19388418998048146
Classification Report:
              precision    recall  f1-score   support

           1       0.32      0.30      0.31       495
           2       0.00      0.00      0.00        21
           3       0.17      0.21      0.19       199
           4       0.13      0.09      0.11       154
           5       0.00      0.00      0.00         8
           6       0.11      0.06      0.08        32
           7       0.11      0.07      0.08       199
           8       0.16      0.19      0.17       231
           9       0.13      0.18      0.15       198

    accuracy                           0.19      1537
   macro avg       0.12      0.12      0.12      1537
weighted avg       0.19      0.19      0.19      1537