I’m trying to build a neural network for classifying images into cats and dogs, I tried different architectures like Resnet50, AlexNet, MobileNetV2 and made my own architecture, as a result there were no problems with accuracy on training data and on validation data, accuracy was around 80-91% on training data data and 75-85% on the validation set, but when I check the accuracy on the test set, it is always at the level of 49-51%. And one day I decided to add another class and the accuracy turned out to be 32-34% on the test set, if I add 5 classes then the accuracy is in the region of 19-21% and the class balance starting from the binary classification of 0.5 then 0.33 and 0.2, respectively.

Checks:
Class weights:
{‘cat’: 1.0, ‘dog’: 1.0}

Class Balance:
{‘cat’: 0.5, ‘dog’: 0.5}

train 4800 validation 1200

Code




    from sklearn.model_selection import train_test_split
    train_df, valid_df = train_test_split(data,
                                    test_size = 0.20,
                                    random_state = 41,
                                    stratify = data['labels'])
    print('train', train_df.shape[0], 'validation', valid_df.shape[0])

    datagen = ImageDataGenerator(rescale = 1./255,
       featurewise_center=False,
       samplewise_center=False,
         rotation_range=90,
         width_shift_range=0.1,
         height_shift_range=0.1,
         horizontal_flip=True,
         vertical_flip=True,
         shear_range = 10
    )
    validation_datagen = ImageDataGenerator(rescale = 1./255,
 
                                        )
    train_generator = datagen.flow_from_dataframe(
     dataframe = train_df,
     directory='/content/images/train/',
    x_col='image',
     y_col='labels',
   target_size=(224,224),
    batch_size=32,
    shuffle=True,
    class_mode = 'binary',
    color_mode = 'rgb'
)

    valid_generator = validation_datagen.flow_from_dataframe(
      dataframe = valid_df,
      directory='/content/images/train/',
       x_col='image',
       y_col='labels',
      target_size=(224,224),
      batch_size=32,
       shuffle=False,
       color_mode = 'rgb',
       class_mode = 'binary'
      )

    backbone = MobileNetV2(input_shape=(224,224, 3),
                     weights='imagenet',
                      include_top=False)
    model = Sequential()
    model.add(backbone)
    model.add(GlobalAveragePooling2D())
    model.add(Dense(100, activation='relu'))
    model.add(Dropout(0.4))
    model.add(Dense(20 activation='relu'))
    model.add(Dropout(0.3))
    model.add(Dense(2, activation='sigmoid'))

    model.compile(optimizer=Adam(lr=0.0001), loss='binary_crossentropy', metrics=['accuracy'])
    model.summary()

    model.fit(train_generator,
           steps_per_epoch=train_generator.samples // 32,
           validation_steps=valid_generator.samples // 32,
          validation_data=valid_generator, epochs=5)
  

I’ve already tried everything and increased lr and Adam changed it to sgd and made 2 neurons in the output layer and removed class_mode or changed it to sparse and changed it to sparse_categorical_crossentropy and different architectures changed and increased the dataset (there were 20,000 images), downloaded and changed different image datasets, everything Equally, the accuracy corresponds to the balance of classes. Maybe I’m processing the images incorrectly or need to be processed somehow. Maybe there is a problem with google colab itself? Because I write the code there