I am working on developing a deep learning model using Tensorflow and facing some challenges with managing hyperparameters to improve the performance of the model.

My problem is with the correct selection of hyperparameters such as learning rate, batch size and dropout rate.

Then any solution or strategy to achieve the tuning without overfitting?

I wanna automate, so what part should I emphasize in this case?

import tensorflow as tf
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout
from tensorflow.keras.models import Sequential

# Define the model architecture
model = Sequential([
    Conv2D(32, (3, 3), activation='relu', input_shape=(128, 128, 3)),
    MaxPooling2D((2, 2)),
    Conv2D(64, (3, 3), activation='relu'),
    MaxPooling2D((2, 2)),
    Flatten(),
    Dense(128, activation='relu'),
    Dropout(0.2),
    Dense(10, activation='softmax')
])

# Compile the model
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Train the model
history = model.fit(train_images, train_labels, epochs=10, validation_data=(val_images, val_labels))

I tried manual tuning just based on my intuition, but not what I expeccted came out of it.

Also tried grid searching in hyperparameters, and random search for solution.

I want a detailed answer about this evaluation metric improvement system, improvised techniques, automated tuning tools and starting hyperparameter selection decision.