I manually “compile” the model, but I was unable to get the model weights when using a custom callback.

According to the documentation, model.compile() configures the model for training. I’m wondering if I manually set all the configurations and also defined my own loss function, do I still need to compile the model?

First, I defined the loss and gradient steps:

def loss(model, x_batch, y_batch, training):
    y_pred_batch = model(x_batch, training=training)
    ...
    ...
    # A penalty term is calculated using both x_batch and y_batch in the current batch
    penalty = ...
    ...
    ...
    return tf.keras.losses.MSE(y_true=y_batch, y_pred=y_pred_batch) + penalty

def grad(model, inputs, targets):
  with tf.GradientTape() as tape:
    loss_value = loss(model, inputs, targets, training=True)
  return loss_value, tape.gradient(loss_value, model.trainable_variables)

After these, assuming my model is a Sequential() model called model, and the hyperparameters epochs, learning_rate, and optimizer are all defined. Training, validation, and test data are stored in train_dataset, val_dataset, test_dataset respectively.

I begin training using

logs = {}

# stop if val_loss is not improving for 10 epochs
early_stopping_callback = tf.keras.callbacks.EarlyStopping(
        monitor='val_loss', patience=10, restore_best_weights=True)

# 300 epochs in total
for epoch in range(epochs):
    batch_train_loss, batch_val_loss = [], []

    # iterate through the training data
    for x_batch, y_batch in train_dataset:
        loss_value, grads = grad(model, x_batch, y_batch)
        optimizer.apply_gradients(zip(grads, model.trainable_variables))
        batch_train_loss.append(loss_value)

    for x_batch, y_batch in val_dataset:
        batch_loss = loss(model, x_batch, y_batch, training=False)
        batch_val_loss.append(batch_loss)
        
    # calculate a single loss value for the current batch
    train_loss = tf.reduce_mean(batch_train_loss)
    val_loss = tf.reduce_mean(batch_val_loss)

    # Log losses
    logs['train_loss'] = train_loss
    logs['val_loss'] = val_loss

    print(f'Epoch {epoch}/{epochs}: train_loss: {train_loss.numpy():0.3f}, ', end=' ')
    print(f'val_loss: {val_loss.numpy():0.3f}')

I wanted to use a early stopping callback that monitors the validation loss for each epoch.

# after printing the loss for each epoch
early_stopping_callback.on_epoch_end(epoch, logs=logs) 
if early_stopping_callback.model.stop_training:
    break 

However, I got the following error

---> 53 early_stopping_callback.on_epoch_end(epoch, logs=logs)
-> 1790   self.best_weights = self.model.get_weights()
AttributeError: 'NoneType' object has no attribute 'get_weights'

After some research, it seems like the model is not properly compiled and the callback is not properly linked to the model, but I’m not sure how to fix it. Is the logging processing causing this error?