I have the following Tensorflow model that I want to quantize:
model = Sequential([
Input(shape=input_shape),
LSTM(lstm_units_1, return_sequences=True),
Dropout(dropout_rate),
LSTM(lstm_units_2, return_sequences=False),
Dropout(dropout_rate),
Dense(4, activation=’softmax’)
])
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
early_stopping = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True)
model_checkpoint = ModelCheckpoint(model_path, monitor='val_loss', save_best_only=True, save_weights_only=False, mode='min')
history = model.fit(X_train, y_train,
epochs=epochs,
batch_size=batch_size,
validation_split=0.2,
callbacks=[early_stopping],
verbose=1)
model.save(model_path)
I am trying to perform the quantization like this:
annotated_model = tfmot.quantization.keras.quantize_annotate_model(model)
with tfmot.quantization.keras.quantize_scope():
quant_aware_model = tfmot.quantization.keras.quantize_apply(annotated_model)
quant_aware_model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
But I am receiving this error: ValueError: `to_annotate` can only be a `keras.Model` instance. Use the `quantize_annotate_layer` API to handle individual layers. You passed an instance of type: Sequential.
Trying to quantize each layer as the error suggest didn’t work for me as well with another value error about LSTM layers not being accepted as inputs.
annotated_model = tf.keras.Sequential([
tfmot.quantization.keras.quantize_annotate_layer(layer)
for layer in model.layers
])
What is the correct way to quantize the particular model I am using here?