I have trained my model using GPT-2 for my dataset. It has been trained and is giving the correct output. Now, I want to train my model on more data while retaining the previously trained model. I do not want to create a new model; I only want to use the previous one with the updated data for training.
I tried doing overwrite=False in the training, but I don’t know whether it will work or not.
TANVI is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
1
Since you are using transformers, you can simply load your fine-tuned model from disk using from_pretrained() with a local path, instead of specifying the model name, which loads the base (non-fine-tuned) model from HuggingFace (or local cache). If you have not saved your fine-tuned model to disk yet, do so using save_pretrained(), and use the same path that you specify here in from_pretrained() to load the model and continue training.
This assumes that you are not using things like accelerate, which has its own mechanism for checkpointing, i.e., saving and loading model weights while training.
8